April 2016 - dovecot - dovecot.org

Released Pigeonhole v0.4.14 for Dovecot v2.2.24.
by Stephan Bosch 27 Apr '16

27 Apr '16

Hello Dovecot users, The most important feature that this new release adds is the IMAPSIEVE capability, which allows executing Sieve scripts at IMAP events. It also adds the ability to obtain trace logs from Sieve execution at actual delivery (or from IMAP), rather than only using the sieve-test command line tool. This release also fixes a bug in the Sieve extprograms plugin that has been bothering a few people for a while now. Changelog v0.4.14: * The address test now allows specifying the X-Original-To header. + Implemented the Sieve imapsieve extension and its IMAP counterpart (RFC 6785) as a set of plugins. This allows running Sieve scripts at IMAP activity, rather than at delivery. There are also facilities for the familiar sieve_before/sieve_after administrator scripts. A user script is defined for a mailbox using an IMAP METADATA entry, whereas administrator scripts are configured using mailbox matching rules defined in the plugin settings. + Adjusted the Sieve ihave extension to allow capability tests to be performed at runtime. This way, scripts can be written that work both at delivery and from IMAP. + Implemented support for runtime trace debugging. This means that detailed information about which commands, actions and tests are performed is written to a file. That file is created in the configured directory, but only if that directory exists. This way, a particular user can be easily singled out for debugging. This works much like the Dovecot rawlog facility. The trace output is identical to what is produced using sieve-test with its "-t" command line option. + Added a "sieve_user_email" setting that configures the user's primary email address. This is mainly useful to have a user email address available in IMAP, where envelope data is unavailable. + Implemented the dovecot-specific "vnd.dovecot.report" extension. This allows sending report messages in the Message Abuse Reporting Format (RFC 5965). - extprograms plugin: Fixed epoll() panic caused by closing the output FD before the output stream. - Made sure that the local part of a mail address is encoded properly using quoted string syntax when it is not a dot-atom. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.14.ta… http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.14.ta… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

1 0

help with sieve
by dovecot＠paulkoppen.com 27 Apr '16

27 Apr '16

Dear Dovecot / Pigeonhole community, I have a very specific issue, but trying to solve this for a full day, I am now getting so bold from pulling my hairs out, I thought it was time I asked for help. I hope this is a good place for that. At my hosting provider, who have been extremely helpful by the way but are not specifically skilled in Sieve, I have a catch-all address set up: *(a)domain.org (where * is literal), with a sieve script to control how messages are forwarded. Currently my rules are of the form: if address :localpart :is ["To","Cc","Bcc"] ["name1", "name2", ...] { ... } This works most of the time, but sometimes the messages are not directly addressed to me and so they slip through the filters. I figured that using envelope would solve this, so I require envelope and then do: if envelope :localpart :is "To" ["name1", "name2", ...] { ... } But this fails. After endlessly trying different combinations I figured out that the reason for failure is that the envelope believes the "To" is actually "*(a)domain.org", so it is never going to match any of the names. However, the following test succeeds: if header :contains "Envelope-To" "name1@" { ... } From this I deduce that a previous script has already done the conversion from name1 to *, and at the point where my sieve script is run the envelope picks up the *(a)domain.org address, leaving the Envelope-To header for what it is. But I am new to this. Is that likely to be the case? Is it a common sieve set-up? And most importantly, is there a better way around the issue than testing header :contains? With kind regards, Paul

2 1

v2.2.24 released
by Timo Sirainen 26 Apr '16

26 Apr '16

http://dovecot.org/releases/2.2/dovecot-2.2.24.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.24.tar.gz.sig This should be a good release. :) * doveconf now warns if it sees a global setting being changed when the same setting was already set inside some filters. (A common mistake has been adding more plugins to a global mail_plugins setting after it was already set inside protocol { .. }, which caused the global setting to be ignored for that protocol.) * LMTP proxy: Increased default timeout 30s -> 125s. This makes it less likely to reach the timeout and cause duplicate deliveries. * LMTP and indexer now append ":suffix" to session IDs to make it unique for the specific user's delivery. (Fixes duplicate session ID warnings in stats process.) + Added dict-ldap for performing read-only LDAP dict lookups. + lazy-expunge: All mails can be saved to a single specified mailbox. + mailbox { autoexpunge } supports now wildcards in mailbox names. + doveadm HTTP API: Added support for proxy commands + imapc: Reconnect when getting disconnected in non-selected state. + imapc: Added imapc_features=modseq to access MODSEQs/HIGHESTMODSEQ. This is especially useful for incremental dsync. + doveadm auth/user: Auth lookup performs debug logging if -o auth_debug=yes is given to doveadm. + Added passdb/userdb { auth_verbose=yes|no } setting. + Cassandra: Added user, password, num_threads, connect_timeout and request_timeout settings. + doveadm user -e <value>: Print <value> with %variables expanded. - Huge header lines could have caused Dovecot to use too much memory (depending on config and used IMAP commands). (Typically this would result in only the single user's process dying with out of memory due to reaching service { vsz_limit } - not a global DoS). - dsync: Detect and handle invalid/stale -s state string better. - dsync: Fixed crash caused by specific mailbox renames - auth: Auth cache is now disabled passwd-file. It was unnecessary and it broke %variables in extra fields. - fts-tika: Don't crash if it returns 500 error - dict-redis: Fixed timeout handling - SEARCH INTHREAD was crashing - stats: Only a single fifo_listeners was supported, making it impossible to use both auth_stats=yes and mail stats plugin. - SSL errors were logged in separate "Stacked error" log lines instead of as part of the disconnection reason. - MIME body parser didn't handle properly when a child MIME part's --boundary had the same prefix as the parent.

2 2

Subject: Re: v2.2.24 released
by Joseph Tam 26 Apr '16

26 Apr '16

Timo Sirainen wrote: > - auth: Auth cache is now disabled passwd-file. It was unnecessary and > it broke %variables in extra fields. So settings like auth_cache_size = 65536 auth_cache_ttl = 86400 auth_cache_negative_ttl = 600 are ignored/useless, and passwd-file auth lookups always results in file read operation? Joseph Tam <jtam.home(a)gmail.com>

2 2

Solr with shared mailboxes
by Ricardo Branco 26 Apr '16

26 Apr '16

We have multiple logins assigned to single mailboxes, so we can give one login to different devices. Does this cause a problem with mailbox guids in solr or is the ID stored within the indexes and not related to username.

2 3

doveadm search when using solr problems
by Ricardo Branco 26 Apr '16

26 Apr '16

Is there a way when using doveadm to search a users mailbox to disable using solr plugin (without changing config file). We have found that solr does not work well with certain search types and reverting to non solr searching works much better.

2 3

Possible to use Doveadm for a time range deletion
by Kenneth Ulrich 26 Apr '16

26 Apr '16

Hello All, I was wondering if it s possible to remove emails between a time range. For example, End Users have emails dating back to Aug 2000. He would like all emails deleted between the time stamp of 07/2014 to 07/2015 yet keep any emails outside of this area. I've ran into some similar examples, but nothing definite.

2 1

Strange search behavior
by Cornelius Charlie 26 Apr '16

26 Apr '16

Hello, I experienced a strange behavior in the way dovecot is performing searches, I only experienced this exact issue with the flag "seen". If I perform this command: UID SORT (SIZE) UTF-8 FLAGGED SEEN Dovecot return every seen messages even if the message is not "flagged'. This is only happening when there is no unseen messages in the mailbox, if there is at least one, this is working properly. Any though ? Thanks.

3 4

push-notification plugin and imap-metadata permissions
by Giovanni S. Fois 26 Apr '16

26 Apr '16

Ultra short version: Why cant I set the following mailbox metadata key? /private/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/http-notify Let me explain the context: I'm using the Dovecot version 2.23.1, but the same happens for the 2.2.22 The push-notification plugin is supposed to send out a notification whenever a mailbox get a new email message. In order to tell if a mailbox is enabled to send out the notification, the plugin looks out for the following mailbox metadata key: /private/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/http-notify If the key is set then everything is OK and the notification is sent, otherwise the action is skipped. If I try to setup the metadata key by hand (telnet as the user over the imap port): setmetadata INBOX (/private/vendor/vendor.dovecot/pvt/server/vendor/vendor.dovecot/http-notify "user=myuser(a)mydomain.com") I get the error message: "Internal mailbox attributes cannot be accessed" Digging in the Dovecot 2.2.22 sources i found that: This string is from lib-imap-storage/imap-metadata.c - line 36 - Dovecot 2.2.22 The message is triggered by the following condition - same file - line 125 - Dovecot 2.2.22 if (strncmp(*key_r, MAILBOX_ATTRIBUTE_PREFIX_DOVECOT_PVT, strlen(MAILBOX_ATTRIBUTE_PREFIX_DOVECOT_PVT)) == 0) { So the path pvt/server appears to be forbidden. But, in the file lib-storage/mailbox-attribute.h we can read the following comment: /* User can get/set all non-pvt/ attributes and also pvt/server/ (but not pvt/server/pvt/) attributes. */ And, after said comment there is the definition of the macro MAILBOX_ATTRIBUTE_KEY_IS_USER_ACCESSIBLE(key) which has the same basic function of the condition in imap-metadata.c , but in this case the same imap key is seen as accessible. Now my questions: Can we use a negated version of MAILBOX_ATTRIBUTE_KEY_IS_USER_ACCESSIBLE(key) in imap-metadata? How can the push-notification plugin work out-of-the-box without changes and recompilation? Thank you for your valuable time and forgive me if I'm posing a dumb question. Best wishes, Giovanni S. Fois

4 6

Kernel panic in dovecot-ee-lmtp on Debian 8
by Tobi 26 Apr '16

26 Apr '16

Hi list I just realized that that I don not receive all mails in my mailbox (running dovecot-ee 2.2.23.1-1 on Debian 8). On my frontend servers (running postfix) the queue fills up with mails that cannot be delivered via lmtp to my backend servers. The error message on the frontend is "lost connection with backend while sending data" When I checked the logs on the backend server I found upon every delivery attempt kernel panics Apr 25 12:33:36 mbox1 dovecot: lmtp(REDACTED): Panic: epoll_ctl(del, 18) failed: Bad file descriptor Apr 25 12:33:36 mbox1 dovecot: lmtp(REDACTED): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x85dee) [0x7faa19225dee] -> /usr/lib/dovecot/libdovecot.so.0(+0x85edc) [0x7faa19225edc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7faa191ca5be] -> /usr/lib/dovecot/libdovecot.so.0(+0x9afc1) [0x7faa1923afc1] -> /usr/lib/dovecot/libdovecot.so.0(+0x994af) [0x7faa192394af] -> /usr/lib/dovecot/libdovecot.so.0(+0xa5eda) [0x7faa19245eda] -> /usr/lib/dovecot/libdovecot.so.0(+0xa4566) [0x7faa19244566] -> /usr/lib/dovecot/libdovecot.so.0(o_stream_destroy+0xe) [0x7faa192446ee] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x7e44b) [0x7faa17d3a44b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x7ea14) [0x7faa17d3aa14] -> /usr/lib/dovecot/libdovecot.so.0(+0xa6d88) [0x7faa19246d88] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7faa19239c6c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7faa1923b0d1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7faa19239cf5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7faa19239e98] -> /usr/lib/dovecot/libdovecot-sieve.so.0(program_client_run+0xe8) [0x7faa17d3b308] -> /usr/lib/dovecot/modules/sieve/lib90_sieve_extprograms_plugin.so(+0x45b8) [0x7faa15d855b8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0x7c) [0x7faa17cfa2ac] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7faa17cfa4cb] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x519e0) [0x7faa17d0d9e0] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7faa17d0e567] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b6b) [0x7faa17f73b6b] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) [0x7faa197e5849] -> dovecot/lmtp() [0x4064b3] -> dovecot/lmtp() [0x406cb4] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7faa19239c6c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7faa1923b0d1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7faa19239cf5] Apr 25 12:33:36 mbox1 dovecot: lmtp(REDACTED): Fatal: master: service(lmtp): child 32081 killed with signal 6 (core dumps disabled) the weird thing is that this is not concerning all incoming mails. From time to time the frontends can deliver messages to my backends. I just tested it with a mail from work. It arrived via a frontend that has other mails in queue that could not be delivered so far. Does anyone have an idea where I could look for the root cause of this panic messages? Thanks a lot tobi

4 5