- dovecot - dovecot.org

v2.2.28 released
by Timo Sirainen 10 Mar '17

10 Mar '17

http://dovecot.org/releases/2.2/dovecot-2.2.28.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.28.tar.gz.sig * director: "doveadm director move" to same host now refreshes user's timeout. This allows keeping user constantly in the same backend by just periodically moving the user there. * When new mailbox is created, use initially INBOX's dovecot.index.cache caching decisions. * Expunging mails writes GUID to dovecot.index.log now only if the GUID is quickly available from index/cache. * pop3c: Increase timeout for PASS command to 5 minutes. * Mail access errors are no longer ignored when searching or sorting. With IMAP the untagged SEARCH/SORT reply is still sent the same as before, but NO reply is returned instead of OK. + Make dovecot.list.index's filename configurable. This is needed when there are multiple namespaces pointing to the same mail root (e.g. lazy_expunge namespace for mdbox). + Add size.virtual to dovecot.index when folder vsizes are accessed (e.g. quota=count). This is mainly a workaround to avoid slow quota recalculation performance when message sizes get lost from dovecot.index.cache due to corruption or some other reason. + auth: Support OAUTHBEARER and XOAUTH2 mechanisms. Also support them in lib-dsasl for client side. + auth: Support filtering by SASL mechanism: passdb { mechanisms } + Shrink the mail processes' memory usage by not storing settings duplicated unnecessarily many times. + imap: Add imap_fetch_failure setting to control what happens when FETCH fails for some mails (see example-config). + imap: Include info about last command in disconnection log line. + imap: Created new SEARCH=X-MIMEPART extension. It's currently not advertised by default, since it's not fully implemented. + fts-solr: Add support for basic authentication. + Cassandra: Support automatically retrying failed queries if execution_retry_interval and execution_retry_times are set. + doveadm: Added "mailbox path" command. + mail_log plugin: If plugin { mail_log_cached_only=yes }, log the wanted fields only if it doesn't require opening the email. + mail_vsize_bg_after_count setting added (see example-config). + mail_sort_max_read_count setting added (see example-config). + pop3c: Added pop3c_features=no-pipelining setting to prevent using PIPELINING extension even though it's advertised. - Index files: day_first_uid wasn't updated correctly since v2.2.26. This caused dovecot.index.cache to be non-optimal. - imap: SEARCH/SORT may have assert-crashed in client_check_command_hangs - imap: FETCH X-MAILBOX may have assert-crashed in virtual mailboxes. - imap: Running time in tagged command reply was often wrongly 0. - search: Using NOT n:* or NOT UID n:* wasn't handled correctly - director: doveadm director kick was broken - director: Fix crash when using director_flush_socket - director: Fix some bugs when moving users between backends - imapc: Various error handling fixes and improvements - master: doveadm process status output had a lot of duplicates. - autoexpunge: If mailbox's rename timestamp is newer than mail's save-timestamp, use it instead. This is useful when autoexpunging e.g. Trash/* and an entire mailbox is deleted by renaming it under Trash to prevent it from being autoexpunged too early. - autoexpunge: Multiple processes may have been trying to expunge the same mails simultaneously. This was problematic especially with lazy_expunge plugin. - auth: %{passdb:*} was empty in auth-worker processes - auth-policy: hashed_password was always sent empty. - dict-sql: Merge multiple UPDATEs to a single statement if possible. - fts-solr: Escape {} chars when sending queries - fts: fts_autoindex_exclude = \Special-use caused crashes - doveadm-server: Fix leaks and other problems when process is reused for multiple requests (service_count != 1) - sdbox: Fix assert-crash on mailbox create race - lda/lmtp: deliver_log_format values weren't entirely correct if Sieve was used. especially %{storage_id} was broken. - lmtp_user_concurrency_limit didn't work if userdb changed username

6 7

Maildirsize not updated
by chaouche yacine 08 Mar '17

08 Mar '17

Hello dovecot Problem : maildirsize and dovecot quota get have different values dovecot version : 2.2.13 happens with : some mailboxes This issue happens only to some mailboxes and only for a specific period of time until maildirsize gets updated by something (not me). I can't reproduce this now because it happened this morning and now its 2 PM and the problem is gone (maildirsize was updated). Anybody experienced the same situation ? -- Yassine.

3 6

iOS Mail app and rapid authenticate / disconnect on Dovecot proxy
by Robert Giles 07 Mar '17

07 Mar '17

Hi folks, I have a handful of iOS 10.2.1 Mail app IMAP clients that intermittently break into this unexplained authenticate-then-immediately-disconnect behavior when connecting to a RHEL7 Dovecot (dovecot-2.2.10-7.el7) proxy, providing proxied connections to a backend Panda/UW-IMAP server. From talking to the users, the activity would appear to be spontaneous (ie: not caused by user interaction with the device). The behavior doesn't seem to have any observable implications for the end user, other than momentarily hitting the Dovecot process_limit (which, if not raised to a rather large number, disrupts new IMAP proxy connections momentarily). I reckon this is not an issue with Dovecot, but I'm curious to know if other folks have observed this behavior when dealing with iOS Mail app clients? The log entries look like this: iOS 10 device = 172.16.0.1 RHEL7 Dovecot proxy host = 192.168.0.1 ("proxyhost") Panda/UW-IMAP target = panda.imap.tld Mar 6 12:11:00 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<VvzqehVKhwBCol00> Mar 6 12:11:00 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by client): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS: Disconnected, session=<VvzqehVKhwBCol00> Mar 6 12:11:01 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<jaXxehVKiABCol00> Mar 6 12:11:01 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<jaXxehVKiABCol00> Mar 6 12:11:01 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<RmX4ehVKiQBCol00> Mar 6 12:11:01 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<RmX4ehVKiQBCol00> Mar 6 12:11:02 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<inYBexVKigBCol00> Mar 6 12:11:02 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<inYBexVKigBCol00> Mar 6 12:11:02 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<GwAJexVKiwBCol00> Mar 6 12:11:02 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<GwAJexVKiwBCol00> Mar 6 12:11:03 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<asUPexVKswBCol00> Mar 6 12:11:03 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<asUPexVKswBCol00> Mar 6 12:11:03 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<n9YYexVKjQBCol00> Mar 6 12:11:04 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<n9YYexVKjQBCol00> ...and on and on, usually until the 'service imap-login' process_limit is reached. You could naturally apply some iptables rate-limiting to avoid hitting process_limit, but it'd be nice to have the iOS client simply behave properly instead. dovecot -n: --- # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-514.6.2.el7.x86_64 x86_64 Red Hat Enterprise Linux Server release 7.3 (Maipo) auth_mechanisms = plain login auth_verbose = yes first_valid_uid = 1000 imap_capability = +I18NLEVEL=1 mbox_write_locks = fcntl passdb { args = nopassword=y default_fields = proxy=y ssl=any-cert host=panda.imap.tld driver = static } protocols = imap pop3 service imap-login { process_limit = 400-ish at the moment process_min_avail = 2 } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } ssl = required ssl_ca = </etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem ssl_cert = <proxyhost.crt ssl_dh_parameters_length = 2048 ssl_key = <proxyhost.key ssl_protocols = !SSLv3 !SSLv3 ssl_require_crl = no userdb { driver = static } --- Thanks for any insight out there. Robert Giles

1 0

SiS hashes file ?
by Sai Kiran Gummaraj 07 Mar '17

07 Mar '17

Hello, In mdbox SiS POSIX mail box setting - There are hard-links under - <mail_attachment_dir>/ha/sh/<hash>-<guid-1> <mail_attachment_dir>/ha/sh/<hash>-<guid-2> Why is a copy of the file kept under hashes/<hash> ? -- Regards, Sai Kiran

1 0

Transitioning away from mail_location = maildir:~
by Chris Stankevitz 06 Mar '17

06 Mar '17

Hi, I have been using this setup for years: mail_home = /var/mail/vhosts/%d/%n mail_location = maildir:~ I have since learned that mail_home and mail_location should be different. I plan to use this: mail_home = /var/mail/vhosts/%d/%n mail_location = maildir:~/mail I would like the transition to be transparent for my email clients. I don't want mail/directories/sieve_scripts to disappear. 1. Should I manually create the maildir:~/mail directories? 2. Should I move files from /var/mail/vhosts/%d/%n into maildir:~/mail? Which files? (In other words: which files are "home directory" files and which files are "mail files"?) Thank you, Chris Example home directory: drwx------ . drwxr-xr-x .. drwx------ .Archives drwx------ .Archives.2016 drwx------ .Archives.2017 drwx------ .Drafts drwx------ .Junk drwx------ .Notes drwx------ .Sent drwx------ .Sent drwx------ .Trash drwx------ .work drwx------ .home drwx------ .todo lrwx------ .dovecot.sieve -rw------- .dovecot.sieve.log -rw------- .dovecot.sieve.log.0 -rw------- .dovecot.svbin drwx------ cur -rw------- dovecot-keywords -rw------- dovecot-uidlist -rw------- dovecot-uidvalidity -r--r--r-- dovecot-uidvalidity.55411048 -rw------- dovecot.index -rw------- dovecot.index.cache -rw------- dovecot.index.log -rw------- dovecot.mailbox.log drwx------ new drwx------ sieve -rw------- subscriptions drwx------ tmp $ dovecot -n # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: FreeBSD 10.3-RELEASE-p11 amd64 lmtp_save_to_detail_mailbox = yes mail_gid = 1002 mail_home = /var/mail/vhosts/%d/%n mail_location = maildir:~ mail_privileged_group = vpostfix mail_uid = 1002 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users driver = passwd-file } plugin { recipient_delimiter = - sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap pop3 lmtp sieve recipient_delimiter = - service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = postfix mode = 0600 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = xxx ssl_key = xxx userdb { args = username_format=%u /usr/local/etc/dovecot/users driver = passwd-file } protocol lmtp { mail_plugins = " sieve" postmaster_address = xxx } protocol lda { mail_plugins = " sieve" }

5 6

quota-warning: possible to have size also?
by dovecot＠avv.solutions 06 Mar '17

06 Mar '17

Hello Community, My dovecot setup works fine with quotas and quota-warning. I have a questions though: when running the warning script, the example foundis it possible to pass the *quota size *as argument also? This would be useful with per-user quota. e.g. /some/script xx% username *xxxbytes* (order is not relevant of course) Thank you for your help. Cheers

4 4

dovecot 2.2.x listening on multiple interfaces
by David Mehler 06 Mar '17

06 Mar '17

Hello, I'm wondering if it's possible to get dovecot to listen on multiple interfaces without designating * for everything? I've got a machine with three interfaces going to three different networks. I'm not wanting to let dovecot listen on all interfaces * just two of them, standard ports. Is this doable? Thanks. Dave.

2 1

Upgrade from 2.0. to 2.2
by Jakob Curdes 06 Mar '17

06 Mar '17

Hello, we have a centos 6 server running dovecot 2.0 provided with the OS, currently 2.0.9. We would like to upgrade to the 2.2.19 package provided by the mailserver.guru repo. I read the upgrading docs for 2.1 and 2.2 and found the following issues: 2.0 - 2.1: - define namespace inbox or delete 15-mailboxes.conf (we do not "care about special use mailboxes") - plugins and UTF-8: we do not use one of the listed plugins - to allow mixed usernames set auth_username_format to "nothing" - solr backend: not used in our setup - expire plugin: not used - dsync: not used 2.1 - 2.2 - fts-solr: not used (?) - autocreate: not used - other changes: trivial or not used Are there other things that we need to observe? Is a direct update from 2.0.9 to 2.2.19 feasible with the above precautions or do we need to put in a 2.1 version in between? Thank you for hints, Jakob

4 6

welcome plugin
by David Mehler 06 Mar '17

06 Mar '17

Hello, Is anyone using the welcome plugin? I'm trying to utilize it to send a message when a user first logs in to the system, containing important information for them to know. The plugin loads, I don't have a configuration problem, but the message never gets sent. What can I provide to more easily troubleshoot this? Thanks. Dave.

3 5

[enhancement] fts-solr low performance
by azurit＠pobox.sk 05 Mar '17

05 Mar '17

Hi, we have activated fts-solr about a week ago and immediately started to experience really *low* performance with MOVE and EXPUNGE commands. After several days of googling, tcpdumping and straceing i was able to find and resolve the problem. We are using Dovecot 2.2.27 from Debian Jessie (jessie-backports), which is doing a soft commit in solr after every MOVE or EXPUNGE command - this behavior cannot be, currently, changed. The problem is that this was causing every MOVE/EXPUNGE to take about 6 seconds to complete. The problem appears to be in very old version of Solr - 3.6.2 (!!). This is the only version which is shipped with current (Jessie) and also next (Stretch) version of Debian, don't ask my why, i don't understand it either. Solr versions below 4.0 are NOT supporting soft commits, so all commits are hard and this was the problem. Finally, i decided to patch our Dovecot to not send a commit at all and everything started to be super fast. I'm doing hard commits every minute via cron so the only consequence of this is that you cannot search for messages delivered before less then a minute (which you, usually, don't need to do anyway). While googling i also find out that Solr supports autoCommit function (and from version 4.0 also autoSoftCommit), so there's no reason for Dovecot to handle this on it's own (and potentially doing hundreds or thousands of soft commits every second) - you can just set Solr to, for example, do autoSoftCommit every second and autoCommit every minute: https://cwiki.apache.org/confluence/display/solr/UpdateHandlers+in+SolrConf… Also this wiki page should be updated with warning about old versoins of Solr not supporting soft commits (you could also mention the auto[Soft]Commit function): http://wiki2.dovecot.org/Plugins/FTS/Solr I suggest to allow completely disable Solr commits in Dovecot by configuration, so people like me can handle this easily. What do you think? azur

1 0