March 2017 - dovecot - dovecot.org

iOS Mail app and rapid authenticate / disconnect on Dovecot proxy
by Robert Giles 07 Mar '17

07 Mar '17

Hi folks, I have a handful of iOS 10.2.1 Mail app IMAP clients that intermittently break into this unexplained authenticate-then-immediately-disconnect behavior when connecting to a RHEL7 Dovecot (dovecot-2.2.10-7.el7) proxy, providing proxied connections to a backend Panda/UW-IMAP server. From talking to the users, the activity would appear to be spontaneous (ie: not caused by user interaction with the device). The behavior doesn't seem to have any observable implications for the end user, other than momentarily hitting the Dovecot process_limit (which, if not raised to a rather large number, disrupts new IMAP proxy connections momentarily). I reckon this is not an issue with Dovecot, but I'm curious to know if other folks have observed this behavior when dealing with iOS Mail app clients? The log entries look like this: iOS 10 device = 172.16.0.1 RHEL7 Dovecot proxy host = 192.168.0.1 ("proxyhost") Panda/UW-IMAP target = panda.imap.tld Mar 6 12:11:00 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<VvzqehVKhwBCol00> Mar 6 12:11:00 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by client): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS: Disconnected, session=<VvzqehVKhwBCol00> Mar 6 12:11:01 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<jaXxehVKiABCol00> Mar 6 12:11:01 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<jaXxehVKiABCol00> Mar 6 12:11:01 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<RmX4ehVKiQBCol00> Mar 6 12:11:01 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<RmX4ehVKiQBCol00> Mar 6 12:11:02 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<inYBexVKigBCol00> Mar 6 12:11:02 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<inYBexVKigBCol00> Mar 6 12:11:02 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<GwAJexVKiwBCol00> Mar 6 12:11:02 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<GwAJexVKiwBCol00> Mar 6 12:11:03 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<asUPexVKswBCol00> Mar 6 12:11:03 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<asUPexVKswBCol00> Mar 6 12:11:03 proxyhost dovecot: imap-login: proxy(jdoe): started proxying to panda.imap.tld:993: user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<n9YYexVKjQBCol00> Mar 6 12:11:04 proxyhost dovecot: imap-login: proxy(jdoe): disconnecting 172.16.0.1 (Disconnected by server): user=<jdoe>, method=PLAIN, rip=172.16.0.1, lip=192.168.0.1, TLS, session=<n9YYexVKjQBCol00> ...and on and on, usually until the 'service imap-login' process_limit is reached. You could naturally apply some iptables rate-limiting to avoid hitting process_limit, but it'd be nice to have the iOS client simply behave properly instead. dovecot -n: --- # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-514.6.2.el7.x86_64 x86_64 Red Hat Enterprise Linux Server release 7.3 (Maipo) auth_mechanisms = plain login auth_verbose = yes first_valid_uid = 1000 imap_capability = +I18NLEVEL=1 mbox_write_locks = fcntl passdb { args = nopassword=y default_fields = proxy=y ssl=any-cert host=panda.imap.tld driver = static } protocols = imap pop3 service imap-login { process_limit = 400-ish at the moment process_min_avail = 2 } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } ssl = required ssl_ca = </etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem ssl_cert = <proxyhost.crt ssl_dh_parameters_length = 2048 ssl_key = <proxyhost.key ssl_protocols = !SSLv3 !SSLv3 ssl_require_crl = no userdb { driver = static } --- Thanks for any insight out there. Robert Giles

1 0

SiS hashes file ?
by Sai Kiran Gummaraj 07 Mar '17

07 Mar '17

Hello, In mdbox SiS POSIX mail box setting - There are hard-links under - <mail_attachment_dir>/ha/sh/<hash>-<guid-1> <mail_attachment_dir>/ha/sh/<hash>-<guid-2> Why is a copy of the file kept under hashes/<hash> ? -- Regards, Sai Kiran

1 0

Transitioning away from mail_location = maildir:~
by Chris Stankevitz 06 Mar '17

06 Mar '17

Hi, I have been using this setup for years: mail_home = /var/mail/vhosts/%d/%n mail_location = maildir:~ I have since learned that mail_home and mail_location should be different. I plan to use this: mail_home = /var/mail/vhosts/%d/%n mail_location = maildir:~/mail I would like the transition to be transparent for my email clients. I don't want mail/directories/sieve_scripts to disappear. 1. Should I manually create the maildir:~/mail directories? 2. Should I move files from /var/mail/vhosts/%d/%n into maildir:~/mail? Which files? (In other words: which files are "home directory" files and which files are "mail files"?) Thank you, Chris Example home directory: drwx------ . drwxr-xr-x .. drwx------ .Archives drwx------ .Archives.2016 drwx------ .Archives.2017 drwx------ .Drafts drwx------ .Junk drwx------ .Notes drwx------ .Sent drwx------ .Sent drwx------ .Trash drwx------ .work drwx------ .home drwx------ .todo lrwx------ .dovecot.sieve -rw------- .dovecot.sieve.log -rw------- .dovecot.sieve.log.0 -rw------- .dovecot.svbin drwx------ cur -rw------- dovecot-keywords -rw------- dovecot-uidlist -rw------- dovecot-uidvalidity -r--r--r-- dovecot-uidvalidity.55411048 -rw------- dovecot.index -rw------- dovecot.index.cache -rw------- dovecot.index.log -rw------- dovecot.mailbox.log drwx------ new drwx------ sieve -rw------- subscriptions drwx------ tmp $ dovecot -n # 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: FreeBSD 10.3-RELEASE-p11 amd64 lmtp_save_to_detail_mailbox = yes mail_gid = 1002 mail_home = /var/mail/vhosts/%d/%n mail_location = maildir:~ mail_privileged_group = vpostfix mail_uid = 1002 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users driver = passwd-file } plugin { recipient_delimiter = - sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = imap pop3 lmtp sieve recipient_delimiter = - service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = postfix mode = 0600 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = xxx ssl_key = xxx userdb { args = username_format=%u /usr/local/etc/dovecot/users driver = passwd-file } protocol lmtp { mail_plugins = " sieve" postmaster_address = xxx } protocol lda { mail_plugins = " sieve" }

5 6

quota-warning: possible to have size also?
by dovecot＠avv.solutions 06 Mar '17

06 Mar '17

Hello Community, My dovecot setup works fine with quotas and quota-warning. I have a questions though: when running the warning script, the example foundis it possible to pass the *quota size *as argument also? This would be useful with per-user quota. e.g. /some/script xx% username *xxxbytes* (order is not relevant of course) Thank you for your help. Cheers

4 4

dovecot 2.2.x listening on multiple interfaces
by David Mehler 06 Mar '17

06 Mar '17

Hello, I'm wondering if it's possible to get dovecot to listen on multiple interfaces without designating * for everything? I've got a machine with three interfaces going to three different networks. I'm not wanting to let dovecot listen on all interfaces * just two of them, standard ports. Is this doable? Thanks. Dave.

2 1

Upgrade from 2.0. to 2.2
by Jakob Curdes 06 Mar '17

06 Mar '17

Hello, we have a centos 6 server running dovecot 2.0 provided with the OS, currently 2.0.9. We would like to upgrade to the 2.2.19 package provided by the mailserver.guru repo. I read the upgrading docs for 2.1 and 2.2 and found the following issues: 2.0 - 2.1: - define namespace inbox or delete 15-mailboxes.conf (we do not "care about special use mailboxes") - plugins and UTF-8: we do not use one of the listed plugins - to allow mixed usernames set auth_username_format to "nothing" - solr backend: not used in our setup - expire plugin: not used - dsync: not used 2.1 - 2.2 - fts-solr: not used (?) - autocreate: not used - other changes: trivial or not used Are there other things that we need to observe? Is a direct update from 2.0.9 to 2.2.19 feasible with the above precautions or do we need to put in a 2.1 version in between? Thank you for hints, Jakob

4 6

welcome plugin
by David Mehler 06 Mar '17

06 Mar '17

Hello, Is anyone using the welcome plugin? I'm trying to utilize it to send a message when a user first logs in to the system, containing important information for them to know. The plugin loads, I don't have a configuration problem, but the message never gets sent. What can I provide to more easily troubleshoot this? Thanks. Dave.

3 5

[enhancement] fts-solr low performance
by azurit＠pobox.sk 05 Mar '17

05 Mar '17

Hi, we have activated fts-solr about a week ago and immediately started to experience really *low* performance with MOVE and EXPUNGE commands. After several days of googling, tcpdumping and straceing i was able to find and resolve the problem. We are using Dovecot 2.2.27 from Debian Jessie (jessie-backports), which is doing a soft commit in solr after every MOVE or EXPUNGE command - this behavior cannot be, currently, changed. The problem is that this was causing every MOVE/EXPUNGE to take about 6 seconds to complete. The problem appears to be in very old version of Solr - 3.6.2 (!!). This is the only version which is shipped with current (Jessie) and also next (Stretch) version of Debian, don't ask my why, i don't understand it either. Solr versions below 4.0 are NOT supporting soft commits, so all commits are hard and this was the problem. Finally, i decided to patch our Dovecot to not send a commit at all and everything started to be super fast. I'm doing hard commits every minute via cron so the only consequence of this is that you cannot search for messages delivered before less then a minute (which you, usually, don't need to do anyway). While googling i also find out that Solr supports autoCommit function (and from version 4.0 also autoSoftCommit), so there's no reason for Dovecot to handle this on it's own (and potentially doing hundreds or thousands of soft commits every second) - you can just set Solr to, for example, do autoSoftCommit every second and autoCommit every minute: https://cwiki.apache.org/confluence/display/solr/UpdateHandlers+in+SolrConf… Also this wiki page should be updated with warning about old versoins of Solr not supporting soft commits (you could also mention the auto[Soft]Commit function): http://wiki2.dovecot.org/Plugins/FTS/Solr I suggest to allow completely disable Solr commits in Dovecot by configuration, so people like me can handle this easily. What do you think? azur

1 0

fts_solr and connection via https://
by Jan Vonde 04 Mar '17

04 Mar '17

Hi, I am trying to get fts_solr working and my index server is available via HTTPS only. Dovecot is running on a Debian Jessie system and the Solr server has a letsencrypt certificate. My dovecot version is: 2.2.devel (a9ed8ae) The current setup is: 10-mail.conf: mail_plugins = fts fts_solr 90-fts.conf: plugin { fts = solr fts_autoindex = yes fts_solr = url=https://foo.example.com/solr/dovecot/ } When I try to index the mailboxes I am getting error messages like this: doveadm(user@host): Error: fts_solr: Lookup failed: 9002 Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) doveadm(user@host): Error: Mailbox INBOX: Status lookup failed: Internal error occurred. Refer to server log for more information. [2017-01-22 09:52:38] Segmentation fault Contacting the index server via curl on the command line on the same host works, it returns HTTP 200: user@host ~ $ curl -s -o /dev/null -w "%{http_code}" https://foo.example.com/solr/ 200 user@host ~ $ Currently I have the following ssl related settings: user@host ~ $ doveconf -n -P | grep -i ssl ssl_cert = </etc/ssl/certs/mail.example.org.crt ssl_cipher_list = ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM ssl_key = </etc/ssl/private/mail.example.org.key ssl_protocols = !SSLv2 !SSLv3 I tried adding the following settings but that didn't help: ssl_ca = < /etc/ssl/certs/ca-certificates.crt ssl_client_ca_dir = /etc/ssl/certs Can you give me a hint how I can get the ssl certificate accepted? Thanks in advance and have a nice day, Jan :-)

2 10

mdbox Inconsistency in map index
by bOnK 04 Mar '17

04 Mar '17

After a (power)crash two accounts have been corrupted. I tried to rescue things by running force-resync multiple times, but that didn't work out. Searching the archives, I found a recent suggestion (by Timo) to delete the index files, but I'm not sure which files to delete and what the consequences will be. When I deleted ``storage/dovecot.map.index'' in an unimportant account of my own, things only grew worse and I decided to delete the account and start fresh... Something I obviously don't want to do with the other account as it belongs to a client with about 2.5G mails (856 m.* files), which we don't want to lose. I have observed the following behavior in the logs: When a new mail arrives: (The second line below is always the same, however there's no mail with uid 12819 according to doveadm search uid 12819) Mar 04 11:43:28 lda(user): Warning: mdbox /var/mail/user/storage: Inconsistency in map index (117,1060 != 117,1883660) Mar 04 11:43:28 lda(user): Error: Log synchronization error at seq=117,offset=1546660 for /var/mail/user/storage/dovecot.map.index: Extension record inc drops number below zero (uid=12819, diff=-1, orig=0) Mar 04 11:43:28 lda(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:43:28 lda(user): Info: msgid=<d760229d563e99dc787acd0746b18b31(a)feedback.1dayfly.net>: saved mail to INBOX Mar 04 11:43:28 lda(user): Warning: mdbox /var/mail/user/storage: Inconsistency in map index (117,1060 != 117,1883784) Mar 04 11:43:28 lda(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:43:28 lda(user): Warning: mdbox /var/mail/user/storage: rebuilding indexes Mar 04 11:43:28 lda(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:43:28 imap(user): Error: /var/mail/user/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Mar 04 11:43:28 imap(user): Info: IMAP session state is inconsistent, please relogin. in=2011 out=108358 deleted=0 expunged=0 trashed=0 When user logs in next (might be from another box/MUA) Mar 04 11:44:36 imap-login: Info: Login: user=<user>, method=PLAIN, rip=82.95.XX.XX, lip=37.97.XX.XX, mpid=14884, TLS, session=<VfvML+ZJuO9SX18R> Mar 04 11:44:37 imap(user): Warning: mdbox /var/mail/user/storage: Inconsistency in map index (117,1060 != 117,1884336) Mar 04 11:44:37 imap(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:44:37 imap(user): Warning: mdbox /var/mail/user/storage: rebuilding indexes Mar 04 11:44:37 imap(user): Error: /var/mail/user/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Mar 04 11:44:37 imap(user): Info: IMAP session state is inconsistent, please relogin. in=331 out=178391 deleted=0 expunged=0 trashed=0 Mar 04 11:44:37 imap(user): Warning: mdbox /var/mail/user/storage: Inconsistency in map index (117,1060 != 117,1884396) Mar 04 11:44:37 imap(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:44:37 imap(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:44:37 imap-login: Info: Login: user=<user>, method=PLAIN, rip=82.95.XX.XX, lip=37.97.XX.XX, mpid=14886, TLS, session=<AUTbL+ZJuu9SX18R> Besides the nuisance of getting kicked of every time a new mail arrives, user cannot delete mails (as in: move to Trash). TL;DR: which files to delete and what will be the consequences? -- b.

1 0