March 2017 - dovecot - dovecot.org

dovecot 2.2.x listening on multiple interfaces
by David Mehler 06 Mar '17

06 Mar '17

Hello, I'm wondering if it's possible to get dovecot to listen on multiple interfaces without designating * for everything? I've got a machine with three interfaces going to three different networks. I'm not wanting to let dovecot listen on all interfaces * just two of them, standard ports. Is this doable? Thanks. Dave.

2 1

Upgrade from 2.0. to 2.2
by Jakob Curdes 06 Mar '17

06 Mar '17

Hello, we have a centos 6 server running dovecot 2.0 provided with the OS, currently 2.0.9. We would like to upgrade to the 2.2.19 package provided by the mailserver.guru repo. I read the upgrading docs for 2.1 and 2.2 and found the following issues: 2.0 - 2.1: - define namespace inbox or delete 15-mailboxes.conf (we do not "care about special use mailboxes") - plugins and UTF-8: we do not use one of the listed plugins - to allow mixed usernames set auth_username_format to "nothing" - solr backend: not used in our setup - expire plugin: not used - dsync: not used 2.1 - 2.2 - fts-solr: not used (?) - autocreate: not used - other changes: trivial or not used Are there other things that we need to observe? Is a direct update from 2.0.9 to 2.2.19 feasible with the above precautions or do we need to put in a 2.1 version in between? Thank you for hints, Jakob

4 6

welcome plugin
by David Mehler 06 Mar '17

06 Mar '17

Hello, Is anyone using the welcome plugin? I'm trying to utilize it to send a message when a user first logs in to the system, containing important information for them to know. The plugin loads, I don't have a configuration problem, but the message never gets sent. What can I provide to more easily troubleshoot this? Thanks. Dave.

3 5

[enhancement] fts-solr low performance
by azurit＠pobox.sk 05 Mar '17

05 Mar '17

Hi, we have activated fts-solr about a week ago and immediately started to experience really *low* performance with MOVE and EXPUNGE commands. After several days of googling, tcpdumping and straceing i was able to find and resolve the problem. We are using Dovecot 2.2.27 from Debian Jessie (jessie-backports), which is doing a soft commit in solr after every MOVE or EXPUNGE command - this behavior cannot be, currently, changed. The problem is that this was causing every MOVE/EXPUNGE to take about 6 seconds to complete. The problem appears to be in very old version of Solr - 3.6.2 (!!). This is the only version which is shipped with current (Jessie) and also next (Stretch) version of Debian, don't ask my why, i don't understand it either. Solr versions below 4.0 are NOT supporting soft commits, so all commits are hard and this was the problem. Finally, i decided to patch our Dovecot to not send a commit at all and everything started to be super fast. I'm doing hard commits every minute via cron so the only consequence of this is that you cannot search for messages delivered before less then a minute (which you, usually, don't need to do anyway). While googling i also find out that Solr supports autoCommit function (and from version 4.0 also autoSoftCommit), so there's no reason for Dovecot to handle this on it's own (and potentially doing hundreds or thousands of soft commits every second) - you can just set Solr to, for example, do autoSoftCommit every second and autoCommit every minute: https://cwiki.apache.org/confluence/display/solr/UpdateHandlers+in+SolrConf… Also this wiki page should be updated with warning about old versoins of Solr not supporting soft commits (you could also mention the auto[Soft]Commit function): http://wiki2.dovecot.org/Plugins/FTS/Solr I suggest to allow completely disable Solr commits in Dovecot by configuration, so people like me can handle this easily. What do you think? azur

1 0

fts_solr and connection via https://
by Jan Vonde 04 Mar '17

04 Mar '17

Hi, I am trying to get fts_solr working and my index server is available via HTTPS only. Dovecot is running on a Debian Jessie system and the Solr server has a letsencrypt certificate. My dovecot version is: 2.2.devel (a9ed8ae) The current setup is: 10-mail.conf: mail_plugins = fts fts_solr 90-fts.conf: plugin { fts = solr fts_autoindex = yes fts_solr = url=https://foo.example.com/solr/dovecot/ } When I try to index the mailboxes I am getting error messages like this: doveadm(user@host): Error: fts_solr: Lookup failed: 9002 Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings) doveadm(user@host): Error: Mailbox INBOX: Status lookup failed: Internal error occurred. Refer to server log for more information. [2017-01-22 09:52:38] Segmentation fault Contacting the index server via curl on the command line on the same host works, it returns HTTP 200: user@host ~ $ curl -s -o /dev/null -w "%{http_code}" https://foo.example.com/solr/ 200 user@host ~ $ Currently I have the following ssl related settings: user@host ~ $ doveconf -n -P | grep -i ssl ssl_cert = </etc/ssl/certs/mail.example.org.crt ssl_cipher_list = ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM ssl_key = </etc/ssl/private/mail.example.org.key ssl_protocols = !SSLv2 !SSLv3 I tried adding the following settings but that didn't help: ssl_ca = < /etc/ssl/certs/ca-certificates.crt ssl_client_ca_dir = /etc/ssl/certs Can you give me a hint how I can get the ssl certificate accepted? Thanks in advance and have a nice day, Jan :-)

2 10

mdbox Inconsistency in map index
by bOnK 04 Mar '17

04 Mar '17

After a (power)crash two accounts have been corrupted. I tried to rescue things by running force-resync multiple times, but that didn't work out. Searching the archives, I found a recent suggestion (by Timo) to delete the index files, but I'm not sure which files to delete and what the consequences will be. When I deleted ``storage/dovecot.map.index'' in an unimportant account of my own, things only grew worse and I decided to delete the account and start fresh... Something I obviously don't want to do with the other account as it belongs to a client with about 2.5G mails (856 m.* files), which we don't want to lose. I have observed the following behavior in the logs: When a new mail arrives: (The second line below is always the same, however there's no mail with uid 12819 according to doveadm search uid 12819) Mar 04 11:43:28 lda(user): Warning: mdbox /var/mail/user/storage: Inconsistency in map index (117,1060 != 117,1883660) Mar 04 11:43:28 lda(user): Error: Log synchronization error at seq=117,offset=1546660 for /var/mail/user/storage/dovecot.map.index: Extension record inc drops number below zero (uid=12819, diff=-1, orig=0) Mar 04 11:43:28 lda(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:43:28 lda(user): Info: msgid=<d760229d563e99dc787acd0746b18b31(a)feedback.1dayfly.net>: saved mail to INBOX Mar 04 11:43:28 lda(user): Warning: mdbox /var/mail/user/storage: Inconsistency in map index (117,1060 != 117,1883784) Mar 04 11:43:28 lda(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:43:28 lda(user): Warning: mdbox /var/mail/user/storage: rebuilding indexes Mar 04 11:43:28 lda(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:43:28 imap(user): Error: /var/mail/user/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Mar 04 11:43:28 imap(user): Info: IMAP session state is inconsistent, please relogin. in=2011 out=108358 deleted=0 expunged=0 trashed=0 When user logs in next (might be from another box/MUA) Mar 04 11:44:36 imap-login: Info: Login: user=<user>, method=PLAIN, rip=82.95.XX.XX, lip=37.97.XX.XX, mpid=14884, TLS, session=<VfvML+ZJuO9SX18R> Mar 04 11:44:37 imap(user): Warning: mdbox /var/mail/user/storage: Inconsistency in map index (117,1060 != 117,1884336) Mar 04 11:44:37 imap(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:44:37 imap(user): Warning: mdbox /var/mail/user/storage: rebuilding indexes Mar 04 11:44:37 imap(user): Error: /var/mail/user/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Mar 04 11:44:37 imap(user): Info: IMAP session state is inconsistent, please relogin. in=331 out=178391 deleted=0 expunged=0 trashed=0 Mar 04 11:44:37 imap(user): Warning: mdbox /var/mail/user/storage: Inconsistency in map index (117,1060 != 117,1884396) Mar 04 11:44:37 imap(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:44:37 imap(user): Warning: fscking index file /var/mail/user/storage/dovecot.map.index Mar 04 11:44:37 imap-login: Info: Login: user=<user>, method=PLAIN, rip=82.95.XX.XX, lip=37.97.XX.XX, mpid=14886, TLS, session=<AUTbL+ZJuu9SX18R> Besides the nuisance of getting kicked of every time a new mail arrives, user cannot delete mails (as in: move to Trash). TL;DR: which files to delete and what will be the consequences? -- b.

1 0

letsencrypt
by David Mehler 04 Mar '17

04 Mar '17

Hello, I know some users here are using letsencrypt for their CA. If this is to off topic write me privately. I'm wanting letsencrypt to take over as my CA, replacing existing self signed certificates. I've got web working, a certificate for https sites and one for webmail as they have different names. What I'm now wanting to do is get letsencrypt going for my email setup, the smtp handled by postfix, but mail, and imap I believe are handled by dovecot. With the web it was easy just let apache serve the token that letsencrypt needed and I got certificates. How do I do this with regards email? I hope that's clear. Any help appreciated. Thanks. Dave.

8 9

Re: letsencrypt
by Joseph Tam 03 Mar '17

03 Mar '17

> Thanks. Is there another way of doing this? I've got a web server > running on 80 and 443. Are there any other options? I'm getting this list in digest mode, so it's possible by the time this gets to you, I will have repeated someone else' suggestion. In this situation, where your dovecot server lives on the same host as a web server (wembail?), and this web server is already going certificate renewal, then just change the certificate to use SNI extension and add all LS services that live on this host. (This does not count as a cert renewal, but a new cert). (E.g. if you are using a certbot to get a certifiticate for "webmail.mydomain", then add "pop3.mydomain", "imap.mydomain" and "smtp.mydomain" to the certificate.) Your web server will have to virtually host those domains for the purposes of mapping the token pickup folder. Then you can use the same certificate for all TLS services hosted on that server. Joseph Tam <jtam.home(a)gmail.com>

1 0

Re: letsencrypt
by Joseph Tam 03 Mar '17

03 Mar '17

David Mehler <dave.mehler(a)gmail.com> writes: > I'm wanting letsencrypt to take over as my CA, replacing existing self > signed certificates. I've got web working, a certificate for https > sites and one for webmail as they have different names. What I'm now > wanting to do is get letsencrypt going for my email setup, the smtp > handled by postfix, but mail, and imap I believe are handled by > dovecot. SMTP is handled by postfix, imap/pop is handled by dovecot. > With the web it was easy just let apache serve the token that > letsencrypt needed and I got certificates. How do I do this with > regards email? You can do the DNS challenge method if your server has the ability to update DNS entries, or you can use certbot clients in standalone-mode that will act as a simple web server just long enough to serve out the token to complete the authentication. Joseph Tam <jtam.home(a)gmail.com>

1 0

Director+NFS Experiences
by Mark Moseley 03 Mar '17

03 Mar '17

As someone who is about to begin the process of moving from maildir to mdbox on NFS (and therefore just about to start the 'director-ization' of everything) for ~6.5m mailboxes, I'm curious if anyone can share any experiences with it. The list is surprisingly quiet about this subject, and articles on google are mainly just about setting director up. I've yet to stumble across an article about someone's experiences with it. * How big of a director cluster do you use? I'm going to have millions of mailboxes behind 10 directors. I'm guessing that's plenty. It's actually split over two datacenters. In the larger, we've got about 200k connections currently, so in a perfectly-balanced world, each director would have 20k connections on it. I'm guessing that's child's play. Any good rule of thumb for ratio of 'backend servers::director servers'? In my larger DC, it's about 5::1. * Do you use the perl poolmon script or something else? The perl script was being weird for me, so I rewrote it in python but it basically does the exact same things. * Seen any issues with director? In testing, I managed to wedge things by having my poolmon script running on all the cluster boxes (I think). I've since rewritten it to run *only* on the lowest-numbered director. When it wedged, I had piles (read: hundreds per second) of log entries that said: Feb 12 06:25:03 director: Warning: director(10.1.20.5:9090/right): Host 10.1.17.3 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.5:9090/right): Host 10.1.17.3 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host 10.1.17.3 is being updated before previous update had finished (down -> up) - setting to state=up vhosts=0 Feb 12 06:25:03 director: Warning: director(10.1.20.3:9090/left): Host 10.1.17.3 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=0 Because it was in testing, I didn't notice it and it was like this for several days till dovecot was restarted on all the director nodes. I'm not 100% on what happened, but my *guess* is that two boxes tried to update the status of the same backend server in rapid succession. * Assuming you're using NFS, do you still see non-trivial amounts of indexes getting corrupted? * Again, assuming NFS and assuming at least some corrupted indexes, what's your guess for success rate % for dovecot recovering them automatically? And how about success rate % for ones that dovecot wasn't able to do automatically but you had to use doveadm to repair it? Really what I'm trying to figure out is 1) how often sysops will need to manually recover indexes; and 2) how often admins *can't* manually recover indexes? * if you have unrecoverable indexes (and assuming you have snapshots on your NFS server), does grabbing the most recent indexes from the snapshots always work for recovery (obviously, up till the point that the snapshot was taken)? * Any gotchas you've seen anywhere in a director-fied stack? I realize that's a broad question :) * Does one of your director nodes going down cause any issues? E.g. issues with the left and right nodes syncing with each other? Or when the director node comes back up? * Does a backend node going down cause a storm of reconnects? In the time between deploying director and getting mailboxes converted to mdbox, reconnects for us will mean cold local-disk dovecot caches. But hopefully consistent hashing helps with that? * Do you have consistent hashing turned on? I can't think of any reason not to have it turned on, but who knows * Any other configuration knobs (including sysctl) that you needed to futz with, vs the default? I appreciate any feedback!

8 14