June 2015 - dovecot - dovecot.org

doveadm-server hangs with 100% cpu usage
by Dennis Kuhn 02 Sep '15

02 Sep '15

we have some replication issues. From time to time a doveadm-server process takes 100% cpu in the state recv_mailbox_tree_deletes on the replica. The process runs forever until it is manually killed. Strace on this process doesn't show anything. Sometimes we have several doveadm-server processes in this state, all for the same account, all with 100% CPU Load. The logfile on the master says: Error: Timeout during state=recv_mailbox_tree Error: dsync(dobby5.heinlein-support.de): I/O has stalled, no activity for 600 seconds My workaround is to delete the user directory on the replica so that the whole account is replicated again. This solves the problem for this specific account. The dovecot version is 2.2.15 on the master and 2.2.16 on the replica. Dennis Kuhn -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-57 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

3 4

mailbox deletion is not replicated correctly
by Dennis Kuhn 27 Aug '15

27 Aug '15

i can't delete several folders at once. I have a folder structure like folder1/folder2/folder3 (we use slash as hierarchy separator). When i delete folder1 server1 executes the command correctly Apr 16 13:44:28 server1 dovecot: imap(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2/folder3 Apr 16 13:44:28 server1 dovecot: imap(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2 Apr 16 13:44:28 server1 dovecot: imap(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1 but the replica server2 reverses the order: Apr 16 13:44:29 server2 dovecot: dsync-server(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1 Apr 16 13:44:29 server2 dovecot: dsync-server(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2 Apr 16 13:44:29 server2 dovecot: dsync-server(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2/folder3 This creates the folder INBOX/Trash/folder1/folder2 on server2 again which replicates back to server1. Regards, Dennis Kuhn -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-57 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

3 4

PROXY protocol
by Hoggins! 26 Aug '15

26 Aug '15

Hello folks, Any plans on implementing the PROXY protocol to allow Dovecot being behind a TCP proxy, and still logging the real IP address of the users ? See : http://blog.haproxy.com/haproxy/proxy-protocol/ Thanks !

6 15

NFSv4 delegation
by brd 18 Aug '15

18 Aug '15

hi all, i'm managing a large installation of a dovecot cluster in director + NFS backend architecture and we are moving from NFSv3 to NFSv4. Our NFS server is a Netapp in clustered mode and reading technical specs of NFSv4 delegation feature it seems that enabling delegation in this type of dovecot architecture should bring great benefits: only one backend server access a specific mailbox at a time (even deliveries are managed by director hashing via LMTP) so the getattr rpc calls should decrease significantly while the risk of conflicting delegations is very low (impacting negatively on the benefits obtainable). Can anyone confirm this hypothesis? I've heard that the linux implementation of NFSv4 delegation is mature only for read delegation (and not yet for write delegation): is this true? And, if true, could this fact limit the benefits of activating NFS delegation feature? Best regards -brd

4 11

Duplicate mails with pop3 + dsync replication
by Wolfgang Hennerbichler 17 Aug '15

17 Aug '15

Hi, When I enable (and it only happens when it is actually running) replication over TCP my users get duplicate mails via pop3. Here’s what happens: * User gets mail * Mail appears in (Maildir) new/ folder on both replicas * user retrieves mail (retr), calls dele. Mail still exists in new/ folder on both replicas. * user runs retr (and gets the duplicate) and dele a second time, the mail is actually removed. If I disable replication this behaviour goes away. I use dovecot from debian stable resources. Any help would _really_ be appreciated. I found one german thread that dealt with the same problem, but no solution for it (http://www.dovecot-buch.de/mailingliste/pop3-fehler-bei-delete-und-dsync-re…) wogri # dovecot --version 2.2.13 # dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.0 ext4 auth_verbose = yes doveadm_password = XXX first_valid_gid = 113 first_valid_uid = 108 last_valid_gid = 113 last_valid_uid = 108 mail_gid = dovecot mail_location = maildir:/var/mail/%n mail_plugins = quota fts fts_lucene virtual notify replication mail_temp_dir = /var/lib/dovecot/tmp mail_uid = dovecot namespace { list = children location = virtual:/var/mail/%n/virtual prefix = virtual. separator = . } namespace inbox { inbox = yes list = yes location = mailbox "Deleted Messages" { auto = subscribe special_use = \Trash } mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } [48/4646] mailbox Trash { special_use = \Trash } prefix = separator = . subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { default_language = de fts = lucene fts_lucene = whitespace_chars=@. mail_replica = tcp:172.16.1.1:12345 quota = maildir:User quota quota_rule = *:storage=4G quota_rule2 = Trash:storage=+100M quota_rule3 = Spam:ignore quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /etc/sieve/%n.sieve sieve_default = /etc/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +editheader } postmaster_address = postmaster(a)wogri.at protocols = " imap lmtp pop3" service aggregator { fifo_listener replication-notify-fifo { user = dovecot } unix_listener replication-notify { user = dovecot } } service doveadm { inet_listener { port = 12345 } } service imap { process_limit = 1024 } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 2003 } user = dovecot } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service pop3 { process_limit = 1024 } service quota-warning { executable = script /usr/local/sbin/quota-warning.sh unix_listener quota-warning { user = dovecot } user = dovecot } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl = required ssl_cert = </etc/exim4/server.cert.pem ssl_key = </etc/exim4/server.key.pem userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota sieve virtual notify replication } protocol lda { mail_plugins = quota fts fts_lucene virtual notify replication sieve } protocol imap { mail_max_userip_connections = 50 mail_plugins = quota fts fts_lucene virtual notify replication imap_quota }

2 3

[Dovecot] Dovecot FTS using ElasticSearch
by Darvin Denmian 21 Jul '15

21 Jul '15

Hi, I would like to know if is possible to use ElasticSearch instead of Solr for FTS. Regards, Erick Vitor de Barros.

6 6

Large jumps in dovecot-uidlist
by Mark 07 Jul '15

07 Jul '15

Since upgrading to Debian Wheezy I have been observing large jumps in the value stored in dovecot-uidlist. The effect of these jumps is to confuse some mail clients (Thunderbird, Opera Mail) causing them not to display messages from a (random?) point in time onwards in the affected folder. I have been unable to determine the root cause of these jumps. What I do know: 1. I never observed the problem while running Debian Squeeze. 2. The only way I have found to fix the issue is to remove all the dovecot* files from the affected folder and restart dovecot. I have tried removing various subsets of the dovecot* files but haven't yet found a subset that works. 3. I first tried upgrading from Squeeze to Wheezy within a few weeks of the Wheezy release. At that point the problem occurred so frequently that I couldn't sync my mail client with my mail box. Multiple folders exhibited this problem and while I was fixing them the problem appeared in other folders. I was unable to reach a point where the system was stable so I reverted back to Squeeze. 4. Some time later (I think a year but I am not sure) I tried the upgrade to Wheezy again. This time the system was stable. After a few days I noticed that the problem was still occurring but less frequently. Since then I have been fixing the problem as I notice it and experimenting with various settings (mainly locking related) without success. I now have a cron job that looks for problematic folders and fixes them when found. 5. I have updated Dovecot packages to those from Jessie via wheezy-backports and the problem still occurs. 6. Dovecot is installed along with postfix, amavisd-new, clamav-daemon, procmail and fetchmail. 7. The mail server is running on a eSATA SheevaPlug that boots of an external USB drive. All user home dirs (and associated mailboxes) are located on a LUKS/ext3 encrypted partition. 8. The output of dovecot -n may be found at the end of this mail. 9. Uid values for the three most recent instances of this problem are: 3801596527, 3371927249, 3443181615 10. Some folders seem more susceptible to this problem but I have not been able to identify any common factors/triggers. 11. The problem occurs - on average - once a day. I am happy to provide any further information, configure any debug logging and/or test suggested configuration settings that may help track down the root cause of this problem. Any help gratefully appreciated. Mark # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-kirkwood armv5tel Debian 7.8 mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 100 namespace { inbox = yes location = mailbox { special_use = \Drafts name = Drafts } mailbox { special_use = \Junk name = Junk } mailbox { special_use = \Sent name = Sent } mailbox { special_use = \Sent name = Sent Messages } mailbox { special_use = \Trash name = Trash } prefix = name = inbox } passdb { driver = pam } protocols = " imap" service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service { unix_listener { group = postfix mode = 0666 user = postfix path = /var/spool/postfix/private/auth-client } name = auth } service config { name = config } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service { inet_listener { port = 0 name = imap } inet_listener { port = 993 ssl = yes name = imaps } process_min_avail = 5 name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service { executable = imap postlogin name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service { executable = script-login -d rawlog name = postlogin } service replicator-doveadm { name = replicator } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } ssl_cert = </etc/ssl/certs/smtpd.crt ssl_cipher_list = HIGH:MEDIUM:!aNULL:!MD5:!RC4 ssl_key = </etc/ssl/private/smtpd.key userdb { driver = passwd } protocol imap { service replication-notify-fifo { name = aggregator } service anvil-auth-penalty { name = anvil } service auth-worker { name = auth-worker } service auth-client { name = auth } service config { name = config } service dict { name = dict } service login/proxy-notify { name = director } service dns-client { name = dns_client } service doveadm-server { name = doveadm } service imap { name = imap-login } service imap-urlauth { name = imap-urlauth-login } service imap-urlauth-worker { name = imap-urlauth-worker } service token-login/imap-urlauth { name = imap-urlauth } service login/imap { name = imap } service indexer-worker { name = indexer-worker } service indexer { name = indexer } service ipc { name = ipc } service lmtp { name = lmtp } service log-errors { name = log } service pop3 { name = pop3-login } service login/pop3 { name = pop3 } service replicator-doveadm { name = replicator } service login/ssl-params { name = ssl-params } service stats-mail { name = stats } }

2 4

Questions about hardlinks, alternate storage and compression
by Gionatan Danti 06 Jul '15

06 Jul '15

Hi all, I have some questions about hardlinks, alternate storage and compression. I already scanned the list for related information and I have an idea of how things works, but I would like to have a definite answer. System spec: - CentOS 6.6 x64 - dovecot-2.0.9-8.el6_6.4.x86_64 RPM package/version - sdbox mail store 1) About hardlinks: when sending the same message to two different recipients, I see that the two u.x files are created as two different files. Diffing them, I see that the only difference is a single char (see [1] for an example). My questions are: a) it is possible to tell dovecot to create a single file + a single hardlink (linkref=2)? As other IMAP servers support that features (eg: Cyrus, CommunigatePro, etc) I am wondering if I missed something in configuring dovecot... b) If it is not possible, can I run a script that compare the various files and substitute equal ones (minus the changing line) with hardlinks, or it will confuse dovecot? As a side note, why the changing line ever exists? 2) About alternate storage and compression: actually, I use a single mail_location without compression. I would like to have an alternate storage and to enable compression on it only, leaving the main location without compression. I if understand it correctly, it _should_ be done using a command similar to "doveadm -Dv -o "plugin/zlib_save=gz" altmove -uu testuser sentbefore 8d". I'm right thinking that it should work? I will really end with a primary uncompressed mail store and an alternate, zlib-compressed one? Thank you all and sorry if I did some naive questions. [1] 63c63 < G2fd0811c64be8e553d970000eaa8309f --- > G2ed0811c64be8e553d970000eaa8309f -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti(a)assyoma.it - info(a)assyoma.it GPG public key ID: FF5F32A8

1 2

Re: dovecot auth using 100% CPU
by Edward Betts 03 Jul '15

03 Jul '15

Jorge Bastos <mysql.jorge(a)decimal.pt> wrote: > What do you see in the logs? > My guess is that someone is trying a brute force auth against you, Thanks Jorge, I think this is the answer. I'm using dovecot for exim4 SMTP authentication. The exim4 logs show brute force attacks. -- Edward.

3 4

Outlook 2013 not fetching new mail/synchronization issues
by Dragos Pacher 02 Jul '15

02 Jul '15

Hi all, Dovecot 2.2.10 CentOS 6 64 bit. We are having terrible synchronization issues with Outlook 2013 (all updates) on Windows 7 SP1 and 8.1 hosts (all updates). Scenario: 1. Emails were sent since user closed Outlook 2. Next day: start computer, open outlook 3. No new email (even though email were sent overnight) 4. half a day later Outlook fetches emails like nothing happened Second issue: folders created or email send in Outlook 2013 does not synchronize on IMAP, they remain only local .. But I am mostly interested into finding a fix for the first synchronization issue described in scenario. Second is also important but I have a feeling its connected to the first. Any help will be highly appreciated! dovecot.conf listen = * #base_dir = /var/run/dovecot # Enabled mail protocols. #protocols = pop3 imap sieve protocols = pop3 imap # User/group who owns the message files: mail_uid = 89 mail_gid = 89 # Assign uid to virtual users. first_valid_uid = 89 last_valid_uid = 89 mail_max_userip_connections = 200 default_client_limit = 3000 namespace inbox { separator = . prefix = INBOX. inbox = yes # These mailboxes are widely used and could perhaps be created automatically: mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Trash { special_use = \Trash } # For \Sent mailboxes there are two widely used names. We'll mark both of # them as \Sent. User typically deletes one of them if duplicates are created. mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } } # Logging. Reference: http://wiki2.dovecot.org/Logging log_path = /var/log/dovecot.log mail_debug = no auth_verbose = no auth_debug = no auth_debug_passwords = no auth_verbose_passwords = no auth_worker_max_count = 2048 # SSL: Global settings. # Refer to wiki site for per protocol, ip, server name SSL settings: # http://wiki2.dovecot.org/SSL/DovecotConfiguration ssl = yes verbose_ssl = no ssl_ca = </etc/ssl/savvy.cz-CA.pem ssl_cert = </etc/ssl/savvy.cz.crt ssl_key = </etc/ssl/savvy.cz.key disable_plaintext_auth = no # Mail location and mailbox format. #mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ mail_location = maildir:~/Maildir:INDEX=/home/dovecotindex/%Lh/Maildir/ # Authentication related settings. # Append this domain name if client gives empty realm. auth_default_realm = # Authentication mechanisms. auth_mechanisms = PLAIN LOGIN service imap-login { client_limit = 3024 process_limit = 2048 process_min_avail = 64 vsz_limit = 64 M } service auth { client_limit = 3072 unix_listener auth-userdb { } inet_listener { port = 12345 } } service imap { vsz_limit = 256M # Max. number of IMAP processes (connections) process_limit = 2048 #imap_idle_notify_interval = 2 mins } service pop3 { # Max. number of POP3 processes (connections) process_limit = 1024 } # Virtual mail accounts. userdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } plugin { auth_socket_path = /var/run/dovecot/auth-master # Plugin: autocreate. Create and subscribe to default IMAP folders. autocreate = INBOX autocreate2 = Sent autocreate3 = Trash autocreate4 = Drafts autocreate5 = Junk autosubscribe = INBOX autosubscribe2 = Sent autosubscribe3 = Trash autosubscribe4 = Drafts autosubscribe5 = Junk # Plugin: expire. expire = Trash 7 Trash/* 7 Junk 30 expire_dict = proxy::expire # Pigeonhole managesieve service. # Reference: http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration # Per-user sieve settings. sieve_dir = /%Lh/sieve sieve = /%Lh/sieve/dovecot.sieve # Global sieve settings. sieve_global_dir = /home/vpopmail/sieve sieve_global_path = /home/vpopmail/sieve/dovecot.sieve #sieve_before = #sieve_after = } service dict { unix_listener dict { # mode = 0660 # user = vmail # group = vmail } } protocol lda { # Reference: http://wiki2.dovecot.org/LDA #mail_plugins = quota sieve autocreate #auth_socket_path = /var/run/dovecot/auth-master #log_path = /var/log/sieve.log #lda_mailbox_autocreate = yes #postmaster_address = root } protocol imap { mail_max_userip_connections = 2048 imap_client_workarounds = tb-extra-mailbox-sep #mail_plugins = quota imap_quota autocreate #mail_plugins = autocreate } protocol pop3 { #mail_plugins = quota pop3_uidl_format = UID%u-%v pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } Regards, Pacher Dragos

3 12