- dovecot - dovecot.org

TCP replication: high connection rate and looping in incomplete state
by tok 16 Jun '18

16 Jun '18

Hi, I am trying to establish TCP(s)-based replication between two FreeBSD 11.1 machines, both running Dovecot 2.2.35 (via pkg) with 10 virtual users and 1.2GB maildir. I followed the Wiki description (https://wiki2.dovecot.org/Replication) just skipped the SSH part. My resulting configurations are attached below. Problem: After reloading both Dovecot instances, a few seconds later they start to initiate connections in an absurdly high rate (30-50 new connections during 10 secs, closing old ones after 2-3 secs) and even after 1 hour they do not „calm down". E-mails are copied, but judging by size and file count, not completely (approx. 1GB and maybe 90% of files are getting synced). What is going on here? On the destination host (with empty storage before replication) I see "incremental - Waiting for dsync to finish“ in dsync-status, constantly flipping between handshake and waiting, for solely 4 out of 10 accounts. Replicator status says 0 requests and 10 total users. On the source host, replicator status says 4 queued full resyncs, 3 waiting failed and 10 total users. I already tried both tcps and tcp; activating debug log did not reveal any errors (e.g. SSL). Would be happy for any advice to track this down. Best regards, tok Config source host: ----- # 2.2.35 (b1cb664): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.23 (b2e41927) # OS: FreeBSD 11.1-RELEASE-p10 amd64 zfs # Hostname: srv1.TLD auth_mechanisms = plain login doveadm_password = # hidden, use -P to show it doveadm_port = 12345 lmtp_save_to_detail_mailbox = yes login_trusted_networks = 172.16.1.110/32 mail_location = maildir:/srv/mail/vhosts/%d/%n mail_plugins = quota mailbox_alias notify replication namespace inbox { […] } passdb { args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users driver = passwd-file } plugin { mail_replica = tcps:srv2.TLD:12345 quota = maildir:User quota quota_grace = 10%% quota_max_mail_size = 50M quota_rule = *:storage=1G quota_rule2 = Trash:storage=+10%% quota_rule3 = Junk:storage=+20%% quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is over quota / full quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u quota_warning3 = -storage=100%% quota-warning below %u [...] } postmaster_address = postmaster@TLD quota_full_tempfail = yes service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service config { unix_listener config { user = vmail } } service doveadm { inet_listener { port = 12345 ssl = yes } } service imap-login { process_min_avail = 1 service_count = 1 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { process_min_avail = 1 service_count = 1 } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { address = 127.0.0.1 port = 12340 } } service quota-warning { executable = script /usr/local/bin/dovecot-quota-warning.sh user = vmail } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = vmail mode = 0666 } } ssl_cert = </usr/local/etc/ssl/acme.sh/srv1.TLD/fullchain.pem ssl_client_ca_file = /etc/ssl/cert.pem userdb { args = username_format=%u /usr/local/etc/dovecot/users driver = passwd-file } protocol lmtp { mail_plugins = quota mailbox_alias notify replication quota sieve } protocol lda { mail_plugins = quota mailbox_alias notify replication sieve } protocol imap { mail_plugins = quota mailbox_alias notify replication imap_quota imap_sieve } ----- Diff config of destination host: ----- 4c4 < # Hostname: srv2.TLD --- > # Hostname: srv1.TLD 9a10 > login_trusted_networks = 172.16.1.110/32 60c61 < mail_replica = tcps:srv1.TLD:12345 --- > mail_replica = tcps:srv2.TLD:12345 156c157 < ssl_cert = </usr/local/etc/ssl/acme.sh/srv1.TLD/fullchain.pem --- > ssl_cert = </usr/local/etc/ssl/acme.sh/srv1.TLD/fullchain.pem -----

1 0

failed: Stream is smaller than expected (0 < 5098)
by Vladimir Tiukhtin 15 Jun '18

15 Jun '18

Hi Looks like I have got in a situation with missing or corrupted attachments. I am getting from time to time lmtp(17147, user1(a)example.com) Error: read(attachments-connector(zlib(/var/spool/mail/example.com/user2/mail/stor… failed: read(/srv/attachments/user2/4a/e0/4ae062027dd745f56c2cc1c43fa4f349f52986a0-2bcf4e063994235bfb4200006f28d014[base64:19 b/l]) failed: Stream is smaller than expected (0 < 5098) Replication works, but is there a way to recover the logic even with empty files or cleaning indices? Thanks Vladimir

2 2

After logrotation Dovecot still writes to old log file
by Michael Heuberger 15 Jun '18

15 Jun '18

Hi there This is weird. On my latest Ubuntu server Dovecot seems to write to /var/log/dovecot.log.1 instead of a recently created /var/log/dovecot.log Here my logrotate config for Dovecot: /var/log/dovecot.log { su root syslog rotate 7 missingok copytruncate create 666 root syslog sharedscripts postrotate doveadm log reopen endscript } Doveadm should reopen it but no, still writes to old log file. Not sure, maybe it's someone else, not dovecot, writing to the log file? And here the versions: # 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: Linux 4.14.49-rh65-20180612025235.xenU.x86_64 x86_64 Ubuntu 18.04 LTS How can we investigate best? Or is there a mistake in the above logrotate config? - Michael -- Binary Kitchen Michael Heuberger 1/33 Parrish Road Sandringham Auckland 1025 (New Zealand) Mobile (text only) ... +64 21 261 89 81 Email ................ michael(a)binarykitchen.com Website .............. http://www.binarykitchen.com

3 4

new problem
by Walter Ulmke 14 Jun '18

14 Jun '18

1) my inbox is "Posteingang". should I officially declare it somewhere? I now get the following error messages: Jun 14 00:23:32 ulmke2 dovecot[3981]: imap(ulw)<3997><4O/Xbo1uotLAqGQd>: Error: opendir(/u/ulw/Mail) failed: Permission denied (euid=503(ulw) egid=100(users) UNIX perms appear ok (ACL/MAC wrong?)) Jun 14 00:23:32 ulmke2 dovecot[3981]: imap(ulw)<3997><4O/Xbo1uotLAqGQd>: Error: opendir(/u/ulw/Mail) failed: Permission denied (euid=503(ulw) egid=100(users) UNIX perms appear ok (ACL/MAC wrong?)) Jun 14 00:23:32 ulmke2 dovecot[3981]: imap(ulw)<3997><4O/Xbo1uotLAqGQd>: Error: Couldn't create mailbox list lock /u/ulw/Mail/mailboxes.lock: file_create_locked(/u/ulw/Mail/mailboxes.lock) failed: safe_mkstemp(/u/ulw/Mail/mailboxes.lock) failed: Permission denied Jun 14 00:23:33 ulmke2 dovecot[3981]: Error: imap(ulw)<3997><4O/Xbo1uotLAqGQd>: open(/u/ulw/Mail/dovecot.index.log) failed: Permission denied (euid=503(ulw) egid=100(users) UNIX perms appear ok (ACL/MAC wrong?)) Jun 14 00:23:33 ulmke2 dovecot[3981]: Error: imap(ulw)<3997><4O/Xbo1uotLAqGQd>: Mailbox INBOX: file_dotlock_create(/u/ulw/Mail/dovecot-uidlist) failed: Permission denied (euid=503(ulw) egid=100(users) UNIX perms appear ok (ACL/MAC wrong?)) Jun 14 00:23:33 ulmke2 dovecot[3981]: Error: imap(ulw)<3997><4O/Xbo1uotLAqGQd>: Mailbox INBOX: open(/u/ulw/Mail/dovecot-uidlist) failed: Permission denied Jun 14 00:23:43 ulmke2 dovecot[3981]: Error: imap(ulw)<3997><4O/Xbo1uotLAqGQd>: open(/u/ulw/Mail/dovecot.index.log) failed: Permission denied (euid=503(ulw) egid=100(users) UNIX perms appear ok (ACL/MAC wrong?)) Also, when I start Thunderbird I get the following error message: "der aktuelle Vorgang in "Posteingang" ist fehlgeschlagen. Der Server des Kontos antwortete (Serverbug) Internal error ocurred. Refer to server log WHERE IS THE SERVER LOG? -- -- Best Regards, Walter Ulmke Ulmke Machine Tools, 48496 Hopsten, Germany Tel. ++49/5458/93345-0 Fax. ++49/5458/93345-45 Mobile: ++49/172/5357999 eMail: ulw(a)ulmke.com Dipl.-Ing. Walter Ulmke e.K. AG Steinfurt HRA 4384

5 6

Namespaces for Apple Client
by Janzen, Tobias 14 Jun '18

14 Jun '18

Hi, i have a plesk server with postfix an dovecot. Webmail is roundcube. Version of Dovecot is 2.3.0.1 I want apple clients to use the default special folders. My config for namespaces worked till last week. Now I cant get it to work again. Hope someone can help me. # 2.3.0.1 (ffd8a29): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.0.1 (d33dca20) # OS: Linux 4.4.0-042stab130.1 x86_64 Ubuntu 16.04.4 LTS reiserfs auth_mechanisms = plain login digest-md5 cram-md5 apop auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890&.-_@' disable_plaintext_auth = no first_valid_uid = 30 imap_client_workarounds = delay-newmail imap_logout_format = rcvd=%i, sent=%o mail_home = /var/qmail/mailnames/%Ld/%Ln mail_location = maildir:/var/qmail/mailnames/%Ld/%Ln/Maildir mail_log_prefix = "service=%s, user=%u, ip=[%r]. " mail_plugins = " quota" managesieve_logout_format = rcvd=%i, sent=%o managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify namespace inbox { inbox = yes location = mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = create special_use = \Sent } prefix = INBOX. separator = . type = private } passdb { driver = plesk } plugin { mail_crypt_global_private_key = <ecprivkey.pem mail_crypt_global_public_key = <ecpubkey.pem mail_crypt_save_version = 2 quota = maildir:User quota quota_grace = 0 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_logout_format = rcvd=%i, sent=%o, top=%t/%p, retr=%r/%b, del=%d/%m, size=%s protocols = imap pop3 sieve service auth-worker { group = user = } service auth { group = unix_listener auth-userdb { group = popuser mode = 0600 user = popuser } user = } service imap { process_limit = 1024 service_count = 1 } service pop3 { process_limit = 1024 service_count = 1 } ssl_cert = </etc/dovecot/private/ssl-cert-and-key.pem ssl_cipher_list = HIGH:!aNULL:!MD5 ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes userdb { args = uid=popuser gid=popuser driver = static } protocol imap { mail_plugins = " quota imap_quota" } protocol pop3 { pop3_uidl_format = UID%u-%v } protocol lda { mail_plugins = " quota sieve" } HEIM & HAUS Produktion und Vertrieb GmbH Hochstra?e 7-9, 47169 Duisburg Telefon: +49 203 40644-0 Telefax: +49 203 407312 E-Mail: service-du(a)heimhaus.de Internet: http://www.heimhaus.de USt-IdNr.: DE 272301801 Sitz Duisburg, HRB 22438 Gesch?ftsf?hrer: Michael Demming, Dipl.-Ing. Klaus Rappers

1 0

can't login to Dovecot solved
by Walter Ulmke 14 Jun '18

14 Jun '18

solved. It was doen to not being able to log in in "plain" -- -- Best Regards, Walter Ulmke Ulmke Machine Tools, 48496 Hopsten, Germany Tel. ++49/5458/93345-0 Fax. ++49/5458/93345-45 Mobile: ++49/172/5357999 eMail: ulw(a)ulmke.com Dipl.-Ing. Walter Ulmke e.K. AG Steinfurt HRA 4384

1 0

cant login to Dovecot
by Walter Ulmke 13 Jun '18

13 Jun '18

Hi all, I can't login. This is a dovecot server for a small inhouse network, and I doon't use any encryption, and just want to login using the standard Linux login dovecot status gives me the following error messages: un 12 12:30:06 ulmke2 dovecot[5814]: auth: Fatal: APOP mechanism can't be supported with given passdbs Jun 12 12:30:06 ulmke2 dovecot[5811]: master: Error: service(auth): command startup failed, throttling for 32 secs Jun 12 12:30:06 ulmke2 dovecot[5814]: imap-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 10 secs): user=<>, rip=192.168.100.29, lip=192.168.100.2, session=<qXOLWW9uBNPAqGQd> Jun 12 12:30:06 ulmke2 dovecot[5814]: imap-login: Warning: Timeout leak: 0x7fa8a7d372a0 (auth-server-connection.c:397) Jun 12 12:30:16 ulmke2 dovecot[5814]: imap-login: Warning: Auth process not responding, delayed sending initial response (greeting): user=<>, rip=192.168.100.29, lip=192.168.100.2, session=<pw8lWm9uBtPAqGQd> Jun 12 12:30:36 ulmke2 dovecot[5814]: imap-login: Error: Timeout waiting for handshake from auth server. my pid=5874, input bytes=0 Jun 12 12:30:36 ulmke2 dovecot[5814]: imap-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 30 secs): user=<>, rip=192.168.100.29, lip=192.168.100.2, session=<pw8lWm9uBtPAqGQd> Jun 12 12:30:38 ulmke2 dovecot[5814]: auth: Error: stats: open(old-stats-user) failed: Permission denied Jun 12 12:30:38 ulmke2 dovecot[5814]: auth: Fatal: APOP mechanism can't be supported with given passdbs Jun 12 12:30:38 ulmke2 dovecot[5811]: master: Error: service(auth): command startup failed, throttling for 60 secs I'm moving to a new server. stats: open(old-stats-user) failed: Permission denied This file may be somewhere on the old server. Can anyone help me? -- -- Best Regards, Walter Ulmke

4 8

can't login to Dovecot
by Walter Ulmke 13 Jun '18

13 Jun '18

I've managed to turn on "plain" and now it seems I can logon. Jun 13 17:24:20 ulmke2 dovecot[1973]: imap(ulw)<2474><+kyqk4du5M7AqGQd>: Namespace '': Mail storage autodetection failed with home=/u/ulw in=0 out=373 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0 Jun 13 17:24:20 ulmke2 dovecot[1973]: imap-login: Login: user=<ulw>, method=PLAIN, rip=192.168.100.29, lip=192.168.100.2, mpid=2476, session=<GJCrk4du5s7AqGQd> Jun 13 17:24:20 ulmke2 dovecot[1973]: imap(ulw)<2476><GJCrk4du5s7AqGQd>: Error: Namespace '': Mail storage autodetection failed with home=/u/ulw All users mailboxes are $HOME/Mail How do I tell Dovecot this? -- -- Best Regards, Walter Ulmke Ulmke Machine Tools, 48496 Hopsten, Germany Tel. ++49/5458/93345-0 Fax. ++49/5458/93345-45 Mobile: ++49/172/5357999 eMail: ulw(a)ulmke.com Dipl.-Ing. Walter Ulmke e.K. AG Steinfurt HRA 4384

2 1

can't login to Dovecot
by Walter Ulmke 13 Jun '18

13 Jun '18

I have now deleted EVERYTHING and installed fresh. Now at least it starts doveconf -n: managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = passdb { driver = pam } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } ssl_cipher_list = ALL:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK:!RC4:!ADH:!LOW@STRENGTH ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = passwd } ----------------------------------- looks like PAM is already configured. I CAN'T LOKIN USING PLAIN TEXT. HOW DO i TURN THIS OFF?? -- -- Best Regards, Walter Ulmke Ulmke Machine Tools, 48496 Hopsten, Germany Tel. ++49/5458/93345-0 Fax. ++49/5458/93345-45 Mobile: ++49/172/5357999 eMail: ulw(a)ulmke.com Dipl.-Ing. Walter Ulmke e.K. AG Steinfurt HRA 4384

1 0

Re: 2.3.1 Replication is throwing scary errors
by Thore Bödecker 13 Jun '18

13 Jun '18

Aaaaand I forgot to CC the list, sorry for that, it's way too early in the morning :P On 07.06.18 - 07:39, Thore Bödecker wrote: > What does the output of these two commands show after that error has > been logged? > > doveadm replicator status > > doveadm replicator dsync-status > > If there are *waiting failed* requests, that never make it "through" > (after being temporarily in state *queued failed* and then returning > to *waiting failed*) this means there is something wrong with the > replication. > > You can try forcing replication of all known users using > > doveadm replicator replicate '*' > > And see if that resolves the failed requests, but I doubt it. > > Please let us know how your status outputs look like. > > > Cheers, > Thore > > -- > Thore Bödecker > > GPG ID: 0xD622431AF8DB80F3 > GPG FP: 0F96 559D 3556 24FC 2226 A864 D622 431A F8DB 80F3 > > > On 07.06.18 - 15:21, Reuben Farrelly wrote: > > Still not quite right for me. > > > > Jun 7 15:11:33 thunderstorm.reub.net dovecot: doveadm: Error: > > dsync(lightning.reub.net): I/O has stalled, no activity for 600 seconds > > (last sent=mail, last recv=mail (EOL)) > > Jun 7 15:11:33 thunderstorm.reub.net dovecot: doveadm: Error: Timeout > > during state=sync_mails (send=mails recv=recv_last_common) > > > > I'm not sure if there is an underlying replication error or if the message > > is just cosmetic, though. > > > > Reuben > > > > > > On 7/06/2018 4:55 AM, Remko Lodder wrote: > > > Hi Timo, > > > > > > Yes this seems to work fine so far. I’ll ask the people to add it to the > > > current FreeBSD version.. > > > > > > Cheers > > > Remko > > > > > > > On 6 Jun 2018, at 19:34, Timo Sirainen <tss(a)iki.fi > > > > <mailto:tss@iki.fi>> wrote: > > > > > > > > Should be fixed by https://github.com/dovecot/core/commit/a952e178943a5944255cb7c053d970f8e6d4… > > > > > > >

1 2

2.3.1: Core dump on quota reach
by Tom Sommer 12 Jun '18

12 Jun '18

Just upgraded from 2.2.34 to 2.3.1: Jun 12 16:06:38 lmtp(example(a)example.name)<46176><8JrHEW7TH1tgtAAAd9aEAw>: Info: msgid=<iOIOOFTVyrtapfaoLCJKLCGX2P21E2eVWiGnXPfSSik.ezvWGm_MVMKKL2i0I81WDbYtapg7RisY9PCL91ga0WQ(a)ketopasuse.date>: save failed to INBOX: The email account that you tried to reach is over quota (Mailbox is full). Jun 12 16:06:38 lmtp(example(a)example.name)<46176><8JrHEW7TH1tgtAAAd9aEAw>: Fatal: master: service(lmtp): child 46176 killed with signal 11 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) And also, unrelated: Jun 12 16:21:32 lmtp(53430): Error: lmtp-server: conn director.local:39204 [6]: Connection lost during data transfer: read(director.local:39204 [6]) failed: <no error> -- Tom

2 1

Race condition when setting flags (\Deleted) + expunge quickly, leaving mails not deleted
by Tomas Forsman 12 Jun '18

12 Jun '18

Hello. I seem to have found a race condition, when setting flags on multiple emails rapidly. 5 commands including login to reproduce. Problem found using mutt in real world usage. Seems to happen both with UID STORE 1:3 and UID STORE 1,2,3 .. I have tried with the following packages, with a minimized config in a throwaway vm: https://packages.debian.org/stretch/dovecot-core Package: dovecot-core (1:2.2.27-3+deb9u2) https://packages.debian.org/stretch-backports/dovecot-core Package: dovecot-core (1:2.2.34-2~bpo9+1) Also tested pristine: https://dovecot.org/releases/2.3/dovecot-2.3.0.1.tar.gz https://dovecot.org/releases/2.2/dovecot-2.2.34.tar.gz https://dovecot.org/releases/2.2/dovecot-2.2.35.tar.gz .. built on Debian9 with: # apt-get build-dep dovecot-core # bash configure --prefix=/scratch/dovecot-test Tried with both mailbox over NFS (backed by ext4), and also mailbox on ext4. On my test systems, I am only able to reproduce it with mbox_lazy_writes=no, but with production settings on a Ubuntu 14.04 with 1:2.2.9-1ubuntu2.4 that has mbox_lazy_writes=yes it is still reproducable with mutt. I have found two different ways to reproduce it with netcat + cut'n'paste, below is done with 2.2.34/35. All servers are VMs in Ganeti/kvm clusters (Debian hosts in one cluster&organization, Ubuntu host in another cluster&organization). Debian minimized config: > doveconf -n # 2.2.34 (874deae): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.22 (22940fb7) # OS: Linux 4.9.0-6-amd64 x86_64 Debian 9.4 # Hostname: mail2-vm.cs.umu.se auth_mechanisms = plain login auth_verbose = yes debug_log_path = syslog info_log_path = syslog mail_location = mbox:~/Mail:INBOX=/var/mail/%u mail_privileged_group = mail mbox_lazy_writes = no namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } protocols = imap service auth { client_limit = 2500 unix_listener auth-client { group = postfix mode = 0660 } } service imap-login { inet_listener imap { port = 143 } } service imap { executable = imap } ssl = no syslog_facility = local1 userdb { driver = passwd } Attached is a mailbox that can be used to reproduce the problem, called "error-box", anonymized with mbox-anonymize.pl. It was created by sending test messages with mutt 1.7.2 through Dovecot/Postfix/Amavis. **** Method 1, with IDLE: Setup a local account, MY-USER / MY-PASSWORD (replace below). > mkdir Mail > cp error-box Mail/error1 > nc -v localhost 143 Paste the following 3 lines in a go: a0000 LOGIN "MY-USER" "MY-PASSWORD" a0010 SELECT "error1" a0020 IDLE then wait a second.. then paste the following 3 in a go: DONE a0030 UID STORE 1:3 +FLAGS (\Deleted) a0040 EXPUNGE Notice that the results from UID STORE + EXPUNGE gives: * 2 FETCH (UID 2 FLAGS (\Recent)) * 3 FETCH (UID 3 FLAGS (\Recent)) * 1 EXPUNGE * 3 RECENT a0030 OK Store completed (0.004 + 0.000 + 0.003 secs). a0040 OK Expunge completed (0.004 + 0.000 + 0.003 secs). Verify what's left in the mailbox with: a0050 UID FETCH 1:4 FLAGS * 1 FETCH (UID 2 FLAGS (\Recent)) * 2 FETCH (UID 3 FLAGS (\Recent)) * 3 FETCH (UID 4 FLAGS (\Recent)) a0050 OK Fetch completed (0.001 + 0.000 secs). Since 1 to 3 should be \Deleted and then EXPUNGEd, we should only have UID 4 left, not uid 2-4 without Deleted flag. Try same again, but with 1 inserted delay (use a new mailbox name for every test, otherwise UID numbering will vary, indexes etc): > cp error-box Mail/error2 > nc -v localhost 143 Paste the following 3 lines in a go: a0000 LOGIN "MY-USER" "MY-PASSWORD" a0010 SELECT "error2" a0020 IDLE then wait a second.. then paste 1 line: DONE then [newly introduced] wait a second.. then paste 2 lines: a0030 UID STORE 1:3 +FLAGS (\Deleted) a0040 EXPUNGE Notice that the results from UID STORE + EXPUNGE gives: * 1 FETCH (UID 1 FLAGS (\Deleted \Recent)) * 2 FETCH (UID 2 FLAGS (\Deleted \Recent)) * 3 FETCH (UID 3 FLAGS (\Deleted \Recent)) a0030 OK Store completed (0.003 + 0.000 + 0.002 secs). * 3 EXPUNGE * 2 EXPUNGE * 1 EXPUNGE * 1 RECENT a0040 OK Expunge completed (0.003 + 0.000 + 0.002 secs). Verify what's left in the mailbox with: a0050 UID FETCH 1:4 FLAGS * 1 FETCH (UID 4 FLAGS (\Recent)) a0050 OK Fetch completed (0.001 + 0.000 secs). So if I wait some between DONE and UID STORE, it seems to be ok. If I, in the problem-triggering method, change a0030 UID STORE 1:3 +FLAGS (\Deleted) a0040 EXPUNGE to a0030 UID STORE 1:3 +FLAGS (\Deleted) a0035 UID FETCH 1:4 FLAGS a0040 EXPUNGE The untagged responses become: * 1 FETCH (UID 1 FLAGS (\Deleted \Recent)) * 2 FETCH (UID 2 FLAGS (\Deleted \Recent)) * 3 FETCH (UID 3 FLAGS (\Deleted \Recent)) * 4 FETCH (UID 4 FLAGS (\Recent)) * 2 FETCH (UID 2 FLAGS (\Recent)) * 3 FETCH (UID 3 FLAGS (\Recent)) * 1 EXPUNGE * 3 RECENT Full transcript (in&out from screen) from first, failing, run: localhost [127.0.0.1] 143 (imap2) open * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot (Debian) ready. a0000 LOGIN "MY-USERNAME" "MY-PASSWORD" a0010 SELECT "error1" a0020 IDLE a0000 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY] Logged in * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 4 EXISTS * 4 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1520944853] UIDs valid * OK [UIDNEXT 5] Predicted next UID a0010 OK [READ-WRITE] Select completed (0.003 + 0.000 + 0.004 secs). + idling DONE a0030 UID STORE 1:3 +FLAGS (\Deleted) a0040 EXPUNGE a0020 OK Idle completed (7.805 + 7.804 + 7.804 secs). * 2 FETCH (UID 2 FLAGS (\Recent)) * 3 FETCH (UID 3 FLAGS (\Recent)) * 1 EXPUNGE * 3 RECENT a0030 OK Store completed (0.004 + 0.000 + 0.003 secs). a0040 OK Expunge completed (0.004 + 0.000 + 0.003 secs). a0050 UID FETCH 1:4 FLAGS * 1 FETCH (UID 2 FLAGS (\Recent)) * 2 FETCH (UID 3 FLAGS (\Recent)) * 3 FETCH (UID 4 FLAGS (\Recent)) a0050 OK Fetch completed (0.001 + 0.000 secs). Full transcript (in&out from screen) from the second, slower, run: localhost [127.0.0.1] 143 (imap2) open * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot (Debian) ready. a0000 LOGIN "MY-USERNAME" "MY-PASSWORD" a0010 SELECT "error2" a0020 IDLE a0000 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY] Logged in * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 4 EXISTS * 4 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1520944957] UIDs valid * OK [UIDNEXT 5] Predicted next UID a0010 OK [READ-WRITE] Select completed (0.003 + 0.000 + 0.004 secs). + idling DONE a0020 OK Idle completed (2.830 + 2.829 + 2.829 secs). a0030 UID STORE 1:3 +FLAGS (\Deleted) a0040 EXPUNGE * 1 FETCH (UID 1 FLAGS (\Deleted \Recent)) * 2 FETCH (UID 2 FLAGS (\Deleted \Recent)) * 3 FETCH (UID 3 FLAGS (\Deleted \Recent)) a0030 OK Store completed (0.003 + 0.000 + 0.002 secs). * 3 EXPUNGE * 2 EXPUNGE * 1 EXPUNGE * 1 RECENT a0040 OK Expunge completed (0.003 + 0.000 + 0.002 secs). a0050 UID FETCH 1:4 FLAGS * 1 FETCH (UID 4 FLAGS (\Recent)) a0050 OK Fetch completed (0.001 + 0.000 secs). Transcript from .muttdebug0 when using mutt 1.7.2 against the production server with NFS backed storage, marking message 1-3 for deletion and hitting $ for sync. This is how the problem was first found. [2018-03-12 13:46:28] 5> a0020 IDLE^M [2018-03-12 13:46:28] 5< + idling ... [2018-03-12 13:46:29] Marking 3 messages deleted... [2018-03-12 13:46:29] Expunging messages from server... [2018-03-12 13:46:29] 5> DONE^M a0021 UID STORE 1:3 +FLAGS.SILENT (\Deleted)^M a0022 EXPUNGE^M [2018-03-12 13:46:29] 5< a0020 OK Idle completed (0.001 + 1.634 + 1.633 secs). [2018-03-12 13:46:29] 5< * 2 FETCH (FLAGS (\Recent)) [2018-03-12 13:46:29] Handling FETCH [2018-03-12 13:46:29] FETCH response ignored for this message [2018-03-12 13:46:29] 5< * 3 FETCH (FLAGS (\Recent)) [2018-03-12 13:46:29] Handling FETCH [2018-03-12 13:46:29] FETCH response ignored for this message [2018-03-12 13:46:29] 5< * 1 EXPUNGE [2018-03-12 13:46:29] Handling EXPUNGE [2018-03-12 13:46:29] 5< * 3 RECENT [2018-03-12 13:46:29] 5< a0021 OK Store completed (0.001 + 0.000 + 0.001 secs). [2018-03-12 13:46:29] 5< a0022 OK Expunge completed (0.001 + 0.000 secs). [2018-03-12 13:46:29] IMAP queue drained [2018-03-12 13:46:29] imap_cmd_finish: Expunging mailbox *** Method 2, without IDLE (seems to require that I fetch flags before the wait/store) Copy'n'paste the following: a0000 LOGIN "MY-USER" "MY-PASSWORD" a0010 SELECT "error20" a0015 FETCH 1:4 (UID FLAGS) <wait a second, then paste the next two in one go> a0020 UID STORE 1:3 +FLAGS (\Deleted) a0025 EXPUNGE Gives the following transcript: mail3-vm:~/Mail> nc -v localhost 143 localhost [127.0.0.1] 143 (imap2) open * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready. a0000 LOGIN "MY-USER" "MY-PASSWORD" a0010 SELECT "error20" a0015 FETCH 1:4 (UID FLAGS) a0000 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY] Logged in * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 4 EXISTS * 4 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1521552107] UIDs valid * OK [UIDNEXT 5] Predicted next UID a0010 OK [READ-WRITE] Select completed (0.001 + 0.000 + 0.001 secs). * 1 FETCH (UID 1 FLAGS (\Recent)) * 2 FETCH (UID 2 FLAGS (\Recent)) * 3 FETCH (UID 3 FLAGS (\Recent)) * 4 FETCH (UID 4 FLAGS (\Recent)) a0015 OK Fetch completed (0.001 + 0.000 secs). a0020 UID STORE 1:3 +FLAGS (\Deleted) a0025 EXPUNGE * 2 FETCH (UID 2 FLAGS (\Recent)) * 3 FETCH (UID 3 FLAGS (\Recent)) * 1 EXPUNGE * 3 RECENT a0020 OK Store completed (0.001 + 0.000 secs). a0025 OK Expunge completed (0.001 + 0.000 secs). /Tomas -- Tomas Forsman, stric(a)cs.umu.se, http://people.cs.umu.se/stric/ `- SysAdmin at Computing Science, University of Umeå

2 3

outlook hangs using TLS
by Vladimir Tiukhtin 11 Jun '18

11 Jun '18

Hello I am using dovecot 2.2.10 on CentOS 7 Any Outlook versions (2007, 2010, 2013...) hang if I tried to use TLS, it works if I switch in client TLS to SSL. Thunderbird works perfect both scenarios Please find debug log mail dovecot[24287]: imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [X.X.X.X] mail dovecot[24287]: imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [X.X.X.X] mail dovecot[24287]: imap-login: Debug: SSL: where=0x2002, ret=-1: SSLv2/v3 read client hello A [X.X.X.X] mail dovecot[24287]: imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges mail dovecot[24287]: imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges mail dovecot[24287]: auth: Debug: auth client connected (pid=24300) mail dovecot[24287]: imap-login: Disconnected (no auth attempts in 31 secs): user=<>, rip=X.X.X.X, lip=X.X.X.X, TLS handshaking: Disconnected, session=<bivt8iNuBgA+A08O> Please find my config # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-862.3.2.el7.x86_64 x86_64 CentOS Linux release 7.5.1804 (Core) auth_cache_size = 16 M auth_cache_ttl = 1 days auth_debug = yes auth_mechanisms = plain login auth_username_chars = abcdefghijklmnopqrstuvwxyz.@ auth_verbose = yes default_client_limit = 1024 default_process_limit = 16 doveadm_password = mysecretpasswordsharedamongservers first_valid_uid = 1000 mail_attachment_dir = /srv/attachments mail_attachment_min_size = 4 k mail_debug = yes mail_home = /var/spool/mail/%d/%n mail_location = mdbox:~/mail mail_plugins = replication notify mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Spam { auto = create special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } mailbox virtual/All { auto = no special_use = \All } prefix = type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap lmtp service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = $default_internal_user } unix_listener replication-notify { mode = 0666 user = $default_internal_user } } service auth { unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 55555 } } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 2 service_count = 1 } service imap { client_limit = 0 } service lmtp { unix_listener lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = </etc/letsencrypt/live/mail.example.com/fullchain.pem ssl_dh_parameters_length = 2048 ssl_key = </etc/letsencrypt/live/mail.example.com/privkey.pem ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 TLSv1 TLSv1.1 TLSv1.2 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_ssl = yes protocol imap { imap_client_workarounds = tb-extra-mailbox-sep delay-newmail }

4 6

Doveadm sync: Can't create mailbox mail_public/public: Permission denied if synchronize all the available namespaces
by vkosharskiy＠gmail.com 09 Jun '18

09 Jun '18

I have old dovecot server: (# 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.21 (92477967) # OS: Linux 4.14.15-3.el7xen.x86_64 x86_64 CentOS Linux release 7.4.1708 (Core) and new dovecot server: # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.devel (61b47828) # OS: Linux 4.14.47-5.el7xen.x86_64 x86_64 CentOS Linux release 7.5.1804 (Core) I try to sync old server with doveadm sync with parameter -N (Synchronize all the available namespaces) Sync successfull in log, but destination server in log have "Can't create mailbox mail_public/public: Permission denied." Config dumps, logs in attaches.

1 0

Re: 2.3.1 Replication is throwing scary errors
by Reuben Farrelly 09 Jun '18

09 Jun '18

Hi Andy, Funny you say that - I've been testing this very problem for some time today to narrow down the commits where I think the problem started occurring. So far I have been able to roll forward to commit from git: 74ef8506dcb946acc42722683e49fdcb964eed19 >> "doveadm: Unref header search context after use" . So far I have been running on that code for a few hours and I'm not yet seeing the problem, which indicates the issue is probably subsequent to this. I'm of the view that the problem started occurring somewhere between that commit above on 6 March and the final 2.3.1 release on 27 March. There are a lot of changes between those two dates, and testing to narrow down to one specific commit is extremely time consuming. It would be really helpful if even one of the Dovecot developers could comment on this and give us some direction (or candidate commits to investigate) or just let us know if the root cause has been found. This bug is a showstopper and has stopped me tracking master-2.3 for over 3 months now, as I can't test later builds or even upgrade to the 2.3.1 release while replication is so broken. Reuben > Message: 1 > Date: Sun, 6 May 2018 13:21:57 +1000 > From: Andy Weal <andy(a)bizemail.com.au> > To: dovecot(a)dovecot.org > Subject: 2.3.1 Replication is throwing scary errors > Message-ID: <f4274128-350b-9a74-a344-8e6caeffa112(a)bizemail.com.au> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > Hi all, > > New to the mailing lists but have joined up because of above */2.3.1 > Replication is throwing scary errors > > > /*Brief system configuration > ??? MX1 - Main > ??? ??? Freebsd 11.1-Release-p9 > ??? ??? Hosted on a Vultr VM in Sydney AU > ??? ??? MTA = Postfix 3.4-20180401 > ??? ??? Dovecot = 2.3.1 > ??? ??? File system = ufs > ??? MX2 - Backup > ??? ??? Freebsd 11.1-Release-p9 > ??? ???? Running on bare metal - no VM or jails > ??? ??? MTA = Postfix 3.4-20180401 > ??? ??? Dovecot = 2.3.1 > ??? ??? File system = ufs ( on SSD) > > */ > /*Brief sequence of events > > 1. ??? apx 10 days back upgraded both mx1 and mx2 to dovecot 2.3.1_2 > from 2.3.0 ? (service dovecot stop, portmaster upgrade, service > dovecot start) > 2. ??? both systems ran ok with no errors for 10 days. > 3. ??? Last night I shutdown mx2 and restarted it a few hours later > 4. ??? within minutes i was getting the following types of errors on mx2 > > ???????? May 06 12:56:29 doveadm: Error: Couldn't lock > /var/mail/vhosts/example.net/user1/.dovecot-sync.lock: > fcntl(/var/mail/vhosts/example.net/user1/.dovecot-sync.lock, write-lock, > F_SETLKW) locking failed: Timed out after 30 seconds (WRITE lock held by > pid 1960) > > ? Before i venture down the rabbit hole of fault finding and excess > coffee consumption I was wondering if any of you had any updates on the > problems discussed below. > > > Cheers for now, > Andy > > > > Hi, > > [Formatting is a bit rough, replying from a trimmed digest email] > >> /Message: 1 />/Date: Fri, 6 Apr 2018 15:04:35 +0200 />/From: Michael Grimm <trashcan at ellael.org > <https://dovecot.org/mailman/listinfo/dovecot>> />/To: Dovecot Mailing List <dovecot at dovecot.org > <https://dovecot.org/mailman/listinfo/dovecot>> />/Subject: Re: 2.3.1 Replication is throwing scary errors />/Message-ID: <E7E7A927-68F8-443F-BA59-E66CED8FE878 at ellael.org > <https://dovecot.org/mailman/listinfo/dovecot>> />/Content-Type: text/plain; charset=utf-8 />//>/Reuben Farrelly wrote: />>/From: Michael Grimm <trashcan at ellael.org > <https://dovecot.org/mailman/listinfo/dovecot>> />//>>>/[This is Dovecot 2.3.1 at FreeBSD STABLE-11.1 running in two jails at > distinct servers.] />>>/I did upgrade from 2.2.35 to 2.3.1 today, and I do become pounded by > error messages at server1 (and vice versa at server2) as follows: />>>/| Apr 2 17:12:18 <mail.err> server1.lan dovecot: doveadm: Error: > dsync(server2.lan): I/O has stalled, \ />>>/no activity for 600 seconds (last sent=mail_change, last > recv=mail_change (EOL)) />>>/| Apr 2 17:12:18 <mail.err> server1.lan dovecot: doveadm: Error: Timeout > during state=sync_mails \ />>>/(send=changes recv=mail_requests) />>>/[?] />>>/| Apr 2 18:59:03 <mail.err> server1.lan dovecot: doveadm: Error: > dsync(server2.lan): I/O has stalled, \ />>>/no activity for 600 seconds (last sent=mail, last recv=mail (EOL)) />>>/| Apr 2 18:59:03 <mail.err> server1.lan dovecot: doveadm: Error: Timeout > during state=sync_mails \ />>>/(send=mails recv=recv_last_common) />>>/I cannot see in my personal account any missing replications, *but* I > haven't tested this thoroughly enough. I do have customers being > serviced at these productive servers, *thus* I'm back to 2.2.35 until I > do understand or have learned what is going on. />//>/In my reply to this statement of mine I mentioned that I have seen those > timeouts quite some times during the past year. Thus, I upgraded to > 2.3.1 again, and boom: after some hours I ended up in hanging processes > [1] like (see Remko's mail in addition) ... />//>/doveadm-server: [IP4/6 <USER1> SOME/MAILBOX import:0/0] (doveadm-server) />//>/? at server2 paired with a file like ? />//>/-rw------- 1 vmail dovecot uarch 0 Apr 3 16:52 > /home/to/USER1/.dovecot-sync.lock />//>/Corresponding logfile entries at server2 are like ? />//>/Apr 3 17:10:49 <mail.err> server2.lan dovecot: doveadm: Error: Couldn't > lock /home/to/USER1/.dovecot-sync.lock: \ />/fcntl(/home/to/USER1/.dovecot-sync.lock, write-lock, F_SETLKW) locking > failed: Timed out after 30 seconds \ />/(WRITE lock held by pid 51110) />//>/[1] Even stopping dovecot will not end those processes. One has to > manually kill those before restarting dovecot. />//>/After one day of testing 2.3.1 with a couple of those episodes of > locking/timeout, and now missing mails depending with server your MUA > will connect to, I went back to 2.2.35. After two days at that version I > never had such an episode again. />//>>/It's not just you. This issue hit me recently, and it was impacting />>/replication noticeably. I am following git master-2.3 . />/[...] />>/There is also a second issue of a long standing race with replication />>/occurring somewhere whereby if a mail comes in, is written to disk, is />>/replicated and then deleted in short succession, it will reappear />>/again to the MUA. I suspect the mail is being replicated back from />>/the remote. A few people have reported it over the years but it's not />>/reliable or consistent, so it has never been fixed. />>/And lastly there has been an ongoing but seemingly minor issue />>/relating to locking timing out after 30s particularly on the remote />>/host that is being replicated to. I rarely see the problem on my />>/local disk where almost all of the mail comes in, it's almost always />>/occurring on the replicate/remote system. />//>/It might be time to describe our setups in order to possibly find co > mmon > grounds that might trigger this issue you describe and Rimko and myself > ran into as well. />//>/Servers: Cloud Instances (both identical), around 25ms latency apart. />/Intel Core Processor (Haswell, no TSX) (3092.91-MHz K8-class CPU) />/Both servers are connected via IPsec/racoon tunnels />/OS: FreeBSD 11.1-STABLE (both servers) />/Filesystem: ZFS />/MTA: postfix 3.4-20180401 (postfix delivers via dovecot's LMTP) />/IMAP: dovecot running in FreeBSD jails (issues with 2.3.1, fine with > 2.2.35) />/Replication: unsecured tcp / master-master />/MUA: mainly iOS or macOS mail.app, rarely roundcube / > For me: > > Servers: Main: VM on a VMWare ESXi local system (light > load), local SSD disks (no NFS) > Redundant: Linode VM in Singapore, around 250ms away > Also no NFS. Linode use SSDs for IO. > There is native IPv6 connectivity between both VMs. > As I am using TCPs I don't have a VPN between them - > just raw IPv6 end to end. > OS: Gentoo Linux x86_64 kept well up to date > Filesystem: EXT4 for both > MTA: Postfix 3.4-x (same as you) > IMAP: Dovecot running natively on the machine (no jail/chroot) > Replication: Secured TCP / master-master (but only one master is > actively used for SMTP and MUA access) > MUA: Thunderbird, Maildroid, MS Mail, Gmail client, > Roundcube, LG Mail client on handset etc > >> /I believe it is worthwhile to mention here that I run a poor man's > fail-over approach (round-robin DNS) as follows: />//>/DNS: mail.server.tld resolves to one IP4 and one IP6 address of each > server, thus 4 IP addresses in total / > For me I have a simple pair of servers that replicate mail to each > other. Clients connect to the main server only, which is where mail > comes in. The second one that experiences the problem with locking is > only a redundant server which is a secondary MX, that only gets > occasional SMTP deliveries from time to time. So the amount of > replication back from the "secondary" is very minimal; the vast bulk of > the replication is the primary replicating changes out. > > Reuben >

8 20

sieve, pigeonhole; reject message
by dovecot-user 08 Jun '18

08 Jun '18

Hello everybody. I use the version of dovecot 2.3.0.1 and postfix 3.3.0 on my test computer. Postfix pass incoming messages to Dovecot via the LMTP protocol. I want to understand how to fine tune the "reject" action in sieve-scripts. The task: in my installation I use the translation of a set of aliases into the (hidden) target email address inside postfix (postfix directives: canonical_classes = envelope_recipient; canonical_maps = hash://etc/postfix/canonical), so in dovecot the mails fall into one mailbox; However, the original headers (for example, the alias of the recipient) can be extracted with sieve scripts (the "Received" header). In my case, when I use the "reject" action in the sieve script, dovecot generates a message to the sender with the text: "Your message to <hidden(a)domain.tld> was automatically rejected ..", but this behavior opens the address to the sender of what is hidden behind the alias. Is it possible to redefine the text of the reject message using the extracting values from the "Received" header so that the email address hidden behind the alias does not appear there? Or some other way to customize the reject-message? Thank you.

1 0

dovecot-pigeonhole Missing sieve_imapsieve Plugin?
by Will Foster 07 Jun '18

07 Jun '18

Hi Folks, I am having a problem configuring imapsieve via dovecot-pidgeonhole on CentOS7. After following the docs here: https://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve I get: dovecot[22026]: managesieve: Fatal: Plugin 'sieve_imapsieve' not found from directory /usr/lib64/dovecot/sieve dovecot[22026]: doveconf: Error: managesieve-login: dump-capability process returned 89 There doesn't seem to be that plugin present or provided by dovecot-pigeonhole # ls -lah /usr/lib64/dovecot/sieve/ total 56K drwxr-xr-x. 2 root root 46 Jun 5 12:16 . drwxr-xr-x. 7 root root 4.0K Jun 5 12:16 .. -rwxr-xr-x. 1 root root 52K Aug 3 2017 lib90_sieve_extprograms_plugin.so Am I doing something wrong? This is provided via the CentOS EPEL package Here's my system details: * dovecot-pigeonhole-2.2.10-8.el7.x86_64 * CentOS 7.5 on XFS * Rainloop Webmail / Mutt / IMAP access Here's my dovecot -n ------------ # 2.2.10: /etc/dovecot/dovecot.conf managesieve(root): Fatal: Plugin 'sieve_imapsieve' not found from directory /usr/lib64/dovecot/sieve doveconf: Error: managesieve-login: dump-capability process returned 89 # OS: Linux 3.10.0-862.2.3.el7.x86_64 x86_64 CentOS Linux release 7.5.1804 (Core) first_valid_uid = 1000 info_log_path = /var/log/dovecot-info.log log_path = /var/log/dovecot.log mail_location = mbox:~/Mail:INBOX=/home/%u/Mail/INBOX_MAIL mail_privileged_group = mail mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = %s driver = pam } passdb { driver = pam } plugin { imapsieve_url = sieve://localhost:4190 sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /etc/dovecot/sieve-after sieve_dir = ~/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_user_log = ~/.dovecot.sieve.log } protocols = imap sieve sieve service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } ssl = required ssl_cert = </etc/pki/tls/certs/example.net.crt ssl_key = </etc/pki/tls/private/example.net.key userdb { driver = passwd } userdb { driver = passwd } protocol lmtp { mail_plugins = " sieve" } protocol lda { mail_plugins = " sieve" } protocol imap { mail_plugins = " imap_sieve" } ------------- Thanks for any guidance here. -- @sadsfae // gpg: A31F27E0 // irc: sadsfae come have a squat at https://hobo.house

2 2

Re: 2.3.1 Replication is throwing scary errors
by Michael Grimm 07 Jun '18

07 Jun '18

Am 2018-06-07 08:48, schrieb Remko Lodder: > On Thu, Jun 07, 2018 at 08:04:49AM +0200, Michael Grimm wrote: >> Conclusion: After 12 hours of running a patched FBSD port I do get >> those >> error messages but replictaion seems to work now. But, I still have >> the >> feeling that there might something else going wrong. > > I agree with that. Are you using the new pkg that ler@ prepared ? That > includes > the patch and is a 'native' package.. Yes, I am running this new port from ler@. And: Thanks for his very fast modification! Regards, Michael

1 0

Shared mailboxes, index files and 'per-user-seen' flags
by Thomas Robers 07 Jun '18

07 Jun '18

Hello, i have a dovecot server version 2.3.1 under CentOS 6.9 and we're using shared mailboxes with index files shared. With this configuration I can see a lot of error messages like: Jun 6 13:20:31 mail dovecot: Error: imap(userA(a)tutech.de)<4513> <onjjWPdtDuLAqG3R>: /export/home/imap/userA(a)tutech.de/shared /userB(a)tutech.de/folder/dovecot.index.pvt view is inconsistent In 10-mail.conf the location setting is: location = maildir:%%h/Maildir:INDEXPVT=%h/shared/%%u I thought setting the index files to "not shared" might help to get rid of the errors, so I changed the setting to: location = maildir:%%h/Maildir:INDEX=%h/shared/%%u:INDEXPVT=%h /shared/%%u like it's mentioned in the Dovecot wiki. But that doesn't work as I expected, because the 'per-user-seen' flags do not work correctly anymore, i think. If UserA, who has UserB as shared mailbox, changes the seen flags of UserBs INBOX, UserBs seen flags are also changed. The other way, if UserB changes seen flags in his INBOX they are not changed in the shared view of UserA. Is this the supposed way to work or do i have an error in the configuration? Any help is appreciated. Thanks, Thomas. Here's my currently used configuration: # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.devel (61b47828) # OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final) ext4 # Hostname: mail.tutech.de auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = 12345 imap_max_line_length = 2 M mail_debug = yes mail_location = maildir:/export/home/imap/%Lu/Maildir mail_plugins = acl zlib mail_log notify mail_prefetch_count = 1 mailbox_idle_check_interval = 10 secs managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { hidden = no ignore_on_failure = no inbox = no list = children location = maildir:%%h/Maildir:INDEXPVT=%h/shared/%%u prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = file:/export/home/shared-db/shared-mailboxes mail_log_events = append delete undelete expunge copy mailbox_delete mailbox_rename flag_change mail_log_fields = uid box msgid size from flags mail_replica = tcp:mail2.tutech.de sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global = /var/lib/dovecot/sieve/global/ sieve_user_log = ~/.dovecot.sieve.log zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 lmtp sieve sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service config { unix_listener config { user = vmail } } service doveadm { inet_listener { port = 12345 } user = vmail } service imap-login { inet_listener imaps { port = 993 ssl = yes } process_limit = 500 process_min_avail = 20 } service imap { executable = imap } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3s { port = 995 ssl = yes } } service pop3 { executable = pop3 } service replicator { unix_listener replicator-doveadm { mode = 0666 } } ssl = required ssl_cert = </etc/pki/dovecot/certs/mail.tutech.de.crt_chain ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL:!EXPORT ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it syslog_facility = local6 userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf.ext driver = ldap } protocol lmtp { mail_plugins = acl zlib mail_log notify sieve } protocol imap { mail_max_userip_connections = 100 mail_plugins = acl zlib mail_log notify imap_zlib imap_acl rawlog_dir = /tmp/rawlog/%u }

2 1

Header DATE field not returned when placed after MIME section
by Xavier Guerin 06 Jun '18

06 Jun '18

Hello, I am using Dovecot 2.2.34p0 on OpenBSD 6.3 stable. I am writting an application using the vmime (http://www.vmime.org) library (x-ref issue: https://github.com/kisli/vmime/issues/199) I noticed through testing over my own e-mail server that in some situations the DATE field is not returned upon ENVELOPE or HEADER.FIELDS request. The e-mails at fault happen to have the DATE field at the very end of the header, after the MIME section. Example 1: Delivered-To: Received: Return-Path: Delivered-To: DKIM-Signature: From: To: Message-ID: Subject: MIME-Version: 1.0 Content-Type: text/html;charset=UTF-8 Content-Transfer-Encoding: 7bit Date: Fri, 01 Jun 2018 17:44:44 -0400 Example 2: Return-Path: Delivered-To: Received: Delivered-To: DKIM-Signature: From: Reply-To: To: Subject: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_338_1573405959.1527768044781" X-Mailer: EMail_2.0 Date: Fri, 01 Jun 2018 17:36:01 -0400 Thanks in advance for the insight,

3 2

offtopic: Solr compatible IMAP client
by Andy Smith 04 Jun '18

04 Jun '18

Hi, not sure if I'm allowed to go this off topic on here, sorry if not! I've just installed Solr FTS for Dovecot and its great, as is Dovecot! ;) It works perfectly for fast searches using Roundcube, but can anyone suggest any Windows and/or Linux mail clients that work well with IMAP searches rather than doing their own searches on local copies of mails? I've had a look at Thunderbird and you can use IMAP search, buy you have to manually select it from an advanced search dialogue and it seems you cannot make this the default behaviour. I also tried OperaMail but it doesn't appear to work at all (only header is searchable without downloading local copies). I'm pretty sure Outlook will only use its own search engine on its local copy of data. thanks a lot, Andy. PS this guide was really helpful for setting up Solr FTS with a recent release of Solr (I'm using 6.5), might be worth including a link somewhere on the Dovecot wiki?: http://mor-pah.net/2016/08/15/dovecot-2-2-with-solr-6-or-5

4 7

Re: SSL error after upgrading to 2.31
by Aki Tuomi 01 Jun '18

01 Jun '18

You forgot to cc the list. ssl_ca is used only for validating client certificates. ---Aki TuomiDovecot oy -------- Original message --------From: Marc Perkel <marc(a)perkel.com> Date: 21/05/2018 18:25 (GMT+02:00) To: Aki Tuomi <aki.tuomi(a)dovecot.fi> Subject: Re: SSL error after upgrading to 2.31 On 05/21/2018 07:54 AM, Aki Tuomi wrote: Does ssl_cert file contain intermediates? No - but the ssl_ca does. --- Aki Tuomi Dovecot oy -------- Original message -------- From: Marc Perkel <marc(a)perkel.com> Date: 21/05/2018 16:32 (GMT+02:00) To: dovecot(a)dovecot.org Subject: SSL error after upgrading to 2.31 After upgrading to 2.31 I'm getting this error. Not sure what I'm doing wrong. No (No signatures could be verified because the chain contains only one certificate and it is not self signed.) ssl = yes ssl_cert = </etc/exim/certs/ctyme.com.crt ssl_key = </etc/exim/certs/ctyme.com.key ssl_ca = </etc/exim/certs/ca.crt local mail.ctyme.com { protocol imap { ssl_cert = </etc/exim/certs/ctyme.com.crt ssl_key = </etc/exim/certs/ctyme.com.key ssl_ca = </etc/exim/certs/ca.crt } protocol pop3 { ssl_cert = </etc/exim/certs/ctyme.com.crt ssl_key = </etc/exim/certs/ctyme.com.key ssl_ca = </etc/exim/certs/ca.crt } } local mail.junkemailfilter.com { protocol imap { ssl_cert = </etc/exim/certs/junkemailfilter.com.crt ssl_key = </etc/exim/certs/junkemailfilter.com.key ssl_ca = </etc/exim/certs/ca.crt } protocol pop3 { ssl_cert = </etc/exim/certs/junkemailfilter.com.crt ssl_key = </etc/exim/certs/junkemailfilter.com.key ssl_ca = </etc/exim/certs/ca.crt } }

3 9

Struggling with sieve, fileinto and variables.
by Barry Pearce 31 May '18

31 May '18

Hi, Im on Manjaro linux fully up to date running dovecot 2.3.1-2 pigeonhole 0.5.1-2 All is running well except I am having problems with fileinto - it refuses to use variables, and mailboxexists is also failing for me. Im just dealing with plus addressing - should be simple but the behaviour Im experiencing isnt right. require ["variables", "envelope", "fileinto", "subaddress", "mailbox"]; if envelope :matches :detail "to" "*" { set :lower "name" "${1}"; } fileinto :create "folder3"; This works, but changing the last line to: fileinto :create "${name}"; fails and files into inbox. As does: fileinto :create "${1}"; It makes no difference if the fileinto is placed inside the if statement or otherwise. Using sieve-test suggests the basic script it should work and traces show the right action. On a related matter the script Im actually trying to get to work is: require ["variables", "envelope", "fileinto", "subaddress", "mailbox"]; if envelope :matches :detail "to" "*" { set :lower "name" "${1}"; } if mailboxexists "${name}" { fileinto :create "${name}"; } But fails and files in the inbox. Modifying the script to: require ["variables", "envelope", "fileinto", "subaddress", "mailbox"]; if envelope :matches :detail "to" "*" { set :lower "name" "${1}"; } if mailboxexists "${name}" { fileinto :create "folder7"; } else { fileinto :create "folder8"; } Files into folder8. So the mailboxexists is failing. ## Started executing script 'plus-addressing' 3: envelope test 3: starting `:matches' match with `i;ascii-casemap' comparator: 3: getting `to' part from message envelope 3: extracting `detail' part from address <test+folder4(a)test.net> 3: matching value `folder4' 3: with key `*' => 1 3: finishing match with result: matched 3: jump if result is false 3: not jumping 4: set command 7: modify :lower "folder4" => "folder4" 7: assign `name' [0] = "folder4" 7: mailboxexists test 7: mailbox `folder4' cannot be opened 7: some mailboxes are unavailable 7: jump if result is false 7: jumping to line 9 10: fileinto action 10: store message in mailbox `folder8' ## Finished executing script 'plus-addressing' Here folder4 actually does exist - so the sieve-test confirms that issue. Im at a loss as to whats going on. Im particularly perplexed by the difference in behaviour between sieve-test and the result under the live server. Any ideas?

2 3

strange misflagging of incoming mail as trash
by Robb Aley Allan 31 May '18

31 May '18

Setting up postfix/dovecot on a clean FreeBSD cloud server, and have discovered a strange problem resulting from the interaction of Mac OS X Mail and dovecot (using the Maildir file structure). Essentially, under very specific circumstances, Mail somehow tells dovecot that a new incoming message is trash, dovecot marks it as such, and the message effectively vanishes. The specific circumstance is as follows: 1. A message is sent to a server running postfix. It is placed in the Maildir mail structure in the “new” directory. 2. OS X Mail is used to retrieve mail via IMAP, and dovecot transfers the message from “new” to “cur”. 3. OS X Mail has a rule that seeks to transfer an incoming message to a subfolder if the message sender belongs to a group created in the Contacts app (IOW the sender is matched against an OS-internal contacts database). 4. IF the sender is the user, AND the user is in the group against which the rule is compared, the message is marked as trash (suffix “T”) and is never loaded into the mail client, either in the inbox or in the trash folder. For all intents and purposes, it disappears. The connection log shows the following line: "WROTE May 30 23:38:11.960 [kCFStreamSocketSecurityLevelTLSv1_2] -- host:cloud.helical.com -- port:993 -- socket:0x6080004b6c80 -- thread:0x604000678e40 12.3 UID STORE 125 +FLAGS.SILENT (\Deleted)” which appears to be the client instructing dovecot to mark the message as deleted. I have the log files from the connection on both sides, and the console log from the Mac, but can’t understand 1. how Mail decides that the message should be deleted, and 2. why Mail “loses” the message (doesn’t appear in the Trash mailbox or anywhere else). _____ Robb Aley Allan

1 0

imapsieve: mailbox Trash: FLAG event
by Peter Schiffer 30 May '18

30 May '18

Hi all, after upgrade from Dovecot 2.2 to 2.3 I started to see that emails from Trash folder are automatically deleted after ~7 days. Email clients are not configured to automatically expunge or empty trash on log out (I see this behavior for multiple mailboxes, with different clients). With mail_debug enabled, I see this log message for every email that gets deleted from Trash: May 29 08:02:17 hostname dovecot: imap(some@mail)<7725><xxb6+VFtL+JT0CE 8>: Debug: imapsieve: mailbox Trash: FLAG event (changed flags: \Deleted) I have Dovecot 2.3.1 (8e2f634) and Pigeonhole 0.5.1 for sieve. My configuration can be found here: https://github.com/pschiffe/mailcow-dockerized/tree/master/data/conf/dovecot There is enabled support for sieve config in sql, but all sieve tables are empty, and there are no sieve filters for Trash flagging. Do you know how to find the source of that Trash flag event? I need to disable it or to configure it so it deletes emails at least couple of months old. Thanks all, peter

2 1