July 2019 - dovecot - dovecot.org

Dovecot release v2.3.7
by Aki Tuomi 17 Jul '19

17 Jul '19

Hi! We are pleased to release Dovecot release v2.3.7. Tarball is available at https://dovecot.org/releases/2.3/dovecot-2.3.7.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.7.tar.gz.sig Binary packages are available at https://repo.dovecot.org/ Changes ------- * fts-solr: Removed break-imap-search parameter + Added more events for the new statistics, see https://doc.dovecot.org/admin_manual/list_of_events/ + mail-lua: Add IMAP metadata accessors, see https://doc.dovecot.org/admin_manual/lua/ + Add event exporters that allow exporting raw events to log files and external systems, see https://doc.dovecot.org/configuration_manual/event_export/ + SNIPPET is now PREVIEW and size has been increased to 200 characters. + Add body option to fts_enforced. This triggers building FTS index only on body search, and an error using FTS index fails the search rather than reads through all the mails. - Submission/LMTP: Fixed crash when domain argument is invalid in a second EHLO/LHLO command. - Copying/moving mails using Maildir format loses IMAP keywords in the destination if the mail also has no system flags. - mail_attachment_detection_options=add-flags-on-save caused email body to be unnecessarily opened when FETCHing mail headers that were already cached. - mail attachment detection keywords not saved with maildir. - dovecot.index.cache may have grown excessively large in some situations. This happened especially when using autoexpunging with lazy_expunge folders. Also with mdbox format in general the cache file wasn't recreated as often as it should have. - Autoexpunged mails weren't immediately deleted from the disk. Instead, the deletion from disk happened the next time the folder was opened. This could have caused unnecessary delays if the opening was done by an interactive IMAP session. - Dovecot's TCP connections sometimes add extra 40ms latency due to not enabling TCP_NODELAY. HTTP and SMTP/LMTP connections weren't affected, but everything else was. This delay wasn't always visible - only in some situations with some message/packet sizes. - imapc: Fix various crash conditions - Dovecot builds were not always reproducible. - login-proxy: With shutdown_clients=no after config reload the existing connections could no longer be listed or kicked with doveadm. - "doveadm proxy kick" with -f parameter caused a crash in some situations. - Auth policy can cause segmentation fault crash during auth process shutdown if all auth requests have not been finished. - Fix various minor bugs leading into incorrect behaviour in mailbox list index handling. These rarely caused noticeable problems. - LDAP auth: Iteration accesses freed memory, possibly crashing auth-worker - local_name { .. } filter in dovecot.conf does not correctly support multiple names and wildcards were matched incorrectly. - replicator: dsync assert-crashes if it can't connect to remote TCP server. - config: Memory leak in config process when ssl_dh setting wasn't set and there was no ssl-parameters.dat file. This caused config process to die once in a while with "out of memory". --- Aki Tuomi Open-Xchange oy

5 16

pigeonhole question: filtering on delivered-to in case of fetchmail
by Trever L. Adams 17 Jul '19

17 Jul '19

So, one of the problems I am seeing is that people are trying to fake users into revealing information by sending from an outside domain but with an internal reply to address and claiming to be administration, IT or what not. I can set up something that will reject if from is outside the domain by reply to is internal. The problem is in some setups, there are fetchmail setups. I do not want to reject these with a message. Which is what I am currently doing for the others. Maybe I should discard them all without rejecting. However, my question is this: Since such fetchmail messages will usually end up with two (at least two?!?) Delivered-To headers, one for the fetchmail delivery and one for the original target address's/system's delivery is it possible to do something like this and have it work? require ["fileinto", "regex","reject"]; if address :regex "Reply-To" ".*@<%= @name -%>" { if not address :regex "From" ".*@<%= @name -%>" { if not header :regex "Delivered-To" " .*@<%= @name -%> { reject "We do not allow emails from outside our system to give Reply-To into our system!"; stop; } } } Please, not the <% =@name -%> is just that this is from a puppet module I use to maintain these systems. It is the domain name for the mail system. An example would be .*(a)middleearth.sapphiresunday.org here. Thank you for any help in figuring this out. The reason I want a reject in the case of non-fetchmail email is to let users know if they try to do it (as many have multiple email accounts) and may try it. But in fetchmail cases, no need to leak to the outside world that users are doing fetchmail and what their account is in the other system. Thank you. Trever

2 3

Dovecot v2.3.7 - TLS/SSL issue
by Christos Chatzaras 17 Jul '19

17 Jul '19

I use a helpdesk system that connects to dovecot using POP3 with SSL enabled to fetch the e-mails. After I upgrade to v2.3.7 the helpdesk randomly (some times it works, some times not) doesn't fetch the e-mails. If I configure the e-mail accounts with SSL/TLS disabled then it works. Any idea about this?

2 1

2.3.7 slower than 2.3.6?
by Ralf Hildebrandt 17 Jul '19

17 Jul '19

We're using dovecot (via LMTP) as a backup for all incoming mail. I upgraded from 2.3.6 to 2.3.7 on the 12th: 2019-07-12 14:35:44 upgrade dovecot-imapd:amd64 2:2.3.6-2~bionic 2:2.3.7-8~bionic and it seems that 2.3.7 is slower than 2.3.6 -- mail to the backup IMAP box is suddenly taking quite some time to get delivered and is piling up in the queue. doveconf -n: ============ default_vsz_limit = 2 G lmtp_user_concurrency_limit = 10000 mail_attachment_dir = /home/copymail/attachments mail_fsync = never mail_location = mdbox:~/mdbox mail_plugins = zlib fts mdbox_rotate_size = 128 M ... namespaces ... plugin { fts_autoindex = yes fts_languages = de,en zlib_save = gz zlib_save_level = 5 } protocols = " imap lmtp" userdb { args = username_format=%u /etc/dovecot/passwd driver = passwd-file } protocol lmtp { mail_plugins = zlib fts } -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | https://www.charite.de

2 5

Auto-duplicate mailstore?
by ＠lbutlr 16 Jul '19

16 Jul '19

I am looking for something that is similar to replication, but without a second server. I currently have a system setup using postfix bcc to write out copies of all email, but what I would rather do is just have all the mail written to /usr/local/virtual/%u/Maildir and also to /backup/usr/local/virtual/%u/Maildir (for example) as this will make restoring messages much simpler. -- Gods don't like people not doing much work. People who aren't busy all the time might start to think.

1 2

Unexpected result from LIST EXTENDED command
by Emil Kalchev 16 Jul '19

16 Jul '19

I am executing this command below to dovecot-2.3.5-6.cp1178.x86_64 server Notice that some status responses are missing (For folders INBOX.Archive, INBOX.spam.&-BD0EOQQ9BDkEPQ-). I wonder If this is a bug or working as expected In rfc5819 there is this: If the server runs into unexpected problems while trying to look up the STATUS information, it MAY drop the corresponding STATUS reply. In such a situation, the LIST command would still return a tagged OK reply. May be that is the reason for this response? Is it possible to find more details in server logs why STATUS is missing? C: B00000007 LIST "" "INBOX.*" RETURN (SUBSCRIBED CHILDREN STATUS (HIGHESTMODSEQ)) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.gfgfee888 S: * STATUS INBOX.gfgfee888 (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.99999999 S: * STATUS INBOX.99999999 (HIGHESTMODSEQ 3) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.yyyyyyyyyyyyy S: * STATUS INBOX.yyyyyyyyyyyyy (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.y S: * STATUS INBOX.y (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.fgfgfg S: * STATUS INBOX.fgfgfg (HIGHESTMODSEQ 61) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.Mailspring S: * STATUS INBOX.Mailspring (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Mailspring.Snoozed S: * STATUS INBOX.Mailspring.Snoozed (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.fgfgf S: * STATUS INBOX.fgfgf (HIGHESTMODSEQ 3) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.gggg34 S: * STATUS INBOX.gggg34 (HIGHESTMODSEQ 5) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.gggg S: * STATUS INBOX.gggg (HIGHESTMODSEQ 2) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.gggg.hhhh S: * STATUS INBOX.gggg.hhhh (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.mkmkm S: * STATUS INBOX.mkmkm (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.sdfsf S: * STATUS INBOX.sdfsf (HIGHESTMODSEQ 1) S: * LIST (\HasChildren \UnMarked) "." INBOX.sdfsf.sdfsdfsdfsd S: * STATUS INBOX.sdfsf.sdfsdfsdfsd (HIGHESTMODSEQ 1) S: * LIST (\HasNoChildren \UnMarked) "." INBOX.sdfsf.sdfsdfsdfsd.zzzz S: * STATUS INBOX.sdfsf.sdfsdfsdfsd.zzzz (HIGHESTMODSEQ 2263) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.uuyy S: * STATUS INBOX.uuyy (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.uuyy.sdfsdf S: * STATUS INBOX.uuyy.sdfsdf (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." "INBOX.uuyy.sdfsdf.New Folder" S: * STATUS "INBOX.uuyy.sdfsdf.New Folder" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." "INBOX.uuyy.sdfsdf.New Folder.dfgdfg" S: * STATUS "INBOX.uuyy.sdfsdf.New Folder.dfgdfg" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf" S: * STATUS "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf.sdfsdfsfsdf" S: * STATUS "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf.sdfsdfsfsdf" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf.sdfsdfsfsdf.asdasdasd" S: * STATUS "INBOX.uuyy.sdfsdf.New Folder.dfgdfg.sdfsdfsdf.sdfsdfsfsdf.asdasdasd" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.uuyy.fdfgdfg S: * STATUS INBOX.uuyy.fdfgdfg (HIGHESTMODSEQ 2) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.vvvcv S: * STATUS INBOX.vvvcv (HIGHESTMODSEQ 2627) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.hallo S: * STATUS INBOX.hallo (HIGHESTMODSEQ 96) S: * LIST (\Subscribed \HasChildren \UnMarked) "." "INBOX.New Folder" S: * STATUS "INBOX.New Folder" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.omgfolder" S: * STATUS "INBOX.New Folder.omgfolder" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.testfolder" S: * STATUS "INBOX.New Folder.testfolder" (HIGHESTMODSEQ 1) S: * LIST (\HasNoChildren \UnMarked) "." "INBOX.New Folder.hkhk" S: * STATUS "INBOX.New Folder.hkhk" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.zzz" S: * STATUS "INBOX.New Folder.zzz" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.aaa" S: * STATUS "INBOX.New Folder.aaa" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.kgkgkk" S: * STATUS "INBOX.New Folder.kgkgkk" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.gggfff" S: * STATUS "INBOX.New Folder.gggfff" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.New Folder" S: * STATUS "INBOX.New Folder.New Folder" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.hhhh" S: * STATUS "INBOX.New Folder.hhhh" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.New Folder.nnnn" S: * STATUS "INBOX.New Folder.nnnn" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.asdasd S: * STATUS INBOX.asdasd (HIGHESTMODSEQ 23) S: * LIST (\HasNoChildren \UnMarked) "." INBOX.asdasd.gdfgdfgdfg S: * STATUS INBOX.asdasd.gdfgdfgdfg (HIGHESTMODSEQ 10) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.spam S: * STATUS INBOX.spam (HIGHESTMODSEQ 44) S: * LIST (\HasChildren \UnMarked) "." INBOX.spam.&-BD0EOQQ9BDkEPQ- S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg S: * STATUS INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." "INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.New Folder" S: * STATUS "INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.New Folder" (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.aaa S: * STATUS INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.aaa (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.aaa.aaa S: * STATUS INBOX.spam.&-BD0EOQQ9BDkEPQ-.jhfhg.aaa.aaa (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Sent S: * STATUS INBOX.Sent (HIGHESTMODSEQ 1353) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Trash S: * STATUS INBOX.Trash (HIGHESTMODSEQ 5823) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.Junk S: * STATUS INBOX.Junk (HIGHESTMODSEQ 15) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.Junk.asdasd S: * STATUS INBOX.Junk.asdasd (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.Junk.asdasd.rgthfgh S: * STATUS INBOX.Junk.asdasd.rgthfgh (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Junk.asdasd.rgthfgh.Mailspring S: * STATUS INBOX.Junk.asdasd.rgthfgh.Mailspring (HIGHESTMODSEQ 28) S: * LIST (\Subscribed \HasChildren \UnMarked) "." INBOX.Junk.asdasd.asdasdasd S: * STATUS INBOX.Junk.asdasd.asdasdasd (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Junk.asdasd.asdasdasd.dfgdfg S: * STATUS INBOX.Junk.asdasd.asdasdasd.dfgdfg (HIGHESTMODSEQ 1) S: * LIST (\Subscribed \HasNoChildren \UnMarked) "." INBOX.Drafts S: * STATUS INBOX.Drafts (HIGHESTMODSEQ 1091) S: * LIST (\HasNoChildren \UnMarked) "." INBOX.Archive S: B00000007 OK List completed (0.004 + 0.000 + 0.003 secs).

3 4

Replication Upgrade 2.2 -> 2.3 Question
by Cassidy B. Larson 15 Jul '19

15 Jul '19

Finally getting around to upgrading my setup to 2.3.x. All of our front end proxies are fine on 2.3 connecting to a 2.2 backend storage pair. So the question is, if I stage the upgrade, will it break replication between the two backend storage servers? My thought was to upgrade one to 2.3, and keep the other on 2.2 for a couple of days? If so, can 2.2 replicate to 2.3, and 2.3 replicate to 2.2 without any issue? Or am I going to have to go all-in on 2.3 at once on both? Heh. Thanks, -c

1 0

Listing expunged messages
by Mike 15 Jul '19

15 Jul '19

Hello, Following on my last question about broken mdboxes, removing dovecot.map.log and force-resync does work. My last problem now however is that after doing this, messages which were 'deleted' previously, come back in the inbox as new messages. I have the original mailboxes and can subscribe to the expunged folders and everything in EXPUNGED/INBOX on the original, is appearing in INBOX on this migrated system. I would want to return everything to EXPUNDED/INBOX so my users don't see old dead resurrected messages. Is there a procedure to identify these? Thanks. Mike-

1 0

Weird response to LIST EXTENDED
by Emil Kalchev 15 Jul '19

15 Jul '19

Hello, I was discussing weird response from Dovecot on Mailkit (this is client IMAP C# library) github page. We concluded that this is a bug in Dovecot server but I wanted to double check with you. Here are the details about the issue https://github.com/jstedfast/MailKit/issues/879 Sent with OfficeSuite Mail for Windows PC www.officesuitenow.com<https://www.officesuitenow.com>

1 0

Authdb NSS module
by Eugene 15 Jul '19

15 Jul '19

Hello! Upgrading manual tells that authdb [NSS module was removed][1] some time ago. [1]: https://wiki2.dovecot.org/Upgrading/2.3#line-100 > userdb nss was removed. Use userdb passwd instead. Can this change be reverted? I'd like to use only libnss_sss.so.2 as dovecot userdb source. It's also essential for me to enable files backend in nsswitch.conf so the system could use local user db. At the same time dovecot must not see local users at all. Authdb NSS module could help me there. The other solution would be to use another instance of nsswitch.conf for dovecot authdb passwd module. Is it possible? Thanks! -- Eugene Bright IT engineer Tel: + 79257289622

3 3