April 2017 - dovecot - dovecot.org

v2.2.29 released
by Timo Sirainen 11 Apr '17

11 Apr '17

https://dovecot.org/releases/2.2/dovecot-2.2.29.tar.gz https://dovecot.org/releases/2.2/dovecot-2.2.29.tar.gz.sig * passdb/userdb dict: Don't double-expand %variables in keys. If dict was used as the authentication passdb, using specially crafted %variables in the username could be used to cause DoS (CVE-2017-2669) * When Dovecot encounters an internal error, it logs the real error and usually logs another line saying what function failed. Previously the second log line's error message was a rather uninformative "Internal error occurred. Refer to server log for more information." Now the real error message is duplicated in this second log line. * lmtp: If a delivery has multiple recipients, run autoexpunging only for the last recipient. This avoids a problem where a long autoexpunge run causes LMTP client to timeout between the DATA replies, resulting in duplicate mail deliveries. * config: Don't stop the process due to idling. Otherwise the configuration is reloaded when the process restarts. * mail_log plugin: Differentiate autoexpunges from regular expunges * imapc: Use LOGOUT to cleanly disconnect from server. * lib-http: Internal status codes (>9000) are no longer visible in logs * director: Log vhost count changes and HOST-UP/DOWN + quota: Add plugin { quota_max_mail_size } setting to limit the maximum individual mail size that can be saved. + imapc: Add imapc_features=delay-login. If set, connecting to the remote IMAP server isn't done until it's necessary. + imapc: Add imapc_connection_retry_count and imapc_connection_retry_interval settings. + imap, pop3, indexer-worker: Add (deinit) to process title before autoexpunging runs. + Added %{encrypt} and %{decrypt} variables + imap/pop3 proxy: Log proxy state in errors as human-readable string. + imap/pop3-login: All forward_* extra fields returned by passdb are sent to the next hop when proxying using ID/XCLIENT commands. On the receiving side these fields are imported and sent to auth process where they're accessible via %{passdb:forward_*}. This is done only if the sending IP address matches login_trusted_networks. + imap-login: If imap_id_retain=yes, send the IMAP ID string to auth process. %{client_id} expands to it in auth process. The ID string is also sent to the next hop when proxying. + passdb imap: Use ssl_client_ca_* settings for CA validation. - fts-tika: Fixed crash when parsing attachment without Content-Disposition header. Broken by 2.2.28. - trash plugin was broken in 2.2.28 - auth: When passdb/userdb lookups were done via auth-workers, too much data was added to auth cache. This could have resulted in wrong replies when using multiple passdbs/userdbs. - auth: passdb { skip & mechanisms } were ignored for the first passdb - oauth2: Various fixes, including fixes to crashes - dsync: Large Sieve scripts (or other large metadata) weren't always synced. - Index rebuild (e.g. doveadm force-resync) set all mails as \Recent - imap-hibernate: %{userdb:*} wasn't expanded in mail_log_prefix - doveadm: Exit codes weren't preserved when proxying commands via doveadm-server. Almost all errors used exit code 75 (tempfail). - ACLs weren't applied to not-yet-existing autocreated mailboxes. - Fixed a potential crash when parsing a broken message header. - cassandra: Fallback consistency settings weren't working correctly. - doveadm director status <user>: "Initial config" was always empty - imapc: Various reconnection fixes.

4 6

How to migration my mails from another server ?
by Mik J 10 Apr '17

10 Apr '17

Hello, I would like to host my mails and I would like to retrieve my existing mailbox content from my ISP My ISP ====== I don't have root access on the server The server seem to use Dovecot according to the banner. My machine ========== I have root access on the server I use Dovecot 2.2.21 I use Maildir I read this page multiple times but it's not clear to me https://wiki2.dovecot.org/Migration/Dsync "Set up configuration for the IMAP server you wish to migrate from" Is it my ISP's server or my server ? In which configuration file should I do this ? The same for this option dsync_features = empty-header-workaroundwhere is it supposed to be configured ? My most important case is quite "simple": the mailboxes (there are 2 or 3) I want to migrate don't exist on my machine, so there is no such problems as merge existing mails. Will this migration use imap and retrieve the content of the mails ? Will it use port 143 or 993 ? I'll use the backup option I have another case which there are mails on both servers. So I would like to keep existing mails on my machine. I'll use the sync -1 option ? Thank you

3 3

Re: abrt reported "imap killed by SIGBUS"
by Hongying Liu 10 Apr '17

10 Apr '17

Sorry for late response. This issue occurs ran randomly. So I don't know how to run valgrind command. 2017-04-10 17:14 GMT+08:00 Hongying Liu <hongyliu(a)redhat.com>: > Sorry for late response. > > This issue occurs ran randomly. So I don't know how to run valgrind > command. > > I upload the coredump. > > The version of dovecot is dovecot-2.2.10-7.el7.x86_64. > > 2017-04-07 14:07 GMT+08:00 Aki Tuomi <aki.tuomi(a)dovecot.fi>: > >> Please keep responses on list. >> >> Can you please run valgrind on this, as it appears to be some sort of >> memory corruption issue. >> >> Aki >> >> > On April 7, 2017 at 5:37 AM Hongying Liu <hongyliu(a)redhat.com> wrote: >> > >> > >> > Thanks for your response. >> > >> > (gdb) p cache >> > $1 = (struct mail_cache *) 0x7fc4a811a320 >> > (gdb) p cache->hdr >> > $2 = (const struct mail_cache_header *) 0x7fc4a6ddb000 >> > (gdb) p cache->hdr->file_seq >> > Cannot access memory at address 0x7fc4a6ddb008 <---- >> > (gdb) p reset_id >> > $3 = 1488892251 >> > >> > >> > 2017-04-06 19:34 GMT+08:00 Aki Tuomi <aki.tuomi(a)dovecot.fi>: >> > >> > > Hi! >> > > >> > > Responses in middle. >> > > >> > > Aki >> > > >> > > On 06.04.2017 06:05, Hongying Liu wrote: >> > > > Hi sbr-services, >> > > > >> > > > Could you give me some idea? >> > > > >> > > > ### abrt reported the error as below. >> > > > [root@cupop4 log]# abrt-cli list --since 1488267244 id >> > > > ad716dbfd3a68bbe0f055e32ebfe562f4f75df43 >> > > > reason: imap killed by SIGBUS >> > > > time: Sun Mar 19 2017 10:58:27 AM JST >> > > > cmdline: dovecot/imap >> > > > package: dovecot-2.2.10-7.el7 >> > > > uid: 80180 (acaa2325) >> > > > count: 2 >> > > > Directory: /var/spool/abrt/ccpp-2017-03-19-10:58:27-4904 >> > > > 'abrt-cli report を実行する/var/spool/abrt/ccpp-2017 >> -03-19-10:58:27-4904' >> > > > >> > > > >> > > > ### /var/log/messages >> > > > Mar 19 10:58:27 cupop4 abrt-hook-ccpp: Process 4904 (imap) of user >> 80180 >> > > > killed by SIGBUS - dumping core >> > > > >> > > > And, there are lots of ldap error in /var/log/messages. Dovecot >> uses the >> > > > ldap as userdb and passwddb. >> > > > Is the ldap error related to imap crash? >> > > > >> > > > Mar 19 10:58:21 cupop4 nslcd[1534]: [a378de] >> <group/member="postfix"> >> > > > ldap_result() failed: Invalid DN syntax: Invalid DN >> > > >> > > This looks like an invalid DN syntax. >> > > >> > > > >> > > > ### /var/log/dovecot >> > > > >> > > > Mar 19 10:58:27 cupop4 dovecot: imap(acaa2325): Fatal: master: >> > > > service(imap): child 4904 killed with signal 7 (core dumped) >> > > > >> > > Can you do p cache, p cache->hdr, p cache->hdr->file_seq and p >> reset_id >> > > >> > > > ### gdb >> > > > >> > > > Core was generated by `dovecot/imap'. >> > > > Program terminated with signal 7, Bus error. >> > > > #0 0x00007fc4a69f293e in mail_cache_lookup_offset >> (cache=0x7fc4a811a320, >> > > > view=<optimized out>, seq=<optimized out>, offset_r=offset_r@entry >> > > > =0x7ffe23a98900) >> > > > at mail-cache-lookup.c:95 >> > > > 95 while (cache->hdr->file_seq != reset_id) { >> > > > (gdb) bt >> > > > #0 0x00007fc4a69f293e in mail_cache_lookup_offset >> (cache=0x7fc4a811a320, >> > > > view=<optimized out>, seq=<optimized out>, offset_r=offset_r@entry >> > > > =0x7ffe23a98900) >> > > > at mail-cache-lookup.c:95 >> > > > #1 0x00007fc4a69f2afa in mail_cache_lookup_iter_init >> > > > (view=view@entry=0x7fc4a81332b0, >> > > > seq=<optimized out>, ctx_r=ctx_r@entry=0x7ffe23a988e0) >> > > > at mail-cache-lookup.c:154 >> > > > #2 0x00007fc4a69f2f8f in mail_cache_seq (seq=<optimized out>, >> > > > view=0x7fc4a81332b0) at mail-cache-lookup.c:322 >> > > > #3 mail_cache_field_exists (view=view@entry=0x7fc4a81332b0, >> > > seq=seq@entry=760, >> > > > field=field@entry=4) at mail-cache-lookup.c:352 >> > > > #4 0x00007fc4a69f3151 in mail_cache_lookup_field >> (view=0x7fc4a81332b0, >> > > > dest_buf=dest_buf@entry=0x7ffe23a98a00, seq=760, >> > > field_idx=field_idx@entry >> > > > =4) >> > > > at mail-cache-lookup.c:413 >> > > > #5 0x00007fc4a69d9439 in index_mail_cache_lookup_field >> > > > (mail=mail@entry=0x7fc4a8128cd0, >> > > > buf=buf@entry=0x7ffe23a98a00, field_idx=field_idx@entry=4) >> > > > at index-mail.c:68 >> > > > #6 0x00007fc4a69d94a7 in index_mail_get_fixed_field >> > > (mail=0x7fc4a8128cd0, >> > > > field=<optimized out>, data=<optimized out>, data_size=8) at >> > > > index-mail.c:130 >> > > > #7 0x00007fc4a69d9fed in index_mail_get_cached_virtual_size >> > > > (mail=mail@entry=0x7fc4a8128cd0, size_r=size_r@entry=0x7ffe23a98b20) >> at >> > > > index-mail.c:403 >> > > > #8 0x00007fc4a6987c1b in maildir_mail_get_virtual_size >> > > > (_mail=0x7fc4a8128cd0, size_r=0x7ffe23a98b20) at maildir-mail.c:388 >> > > > #9 0x00007fc4a69d9719 in index_mail_update_access_parts >> > > > (mail=mail@entry=0x7fc4a8128cd0) >> > > > at index-mail.c:1490 >> > > > #10 0x00007fc4a69dbb93 in index_mail_set_seq (_mail=0x7fc4a8128cd0, >> > > > seq=760, saving=<optimized out>) at index-mail.c:1521 >> > > > #11 0x00007fc4a69e28cb in search_more_with_mail >> (mail=0x7fc4a8128cd0, >> > > > ctx=0x7fc4a8132730) at index-search.c:1507 >> > > > #12 search_more_with_prefetching (mail_r=<optimized out>, >> ctx=<optimized >> > > > out>) at index-search.c:1579 >> > > > #13 search_more (ctx=ctx@entry=0x7fc4a8132730, >> > > > mail_r=mail_r@entry=0x7ffe23a98c20) >> > > > at index-search.c:1650 >> > > > #14 0x00007fc4a69e2ff4 in index_storage_search_next_nonblock >> > > > (_ctx=0x7fc4a8132730, mail_r=0x7fc4a81120d8, >> tryagain_r=0x7ffe23a98c87) >> > > at >> > > > index-search.c:1674 >> > > > #15 0x00007fc4a69bc4ef in mailbox_search_next_nonblock >> > > > (ctx=ctx@entry=0x7fc4a8132730, >> > > > mail_r=mail_r@entry=0x7fc4a81120d8, >> > > > tryagain_r=tryagain_r@entry=0x7ffe23a98c87) at >> mail-storage.c:1787 >> > > > #16 0x00007fc4a69bc55d in mailbox_search_next (ctx=0x7fc4a8132730, >> > > > mail_r=mail_r@entry=0x7fc4a81120d8) at mail-storage.c:1773 >> > > > #17 0x00007fc4a6e97874 in imap_fetch_more_int (ctx=ctx@entry >> > > =0x7fc4a8112078, >> > > > cancel=false) at imap-fetch.c:479 >> > > > #18 0x00007fc4a6e98852 in imap_fetch_more (ctx=0x7fc4a8112078, >> > > > cmd=cmd@entry=0x7fc4a8111ed0) >> > > > at imap-fetch.c:556 >> > > > #19 0x00007fc4a6e8c17d in cmd_fetch (cmd=0x7fc4a8111ed0) at >> > > cmd-fetch.c:284 >> > > > #20 0x00007fc4a6e9601c in command_exec (cmd=cmd@entry=0x7fc4a8111ed0) >> at >> > > > imap-commands.c:158 >> > > > #21 0x00007fc4a6e94f1f in client_command_input (cmd=cmd@entry >> > > =0x7fc4a8111ed0) >> > > > at imap-client.c:780 >> > > > #22 0x00007fc4a6e95005 in client_command_input (cmd=0x7fc4a8111ed0) >> at >> > > > imap-client.c:841 >> > > > #23 0x00007fc4a6e952fd in client_handle_next_command >> > > > (remove_io_r=<synthetic pointer>, client=0x7fc4a8111530) at >> > > > imap-client.c:879 >> > > > #24 client_handle_input (client=client@entry=0x7fc4a8111530) at >> > > > imap-client.c:891 >> > > > #25 0x00007fc4a6e956c5 in client_input (client=0x7fc4a8111530) at >> > > > imap-client.c:933 >> > > > #26 0x00007fc4a66e1a87 in io_loop_call_io (io=0x7fc4a811edd0) at >> > > > ioloop.c:388 >> > > > #27 0x00007fc4a66e290f in io_loop_handler_run >> > > > (ioloop=ioloop@entry=0x7fc4a80fc750) >> > > > at ioloop-epoll.c:220 >> > > > #28 0x00007fc4a66e15d8 in io_loop_run (ioloop=0x7fc4a80fc750) at >> > > > ioloop.c:412 >> > > > #29 0x00007fc4a668e9e3 in master_service_run >> (service=0x7fc4a80fc5e0, >> > > > callback=callback@entry=0x7fc4a6e9ed70 <client_connected>) at >> > > > master-service.c:571 >> > > > #30 0x00007fc4a6e89324 in main (argc=1, argv=0x7fc4a80fc390) at >> > > main.c:400 >> > > > >> > > > Thank you. >> > > > >> > > > Best regards, >> > > > Hongying Liu >> > > >> > > >> > >

1 0

[Bug] FTS invalid address parsing
by azurit＠pobox.sk 10 Apr '17

10 Apr '17

Hi, i found another bug in FTS while parsing quoted-printable/base64-encoded sender's name inside 'From' header (again, i was figuring out why some of messages cannot be searched). Header: From: =?UTF-8?Q?A=2CB?= <test(a)example.com> Data send to Solr by Dovecot: <field name="from">A@, B &lt;test(a)example.com&gt;</field> Header: From: =?UTF-8?Q?A_B=2CC?= <test(a)example.com> Data send to Solr by Dovecot: <field name="from">A@</field> Header: From: =?UTF-8?B?QSBCLEMK?= <test(a)example.com> Data send to Solr by Dovecot: <field name="from">A@</field> Dovecot 2.2.27. azur

2 2

[Bug] FTS double escaping
by azurit＠pobox.sk 09 Apr '17

09 Apr '17

Hi, i'm trying to resolve few problems with indexing 'From' headers using FTS/Solr. I was tcpdumping the communication between Dovecot and Jetty/Solr and noticed that 'From' headers, which includes also sender's name, are double escaped. This is what was Dovecot sending to Solr: </field><field name="from">Name Surname &lt;test(a)example.com&gt;</field></doc></add> As you can see, characters < and > were escaped to < and > which were, again, escaped to &lt; and &gt;. This is doing problems while trying to index whole e-mail address, as Solr sees it as '<test(a)example.com>'. I spend hours trying to figure out why i'm able to search in all parts of e-mail addresses but searching for full and exact e-mail address was successfull ONLY for messages which doesn't include sender's name in 'From' header. Finally, after i found this bug, this fixed all search problems: <filter class="solr.PatternReplaceFilterFactory" pattern="&lt;" replacement=""/> <filter class="solr.PatternReplaceFilterFactory" pattern="&gt;" replacement=""/> I hope that, at least, this bug, reported by me, will be fixed. Thank you. azur

3 5

sieve does not seem to be working
by Robert Moskowitz 09 Apr '17

09 Apr '17

my local.conf has: # 90-sieve.conf plugin { sieve_before = /home/sieve/globalfilter.sieve } and cat /home/sieve/globalfilter.sieve require ["fileinto","mailbox"]; if anyof ( header :contains "X-Spam-Flag" "YES", header :contains "subject" "***SPAM***" ) { fileinto :create "Spam"; } There IS a globalfilter.svbin when I tried: sendmail -i testit3(a)test.htt-consult.com < sample-spam-GTUBE-junk.txt amavis is flagging it as ***Spam*** but it stays in inbox. So I tried: sieve-test -e -l /home/vmail/test.htt-consult.com/testit3/ /home/sieve/globalfilter.sieve /home/vmail/test.htt-consult.com/testit3/cur/1491512409.M371278P6513.z9m9z.test.htt-consult.com\,S\=1823\,W\=1868\:2\, info: msgid=<GTUBE1.1010101(a)example.net>: stored mail into mailbox 'Spam'. sieve-test(root): Info: final result: success And it DID get copied to Spam, not moved. I am now seeing it in inbox and Spam. Of course ownership on the message in Spam were wrong (root:root instead of vmail:mail, but I fixed that). So two questions, probably linked: Why did sieve not work. Is the subject test case sensitive? If it is case sensitive, why did the sieve-test work? thanks

2 2

Pigeonhole Sieve Extprograms Plugin
by KT Walrus 09 Apr '17

09 Apr '17

Just discovered the Pigeonhole Sieve Extprograms plugin (https://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms <https://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms>)… I currently have Postfix configured to run a script to store all incoming messages into a Mysql DB and a batch job that retrieves the messages in the DB, processes them and delivers them to the recipients' Dovecot servers. Seems to me it might be better to have Postfix deliver all incoming messages directly to a single Dovecot mailbox and use this Sieve plugin to do the whole job of updating the DB, processing the message, and delivering to the recipients. Also, I have a need to implement updating the DB when the recipient reads a message for the first time (setting the \Seen flag). Would this work best to execute a script using this Sieve plugin from a IMAP Sieve script that watches for \Seen flag changes? This might be a whole lot better than the way I was thinking of doing it before. I’m basically implementing mailing lists where the senders receive a notification reply with a link to a web page where the message may be edited/deleted before it is sent to recipients (after a fixed amount of time has elapsed). This gives the sender the opportunity to change his mind before it is too late or to send to recipients immediately (by marking the message as Urgent). Also, even after the message has been sent out, I want to allow the sender to be able to delete it in the recipients mailboxes if they have not read it yet (downloaded it to their email client) or to see which recipients have read the message (and when they read/downloaded it for the first time). Both the sender and the recipients will have Dovecot mailboxes so I want to give more control over the messages for their entire lifetimes (both for senders and for recipients) than normal email does. Any thoughts? Kevin

1 0

Released Pigeonhole v0.4.18.rc1 for Dovecot v2.2.29.rc1.
by Stephan Bosch 09 Apr '17

09 Apr '17

Hello Dovecot users, This is a rather small release. It fixes one important bug in the new discard script feature. It also adds a couple of new features for IMAPSieve. Changelog v0.4.18: + imapsieve plugin: Implemented the copy_source_after rule action. When this is enabled for a mailbox rule, the specified Sieve script is executed for the message in the source mailbox during a "COPY" event. This happens only after the Sieve script that is executed for the corresponding message in the destination mailbox finishes running successfully. + imapsieve plugin: Added non-standard Sieve environment items for the source and destination mailbox. - multiscript: The execution of the discard script had an implicit "keep", rather than an implicit "discard". The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.18… http://pigeonhole.dovecot.org/releases/2.2/rc/dovecot-2.2-pigeonhole-0.4.18… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this release candidate and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

1 0

Updating mysql db when messages first read by IMAP user
by KT Walrus 09 Apr '17

09 Apr '17

I need to implement updating a mysql db when the \SEEN flag is set by IMAP. New message delivery is done by having Postfix store all new inbound messages in a mysql db and a batch job watching the db for new messages and delivering them to the correct Dovecot server for the recipients. The batch job leaves a log entry in the db (for each recipient) with the current timestamp and adds a header with the log id to the outgoing message that is sent to Dovecot. The log table has a column for the first read timestamp which is set to 0 initially. I plan on implementing a Sieve IMAP script that watches for changes to the \SEEN flag and sends a mailto: notify message that contains the log id (from the message header) and the current date to a mail address dedicated for keep track of \SEEN flag changes. The notify message will be delivered to this mail address where a batch job watches that mail address and updates the db for the log id in the notify message with the timestamp and the current date if the row in the log table has a first read timestamp of 0. I plan on providing a web page that the sender can use to see whether the messages they have sent have been seen by the recipients. The web page will show other info too, but the part that I need to implement next is updating the db when a sent message has been read by the recipients of the message. Is this a good way to do this? Or, is there a better way than using an IMAP Sieve script to send notify messages? Is there any other way to have a Sieve script update a mysql database? Or, is there some other mechanism in Dovecot to use instead of an IMAP Sieve script? Kevin

1 0

IMAPSieve Changed flags cause?
by MRob 09 Apr '17

09 Apr '17

RFC 6785 says I should be able to run scripts when a command changes flags on a message but I can't understand what to put in imapsieve_mailboxXXX_causes. Dovecot logs something like STORE as an invalid cause. How do I run an administrator Sieve script caused from change in flags?

2 5