October 2018 - dovecot

Delete vs. Expunge in Public Namespace
by Chris 03 Oct '18

03 Oct '18

All, my IMAP client [1] marks messages as deleted, but doesn't expunge. Dovecot (2.2.27-3+deb9u2) expunges them. Is this because the mailbox is part of public namespace? .... ::1.38247-::1.00143: MALP6 STORE 48,49,50,51,52,53,54,55,56,57,58,59,60,61 +FLAGS (\Deleted) ::1.00143-::1.38247: * 48 FETCH (FLAGS (\Deleted \Seen)) * 49 FETCH (FLAGS (\Deleted \Seen)) * 50 FETCH (FLAGS (\Deleted \Seen)) * 51 FETCH (FLAGS (\Deleted \Seen)) * 52 FETCH (FLAGS (\Deleted \Seen)) * 53 FETCH (FLAGS (\Deleted \Seen)) * 54 FETCH (FLAGS (\Deleted \Seen)) * 55 FETCH (FLAGS (\Deleted \Seen)) * 56 FETCH (FLAGS (\Deleted \Seen)) * 57 FETCH (FLAGS (\Deleted \Seen)) * 58 FETCH (FLAGS (\Deleted \Seen)) * 59 FETCH (FLAGS (\Deleted \Seen)) * 60 FETCH (FLAGS (\Deleted \Seen)) * 61 FETCH (FLAGS (\Deleted \Seen)) MALP6 OK Store completed. ::1.38247-::1.00143: MALP7 SEARCH DELETED .... Oct 2 10:18:03 hyptest dovecot: imap(chris): delete: box=MailingListen/test125, uid=180, msgid=<740a9ba3-d637-3fa0-1b87-ff8d79491bf0@ hyp.postbox.xyz>, size=1509 Oct 2 10:18:03 hyptest dovecot: imap(chris): expunge: box=MailingListen/test125, uid=167, msgid=<740a9ba3-d637-3fa0-1b87-ff8d79491bf0 @hyp.postbox.xyz>, size=1509 namespace { list = yes location = maildir:/var/vmail/public/:LAYOUT=fs:INDEXPVT=%h/.public-indexes prefix = MailingListen/ separator = / subscriptions = no type = public } - Chris [1] https://raw.githubusercontent.com/quentinsf/IMAPdedup/master/imapdedup.py

2 3

Bug reports - auth is broken in Dovecot 2.3.3
by Berindeie Avram-Teodor 03 Oct '18

03 Oct '18

I applied the patch and recompiled but nothing resolved. What else can I do?

3 3

Bug reports - auth is broken in Dovecot 2.3.3
by Berindeie Avram-Teodor 03 Oct '18

03 Oct '18

Hello, I have compiled Dovecot 2.3.3 from sources but I have problems with authentication. I tried to rebuild Dovecot 2.3.2.1 but the error persists. I attach a .txt file with the information requested to report a bug. My main mail server is fallen, (the same for the test server) I will send the report from a Gmail address. If you need additional information I will be at your disposal.

2 1

Need to convert mbox to Maildir
by Mark Foley 02 Oct '18

02 Oct '18

I have a mbox file of emails. I want to convert this to Maildir giving me individual message files per email. I've looked at dsync, but as far as I can tell this wants a specific target user and it appears that it will "distribute" the converted messages into that user's INBOX. I don't want to put these mbox messages into any particular user's Maildir hierarchy, just export to file-per-message format to a destination directory of my choosing. Is this possible? THX --Mark

2 2

Help for UID THREAD and SORT optimization
by Alessio Cecchi 02 Oct '18

02 Oct '18

Hello, we are developing a library to show last arrived messages of all threads in a folder with more than 300k messages. As per the RFC 5256, the IMAP thread command has only the option to specify the grouping criteria (REFERENCES vs ORDEREDSUBJECT). So we implemented an algorithm which gets the full UID SORT ordered by date in reverse order then gets all thread relations and then post process all outputs to obtain the final result. Once we have both output we loop through the first uids set (which is ordered) and for every item of it we loop through the other uids set with all the thread relations. If the uid of the first set is contained in one of the elements of the thread messages, we pick it up and push it in the resulting array. This is a pseudo-code that shows what we are doing: array_of_latest_messages_in_thread = [] array_of_sorted_uids = [n,n,n,n,...] // UID SORT (REVERSE DATE) UTF-8 ALL array_of_thread_uids = [n,[n,n],n,[n,n,n],[n,n],n,n,...] // UID THREAD (REFERENCES) UTF-8 ALL foreach(array_of_sorted_uids as s_uid){ foreach(array_of_thread_uids as t_uids){ if(t_uids contains s_uid){ // or a function to loop recursively t_uids to search if a leaf is equal to s_uid array_of_latest_messages_in_thread.push(s_uid) break } } } We have also made some little optimizations in the above code like for example skipping from the outer loop the uids already processed in the inner loop that have not being selected. Anyway, the loop is very expensive in term of computation and we are wondering if there is a better approach to this issue directly using IMAP instead of post processing the output of the two commands at application level. Thanks. -- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice

2 2

Bug in conditionals to assign values to variables?
by Angel L. Mateo 02 Oct '18

02 Oct '18

Hi, According to https://wiki2.dovecot.org/Variables you could use conditionals to assign values to a varible. The syntax is: %{if;value1;operator;value2;value-if-true;value-if-false} where any of the fields can refer to another field using %v or %{value} syntax. The problem is that when I use a config like: user_attrs = ...,=relpath=%{if;%u;eq;somevalue;valuetrue;valuefalse} it works without any problem. But when I use: user_attrs = ...,=relpath=%{if;%{user};eq;somevalue;valuetrue;valuefalse} then it reports in logs: Sep 28 14:23:22 myotis60 dovecot: auth: Error: var_expand_long(if;%{user}) failed: if: requires four or five parameters, got 1 anyway, the variable is correctly initialized, but I get the log. Is this a bug? PS: I'm running dovecot 2.2.33 -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337

2 3

split auth from other logging
by Kai Schaetzl 02 Oct '18

02 Oct '18

Is there a way to split the auth logging (logins and failed logins) from the other logging that goes to info_log_path = /var/log/dovecot/dovecot.log ? This log gets a lot of other info as well, most notably the lmtp notifications about every filed mail (with no level stamping, btw). This makes it really hard to find authentication errors quickly and comfortably. It would be nice to be able to split at least the lmtp messages away. Kai

4 9

Released Pigeonhole v0.5.3 for Dovecot v2.3.3.
by Stephan Bosch 02 Oct '18

02 Oct '18

Hello Dovecot users, Here's the Pigeonhole release that goes with Dovecot v2.3.3. Only two insignificant memory leak fixes were added since the release candidate (not mentioned in the changelog). Changelog v0.5.3: - Fix assertion panic occurring when managesieve service fails to open INBOX while saving a Sieve script. This was caused by a lack of cleanup after failure. - Fix specific messages causing an assert panic with actions that compose a reply (e.g. vacation). With some rather weird input from the original message, the header folding algorithm (as used for composing the References header for the reply) got confused, causing the panic. - IMAP FILTER=SIEVE capability: Fix FILTER SIEVE SCRIPT command parsing. After finishing reading the Sieve script, the command parsing sometimes didn't continue with the search arguments. This is a time- critical bug that likely only occurs when the Sieve script is sent in the next TCP frame. The release is available as follows: https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.3.ta… https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.3.ta… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

1 0

v2.3.3 rc1 - Error: sieve: !!BUG!!: Binary compiled from dovecot.sieve is still corrupt
by James 01 Oct '18

01 Oct '18

I have a sieve problem with dovecot v2.3.3 rc1 with either of pigeonhole 0.5.2 or 0.5.3.rc1. The the change that triggers this is in dovecot not pigeonhole. Sieve filtering has been working with v2.3.2 and many earlier versions. Log report: Sep 24 10:36:13 mailhost dovecot: [ID 702911 mail.error] lmtp(user(a)domain.tld)<15816><RnikKw2wqFvIPQAAs831Jw>: Error: sieve: !!BUG!!: Binary compiled from /path/to/domain/user/dovecot.sieve is still corrupt; bailing out and reverting to default delivery This is a sieve script with a vacation handler. My other sieve filters are working. My actual sieve filter worked before and was written by Roundcube web mail. My failing test script is cut-n-paste from the sieve examples https://wiki2.dovecot.org/Pigeonhole/Sieve/Examples so should not be in doubt. The text script compiles to dovecot.svbin and newly compiled files are identical to older versions, the compilation has not changed. James.

5 21

v2.3.3 released
by Timo Sirainen 01 Oct '18

01 Oct '18

https://dovecot.org/releases/2.3/dovecot-2.3.3.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.3.tar.gz.sig * doveconf hides more secrets now in the default output. * ssl_dh setting is no longer enforced at startup. If it's not set and non-ECC DH key exchange happens, error is logged and client is disconnected. + Added log_debug=<filter> setting. + Added log_core_filter=<log filter> setting. + quota-clone: Write to dict asynchronously + --enable-hardening attempts to use retpoline Spectre 2 mitigations + lmtp proxy: Support source_ip passdb extra field. + doveadm stats dump: Support more fields and output stddev by default. + push-notification: Add SSL support for OX backend. - NUL bytes in mail headers can cause truncated replies when fetched. - director: Conflicting host up/down state changes may in some rare situations ended up in a loop of two directors constantly overwriting each others' changes. - director: Fix hang/crash when multiple doveadm commands are being handled concurrently. - director: Fix assert-crash if doveadm disconnects too early - virtual plugin: Some searches used 100% CPU for many seconds - dsync assert-crashed with acl plugin in some situations. - mail_attachment_detection_options=add-flags-on-save assert-crashed with some specific Sieve scripts. - Mail snippet generation crashed with mails containing invalid Content-Type:multipart header. - Log prefix ordering was different for some log lines. - quota: With noenforcing option current quota usage wasn't updated. - auth: Kerberos authentication against Samba assert-crashed. - stats clients were unnecessarily chatty with the stats server. - imapc: Fixed various assert-crashes when reconnecting to server. - lmtp, submission: Fix potential crash if client disconnects while handling a command. - quota: Fixed compiling with glibc-2.26 / support libtirpc. - fts-solr: Empty search values resulted in 400 Bad Request errors - fts-solr: default_ns parameter couldn't be used - submission server crashed if relay server returned over 7 lines in a reply (e.g. to EHLO)

1 0