July 2015 - dovecot - dovecot.org

Patch for "doveadm -f table" nit (was Re: Dovecot current number of connections being used.)
by Joseph Tam 08 Sep '15

08 Sep '15

> doveadm who -1 2>/dev/null | wc -l You have to redirect stderr to /dev/null because that's where the first header line is written to. The default format style (table) is inconsistent with the other formats (flow,pager,tab) that write headers and data to stdout. The following patch will pick this nit. This patch will require modifications to scripts that rely on doveadm writing headers to stderr. For example, the above doveadm command could be modified to doveadm who -1 | grep -vc '^username' -------------------------------------------------------------------------------- --- a/dovecot-2.2.16.rc1/src/doveadm/doveadm-print-table.c Wed Mar 18 15:40:40 2015 +++ b/dovecot-2.2.16.rc1/src/doveadm/doveadm-print-table.c Wed Mar 18 15:41:18 2015 @@ -151,3 +151,3 @@ for (i = 0; i < count; i++) { - if (i > 0) fprintf(stderr, " "); + if (i > 0) printf(" "); @@ -155,3 +155,3 @@ DOVEADM_PRINT_HEADER_FLAG_RIGHT_JUSTIFY) == 0) { - fprintf(stderr, "%-*s", (int)headers[i].length, + printf("%-*s", (int)headers[i].length, headers[i].title); @@ -158,3 +158,3 @@ } else { - fprintf(stderr, "%*s", (int)headers[i].length, + printf("%*s", (int)headers[i].length, headers[i].title); @@ -162,3 +162,3 @@ } - fprintf(stderr, "\n"); + printf("\n"); } -------------------------------------------------------------------------------- Joseph Tam <jtam.home(a)gmail.com>

5 9

Re: Null deference pointer in dovecot-2.2.18
by 吴迪 08 Sep '15

08 Sep '15

Dear, use our static analysis tools, I find some bugs (Null deference pointer) for dovecot-2.2.18. Null deference pointer bugs often make program crashes, Please confim them, Thanks! 1. dovecot-2.2.18/src/config/config-request.c 332 'setting_export_section_name(ctx->prefix, def, children[i], i);', pointer 'children' in line 202 assigned NULL and if branch 'case SET_DEFLIST_UNIQUE' not execute, so pointer 'children' is always NULL. It load to a bug of null deference pointer In line 332. The same bugs also appeared in : 1. dovecot-2.2.18/src/lib-index/mail-index-fsck.c line 170 or 174, pointer 'kw_rec ' mybe NULL. 2.dovecot-2.2.18/src/lib-storage/mail-search-args-simplify line 349, pointer 'prev_arg' mybe NULL. best wishes~ Amy

3 2

RFC 5465 (NOTIFY) violation: missing HIGHESTMODSEQ in initial STATUS responses
by Guilhem Moulin 08 Sep '15

08 Sep '15

Quoting RFC 5465 (NOTIFY): “If the NOTIFY command enables MessageNew, MessageExpunge, AnnotationChange, or FlagChange notifications for a mailbox other than the currently selected mailbox, and the client has specified the STATUS indicator parameter, then the server MUST send a STATUS response for that mailbox before NOTIFY's tagged OK. […] If either AnnotationChange or FlagChange are included and the server also supports the CONDSTORE [RFC4551] and/or QRESYNC [RFC5162] extensions, the STATUS response MUST contain UIDVALIDITY and HIGHESTMODSEQ.” — https://tools.ietf.org/html/rfc5465#section-3.1 While unsolicited STATUS responses include HIGHESTMODSEQ indeed, the initial STATUS responses (caused by the presence of the STATUS indicator) do not: ~$ /usr/lib/dovecot/imap * PREAUTH [CAPABILITY IMAP4rev1 … CONDSTORE QRESYNC … NOTIFY SPECIAL-USE] Logged in as guilhem a ENABLE QRESYNC * ENABLED QRESYNC a OK Enabled (0.000 secs). b NOTIFY SET STATUS (SUBSCRIBED (MessageNew MessageExpunge FlagChange)) * STATUS INBOX (MESSAGES 9069 UIDNEXT 109398 UIDVALIDITY 1312585007 UNSEEN 0) […] b OK NOTIFY completed (0.008 secs). [time passes… a new message is delivered to INBOX] * STATUS INBOX (MESSAGES 9070 UIDNEXT 109399 UNSEEN 1 HIGHESTMODSEQ 22216) This defeats the purpose of the STATUS indicator for disconnected clients since they have to issue separate STATUS commands (or a LIST command if LIST-{EXTENDED,STATUS} have been advertized) to find out which mailboxes have got a new HIGHESTMODSEQ. Cheers, -- Guilhem.

2 1

NOTIFY regression: 2.18 no longer notifies of events in INBOX
by Guilhem Moulin 08 Sep '15

08 Sep '15

Here is an example with APPEND $ /usr/lib/dovecot/imap S1: * PREAUTH [CAPABILITY IMAP4rev1 … MOVE NOTIFY SPECIAL-USE] Logged in as guilhem C1: a1 NOTIFY SET (INBOXES (MessageNew MessageExpunge FlagChange)) S1: a1 OK NOTIFY completed (0.000 secs) $ /usr/lib/dovecot/imap S2: * PREAUTH [CAPABILITY IMAP4rev1 … MOVE NOTIFY SPECIAL-USE] Logged in as guilhem C2: a2 APPEND INBOX {1+} C2: x S2: a2 OK [APPENDUID 1384472528 26085] Append completed (0.229 secs). With 2.13 from Debian Jessie, S1 sends a notification for the new message in INBOX, as expected: S1: * STATUS INBOX (MESSAGES 4333 UIDNEXT 26086 UNSEEN 1) However it doesn't with 2.18 from Debian Sid, as if ‘INBOX’ was excluded from the mailbox filter ‘INBOXES’. It does send a notification for ‘virtual/unseen’ instead (as expected), but nothing for ‘INBOX’. S1: * STATUS virtual/unseen (MESSAGES 3 UIDNEXT 15186 UNSEEN 3) This is INBOX-specific because APPENDING the message to another mailbox triggers the notification on both 2.13 and 2.18 as expected: S1: * STATUS test (MESSAGES 2 UIDNEXT 3 UNSEEN 2) S1: * STATUS virtual/unseen (MESSAGES 2 UIDNEXT 15195 UNSEEN 2) I attach the ‘dovecot -n’ output for both 2.13 and 2.18. -- Guilhem.

2 1

"NOTIFY SET (mailboxes INBOX (...))" crashes the IMAP client
by Guilhem Moulin 07 Sep '15

07 Sep '15

Hi list, The NOTIFY extension (RFC 5465) works fine for mailboxes in the "virtual/" namespace, but it crashes the IMAP client when used with a mailbox in the empty "" namespace: ~$ /usr/lib/dovecot/imap * PREAUTH [CAPABILITY IMAP4rev1 … NOTIFY SPECIAL-USE] Logged in as guilhem a NAMESPACE * NAMESPACE (("" "/")("virtual/" "/")) NIL NIL a OK Namespace completed. b NOTIFY SET (mailboxes virtual/all (MessageNew MessageExpunge)) b OK NOTIFY completed (0.001 secs). c NOTIFY SET (mailboxes INBOX (MessageNew MessageExpunge)) imap(guilhem): Panic: file mail-namespace.c: line 679 (mail_namespace_find): assertion failed: (ns != NULL) imap(guilhem): Error: Raw backtrace: … Aborted The "subtree" mailbox filter has the same problem, but the non-parameterized ones ("inboxes", "personal" and "subscribed") work fine. You'll find the output of ‘dovecot -n’ enclosed. Cheers, -- Guilhem.

2 3

internal server error
by Sergey Schwartz 03 Sep '15

03 Sep '15

Hi! Today I've got a client unable to open a mailbox with Thunderbird - constantly drops connection. On the server side I can see an error message in the log Jul 22 11:07:29 mx10 dovecot: imap(oleg.vasilyev(a)bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7c96f) [0x7fd52575b96f] -> /usr/lib/dovecot/libdovecot.so.0(+0x7c9ce) [0x7fd52575b9ce] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fd525709875] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x74) [0x7fd525aa72d4] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_add+0x561) [0x7fd525a91cf1] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_parse_header+0x2b1) [0x7fd525a79241] -> /usr/lib/dovecot/libdovecot.so.0(+0x66e47) [0x7fd525745e47] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x68) [0x7fd5257657a8] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7fd52576614d] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header_next+0x62) [0x7fd52574af02] -> /usr/lib/dovecot/libdovecot.so.0(+0x667ff) [0x7fd5257457ff] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x68) [0x7fd5257657a8] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7fd52576614d] -> /usr/lib/dovecot/libdovecot.so.0(message_get_header_size+0x53) [0x7fd52574e4b3] -> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x257) [0x7fd525ab4527] -> dovecot/imap(+0x1da42) [0x7fd525f3ca42] -> dovecot/imap(+0x1be3f) [0x7fd525f3ae3f] -> dovecot/imap(imap_fetch_more+0x37) [0x7fd525f3be97] -> dovecot/imap(cmd_fetch+0x316) [0x7fd525f2f946] -> dovecot/imap(command_exec+0x4c) [0x7fd525f398cc] -> dovecot/imap(+0x19830) [0x7fd525f38830] -> dovecot/imap(+0x198c4) [0x7fd525f388c4] -> dovecot/imap(client_handle_input+0x115) [0x7fd525f38bc5] -> dovecot/imap(client_input+0x75) [0x7fd525f38fa5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7fd52576e54b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7fd52576f5cb] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7fd52576e5d9] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fd52576e658] Jul 22 11:07:29 mx10 dovecot: imap(oleg.vasilyev(a)bgoperator.com): Fatal: master: service(imap): child 17109 killed with signal 6 (core dumped) Should I report a bug or there is a workaround for this issue? -- Best regards, Sergey Schwartz Senior System Administrator Biblio Globus Tour Operator www.bgoperator.ru T: +7 495 5042500 ext 1532 E: sergey.schwartz(a)bgoperator.com

4 6

doveadm-server hangs with 100% cpu usage
by Dennis Kuhn 02 Sep '15

02 Sep '15

we have some replication issues. From time to time a doveadm-server process takes 100% cpu in the state recv_mailbox_tree_deletes on the replica. The process runs forever until it is manually killed. Strace on this process doesn't show anything. Sometimes we have several doveadm-server processes in this state, all for the same account, all with 100% CPU Load. The logfile on the master says: Error: Timeout during state=recv_mailbox_tree Error: dsync(dobby5.heinlein-support.de): I/O has stalled, no activity for 600 seconds My workaround is to delete the user directory on the replica so that the whole account is replicated again. This solves the problem for this specific account. The dovecot version is 2.2.15 on the master and 2.2.16 on the replica. Dennis Kuhn -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-57 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

3 4

mailbox deletion is not replicated correctly
by Dennis Kuhn 27 Aug '15

27 Aug '15

i can't delete several folders at once. I have a folder structure like folder1/folder2/folder3 (we use slash as hierarchy separator). When i delete folder1 server1 executes the command correctly Apr 16 13:44:28 server1 dovecot: imap(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2/folder3 Apr 16 13:44:28 server1 dovecot: imap(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2 Apr 16 13:44:28 server1 dovecot: imap(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1 but the replica server2 reverses the order: Apr 16 13:44:29 server2 dovecot: dsync-server(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1 Apr 16 13:44:29 server2 dovecot: dsync-server(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2 Apr 16 13:44:29 server2 dovecot: dsync-server(d.kuhn(a)heinlein-support.de): Mailbox deleted: INBOX/Trash/folder1/folder2/folder3 This creates the folder INBOX/Trash/folder1/folder2 on server2 again which replicates back to server1. Regards, Dennis Kuhn -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-57 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

3 4

PROXY protocol
by Hoggins! 26 Aug '15

26 Aug '15

Hello folks, Any plans on implementing the PROXY protocol to allow Dovecot being behind a TCP proxy, and still logging the real IP address of the users ? See : http://blog.haproxy.com/haproxy/proxy-protocol/ Thanks !

6 15

ldap attribute modifiers. how to lowercase non ASCII
by matthias lay 24 Aug '15

24 Aug '15

hi all, I encountered some problem with lowercasing chars like an "Ä" with user_attrs like this: user_attrs = uid=home=/mail/%Lu, =myvar=%L{ldap:uid} the Ä doesnt get lower cased in both cases # doveadm user BLÄTULA field value user blÄtula uid vmail gid vmail home /mail/blÄtula mail maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX myvar blÄtula somebody knows if there a fix for that? Greetz Matze

2 2