November 2011 - dovecot

[Dovecot] zlib_save per namespace/mailbox?
by Lutz Preßler 29 Mar '12

29 Mar '12

Hi, the zlib_save question reminds me of a wish: I think it's not possible to set zlib_save parameter per namespace (or even mailbox). Per namespace would be something for the wish list to get rid of the cron job method to compress archival mailboxes. And maybe an option to add a "Z" flag to compressed maildir message files as recommended in the wiki regarding compress crob job. Lutz

3 2

[Dovecot] Some Dovecot LDA version sends MDN rather than DSN for overquota.
by Mark Sapiro 12 Feb '12

12 Feb '12

It appears that there is some version of the Dovecot LDA that is sending an RFC 3798 MDN rather than an RFC 3464 DSN to report non-delivery to a user who is over quota. See <https://bugs.launchpad.net/mailman/+bug/693134> for a discussion of this and <https://bugs.launchpad.net/mailman/+bug/693134/+attachment/1773018/+files/A…> for the actual MDN message. The issue is that apparently a user was over quota and Dovecot returned an MDN with a disposition of deleted. This is intended to mean the the mail was delivered to the user but the user deleted it without reading it. If the message was actually not delivered, this should be reported to the MTA so it can return an appropriate SMTP status, or the LDA should send an RFC 3464 DSN with an action of failed. Note that I do not actually use the Dovecot LDA nor do I have any access to the system where the notice in the bug report was received, so I may have incomplete information and this may not be relevant to current versions of Dovecot, but I think it is wrong for Dovecot to report over quota non-delivery via an MDN. -- Mark Sapiro <mark(a)msapiro.net> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan

2 3

[Dovecot] Quota Calculation seems to be wrong when using dsync
by Thomas Leuxner 12 Feb '12

12 Feb '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Recently I encountered odd quota increases when using 'dsync' with the latest and greatest: 2.0.8 (89936539e3b8). Say you got something in place like this: plugin { quota = dict:user::file:%h/mdbox/dovecot-quota quota_rule = *:storage=1GB quota_rule2 = Trash:storage=+10%% } Kick off a manual backup: $ dsync -u user(a)domain.tld backup mdbox:/<path to backup>/mdbox This results in doubling the quota for the backed up user. It doesn't seem to matter whether 'mirror' or 'backup' mode is used, both increase the "source" quota. Doesn't sound right to me. You can only get to the "real" usage by running a recalc manually: $ doveadm quota recalc -u user(a)domain.tld Anyone else seen this? Regards Thomas P.S. Merry Christmas -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) iEYEARECAAYFAk0VtH8ACgkQ+meF/S97aXjUkwCfdZFpUPCOY9UiukT+G68jCHHr mmMAnRHz56BXMzpC84CIYvUn4KHpl8OA =M0AZ -----END PGP SIGNATURE-----

2 9

[Dovecot] [PATCH] Allow verification of user's hash in doveadm-pw
by Spack 12 Feb '12

12 Feb '12

Hi, It may be useful for testing purpose to know if a generated password hash correspond to a given clear password. This can be useful to check if a hash generated by another program can be verified by Dovecot without any errors. This patch adds the ability the verify a password hash using `doveadm pw` via the `-V` option. $ doveadm pw -s SSHA.hex Enter new password: Retype new password: {SSHA.HEX}58b910d947c60b35be3e12b0d9897c1f87dfa450e6d5a75c $ doveadm pw -s SSHA.hex -V{SSHA.HEX}58b910d947c60b35be3e12b0d9897c1f87dfa450e6d5a75c Enter new password: Retype new password: {SSHA.HEX}4fdf801f57870fb624bee60895c2308187837dfb3987ca06 (verified) When the user specify a hash, it is checked in place of the usual verification process. This works with all schemes. Just be sure to escape special characters or use simple quotes. $ doveadm pw -s MD5-CRYPT -V'{MD5-CRYPT}$1$R7thDyN.$E0G0czPE/h4S0Hn/b5oVW1' Enter new password: Retype new password: {MD5-CRYPT}$1$rp8nVSqq$gDAPM5iqv1yhBoDZIWsQ4/ (verified) Lets print a failed check: doveadm pw -s CRYPT -V{CRYPT}/uuE/AAAAAAAA Enter new password: Retype new password: reverse password verification check failed FYI the word used for those test cases was "password". Regards, ___ Jimmy Thrasibule <spack.world(a)gmail.com>

2 1

[Dovecot] Strange behavior from shared namespaces and INBOX, probably a bug
by Christoph Bussenius 09 Feb '12

09 Feb '12

Hi, we have experienced some erratic behavior from Dovecot 2.0.15 if a user's INBOX is shared. Some folders of user1, including the INBOX, have been shared using these IMAP commands: . login user1 XXXXXX . setacl INBOX user2 lrwstiekx . setacl box-a user2 lrwstiekx Now if we use telnet to log in as user2 and select "shared/user1", it will contain the same mails as "shared/user1/INBOX". The really strange thing is that "SELECT"-ing "shared/user1" succeeds only if it is the first command afted logging in. If it not the first command (if e. g. the "LIST" or "SELECT" command has already been used), then dovecot will report that the mailbox does not exist. As a result, user agents like Thunderbird will behave very erratically in this setup. They may show both mailboxes ("shared/user1" and "shared/user1/INBOX") with the same mails, but will not always succeed to download mails for "shared/user1". We were able to reproduce this behavior with a quite simple configuration (see further below), so I doubt that the problem is in the configuration. So, is this a bug in Dovecot? I am not quite sure how exactly it should behave, but it definitely does not seem right this way. Below are the results from telnet: Sending: . login user2 XXXXXX . select shared/user1 Receiving: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 5 EXISTS * 0 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1316698162] UIDs valid * OK [UIDNEXT 6] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. Sending: . fetch 1:2 (internaldate rfc822.size) Receiving: * 1 FETCH (INTERNALDATE "25-Apr-2006 23:09:07 +0200" RFC822.SIZE 5660) * 2 FETCH (INTERNALDATE "25-Apr-2006 23:11:01 +0200" RFC822.SIZE 6612) . OK Fetch completed. Sending: . select shared/user1/INBOX Receiving: * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 5 EXISTS * 0 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1316698162] UIDs valid * OK [UIDNEXT 6] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. Sending: . fetch 1:2 (internaldate rfc822.size) Receiving: (Note that these mails are the same as those above) * 1 FETCH (INTERNALDATE "25-Apr-2006 23:09:07 +0200" RFC822.SIZE 5660) * 2 FETCH (INTERNALDATE "25-Apr-2006 23:11:01 +0200" RFC822.SIZE 6612) . OK Fetch completed. Now we select the first mailbox again: Sending: . select shared/user1 Receiving: . NO Mailbox doesn't exist: shared/user1 * OK [CLOSED] Previous mailbox closed. The same command did not fail earlier. Listing the mailboxes shows the following: Sending: . list "" * Receiving: * LIST (\HasNoChildren) "/" "INBOX" * LIST (\Noselect \HasChildren) "/" "shared/user1" * LIST (\HasNoChildren) "/" "shared/user1/box-a" * LIST (\HasNoChildren) "/" "shared/user1/INBOX" . OK List completed. I can reproduce this behavior using the following setup: Virtual users on an ext4 filesystem in /mail/users/{user1,user2} configured in a passwd file (/usr/local/dovecot/etc/dovecot/users): user1:{plain}XXXXXX:::user1:/mail/users/user1:: user2:{plain}XXXXXX:::user2:/mail/users/user2:: Output of doveconf -n: # 2.0.15: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-server x86_64 Ubuntu 10.04.3 LTS disable_plaintext_auth = no mail_gid = vmail mail_plugins = acl mail_uid = vmail namespace { inbox = yes location = maildir:~/Maildir prefix = separator = / } namespace { list = children location = maildir:%%h/Maildir prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/mail/shared-mailboxes } protocols = imap service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } protocol imap { mail_plugins = acl imap_acl } Cheers, Christoph Bußenius -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik TU München +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching

2 1

[Dovecot] Segfaul probably during dsync
by Matthias Rieber 09 Feb '12

09 Feb '12

Hi, I get core dumps, probably during dsync: Core was generated by `dovecot/imap postlogin'. Program terminated with signal 6, Aborted. [New process 7727] #0 0x00002ba911c04ed5 in raise () from /lib/libc.so.6 #0 0x00002ba911c04ed5 in raise () from /lib/libc.so.6 No symbol table info available. #1 0x00002ba911c063f3 in abort () from /lib/libc.so.6 No symbol table info available. #2 0x00002ba91157d775 in default_fatal_finish (type=<value optimized out>, status=0) at failures.c:187 backtrace = 0xcebd838 "/usr/local/lib/dovecot/libdovecot.so.0 [0x2ba91157d762] -> /usr/local/lib/dovecot/libdovecot.so.0 [0x2ba91157d7c6] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x2ba91157dbb8] -> /usr/local/l"... #3 0x00002ba91157d7c6 in i_internal_fatal_handler (ctx=0x7fffadba32a0, format=<value optimized out>, args=<value optimized out>) at failures.c:645 status = 0 #4 0x00002ba91157dbb8 in i_panic (format=<value optimized out>) at failures.c:259 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffadba3370, reg_save_area = 0x7fffadba32b0}} #5 0x00002ba9112b4018 in index_search_result_update_flags (result=0xdebe3e0, uids=0x7fffadba3640) at index-search-result.c:87 search_arg = {next = 0xd25b098, type = SEARCH_UIDSET, value = {subargs = 0x0, seqset = {arr = {buffer = 0xcebd640, element_size = 8}, v = 0xcebd640, v_modifiable = 0xcebd640}, str = 0x0, time = 0, size = 0, flags = 0, search_flags = 0, date_type = 0, thread_type = MAIL_THREAD_NONE, keywords = 0x0, modseq = 0x0, search_args = 0x0, search_result = 0x0, mailbox_glob = 0x0}, context = 0x0, hdr_field_name = 0x0, not = 0, match_always = 0, nonmatch_always = 0, result = -1} ret = <value optimized out> __FUNCTION__ = "index_search_result_update_flags" #6 0x00002ba91276ac92 in virtual_sync_backend_box (ctx=0xdebcdb0, bbox=0xdcb8e38) at virtual-sync.c:725 sync_flags = <value optimized out> status = {messages = 121, recent = 0, unseen = 111, uidvalidity = 1225530594, uidnext = 182153, first_unseen_seq = 0, first_recent_uid = 182152, highest_modseq = 0, virtual_size = 0, keywords = 0x0, cache_fields = 0x0, nonpermanent_modseqs = 0} ret = <value optimized out> __FUNCTION__ = "virtual_sync_backend_box" #7 0x00002ba91276b661 in virtual_storage_sync_init (box=0xcf524c0, flags=MAILBOX_SYNC_FLAG_FIX_INCONSISTENT) at virtual-sync.c:1400 sync_ctx = <value optimized out> ret = <value optimized out> #8 0x00002ba911299b71 in mailbox_sync_init (box=0xcf524c0, flags=MAILBOX_SYNC_FLAG_FIX_INCONSISTENT) at mail-storage.c:947 _data_stack_cur_id = 0 ctx = <value optimized out> #9 0x00002ba91129a337 in mailbox_sync (box=0x1e2f, flags=7727) at mail-storage.c:993 ctx = <value optimized out> status = {sync_delayed_expunges = 0} #10 0x0000000000417602 in imap_status_get (cmd=0xcf00410, ns=0xcefccd0, mailbox=0xcebd330 "rapidsolution", items=0x7fffadba3a50, result_r=0x7fffadba39e0, error_r=0x7fffadba3a38) at imap-status.c:85 client = (struct client *) 0xceffca0 box = (struct mailbox *) 0x1e2f error = <value optimized out> ret = <value optimized out> #11 0x000000000040e763 in cmd_status (cmd=0xcf00410) at cmd-status.c:52 client = (struct client *) 0xceffca0 status = MAILBOX_NAME_EXISTS_MAILBOX args = (const struct imap_arg *) 0xcf02c68 list_args = (const struct imap_arg *) 0xcf02da8 items = {mailbox_items = 17, guid = 0} result = {status = {messages = 124, recent = 0, unseen = 0, uidvalidity = 1301953092, uidnext = 1, first_unseen_seq = 0, first_recent_uid = 0, highest_modseq = 0, virtual_size = 48005140525491, keywords = 0x0, cache_fields = 0x80, nonpermanent_modseqs = 0}, mailbox_guid = "{µW\021©+\000\000\200\000\000\000\000\000\000"} ns = <value optimized out> mailbox = 0xcf02d58 "virtual.rapidsolution" storage_name = 0xcebd330 "rapidsolution" error = <value optimized out> #12 0x000000000041055c in client_command_input (cmd=0xcf00410) at imap-client.c:672 client = (struct client *) 0xceffca0 command = <value optimized out> __FUNCTION__ = "client_command_input" #13 0x000000000041060d in client_command_input (cmd=0xcf00410) at imap-client.c:722 client = (struct client *) 0xceffca0 command = (struct command *) 0x6 __FUNCTION__ = "client_command_input" #14 0x0000000000410785 in client_handle_input (client=0xceffca0) at imap-client.c:763 _data_stack_cur_id = 3 ret = 54 remove_io = <value optimized out> handled_commands = false __FUNCTION__ = "client_handle_input" #15 0x00000000004111a2 in client_input (client=0xceffca0) at imap-client.c:814 cmd = <value optimized out> output = (struct ostream *) 0xcf00288 bytes = <value optimized out> __FUNCTION__ = "client_input" #16 0x00002ba911588e38 in io_loop_call_io (io=0xcf00320) at ioloop.c:384 ioloop = (struct ioloop *) 0xcec5620 t_id = 2 #17 0x00002ba911589dc2 in io_loop_handler_run (ioloop=<value optimized out>) at ioloop-epoll.c:213 ctx = (struct ioloop_handler_context *) 0xcec5850 event = (const struct epoll_event *) 0xcec58c0 list = (struct io_list *) 0xcf00370 io = (struct io_file *) 0x1e2f tv = {tv_sec = 4, tv_usec = 514891} msecs = <value optimized out> ret = <value optimized out> call = false #18 0x00002ba911588dc8 in io_loop_run (ioloop=0xcec5620) at ioloop.c:405 No locals. #19 0x00002ba911576c93 in master_service_run (service=0xcec54f0, callback=0x1e2f) at master-service.c:478 No locals. #20 0x0000000000419401 in main (argc=2, argv=0xcec5370) at main.c:375 service_flags = <value optimized out> storage_service_flags = <value optimized out> postlogin_socket_path = 0xcebd080 "/usr/local/var/run/dovecot/postlogin" username = 0x0 c = <value optimized out> set_roots = {0x41c680, 0x0} Logfile: Apr 10 13:49:34 shrike dovecot: imap-login: Login: user=<matthias>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=14235, secured Apr 10 13:49:34 shrike dovecot: (null)Error: rawlog_open: open() failed for /home/matthias/dovecot.rawlog/20110410-134934-14237.in: Permission denied Apr 10 13:49:43 shrike dovecot: imap(matthias): Panic: file index-search-result.c: line 87 (search_result_update_search): assertion failed: (next_uid == mail->uid) Apr 10 13:49:43 shrike dovecot: imap(matthias): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0 [0x2b4df76ce762] -> /usr/local/lib/dovecot/libdovecot.so.0 [0x2b4df76ce7c6] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x2b4df76cebb8] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_search_result_update_flags+0x2d8) [0x2b4df7405018] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so [0x2b4df88bbc92] -> /usr/local/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0x871) [0x2b4df88bc661] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x2b4df73eab71] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x2b4df73eb337] -> dovecot/imap(imap_status_get+0x92) [0x417602] -> dovecot/imap(cmd_status+0x163) [0x40e763] -> dovecot/imap [0x41055c] -> dovecot/imap [0x41060d] -> dovecot/imap(client_handle_input+0x45) [0x410785] -> dovecot/imap(client_input+0x62) [0x4111a2] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0x2b4df Apr 10 13:49:43 shrike dovecot: master: Error: service(imap): child 14235 killed with signal 6 (core dumped) # dovecot --version 2.0.11 (cef23de90ff1) Matthias

2 3

[Dovecot] BUG(?): Incorrect responses for ACL prohibited actions
by Michael M Slusarz 09 Feb '12

09 Feb '12

In a mailbox that has ACL restrictions on both DELETE and EXPUNGE (i.e. no 'e' or 't' rights), I see the following: 3 UID STORE 3173 (UNCHANGEDSINCE 353) +FLAGS \deleted 3 OK Store completed. (UID 3173 is not flagged \deleted) [...] 6 UID EXPUNGE 1:* 6 OK Expunge completed. (At least 1 UID is flagged \deleted in mailbox) Shouldn't these commands be returning "NO" instead of "OK"? RFC 3501 [6.4.6] for STORE: NO - store error: can't store that data and RFC RFC 3501 [6.4.3] for EXPUNGE: NO - expunge failure: can't expunge (e.g., permission denied) Additionally, RFC 5530 [3] provides the NOPERM response code: NOPERM The access control system (e.g., Access Control List (ACL), see [RFC 4314]) does not permit this user to carry out an operation, such as selecting or creating a mailbox. C: f select "/archive/projects/experiment-iv" S: f NO [NOPERM] Access denied My reading of this is that NOPERM should be returned for ANY ACL prohibited action, not just for selecting or creating a mailbox. Dovecot 2.0.12 does not return NOPERM for DELETE/EXPUNGE actions (at a minimum) that are prohibited. Thoughts? michael

2 3

[Dovecot] [dovecot] Getting duplicates when using snarf plugin with mbox backend
by Jonathan SIegle 09 Feb '12

09 Feb '12

Running dovecot version 2.0.11. To reproduce, open two imap sessions and issue a check command from each at the same time with new mail in the queue. 0 login testuser testpw 1 select inbox -- Deliver mail -- 2 check Thanks, Jonathan

2 1

[Dovecot] fts (lucene): indexing of virtual mailboxes?
by Lutz Preßler 09 Feb '12

09 Feb '12

Hello, (recent 2.1alpha2 variant - my test setup known to Timo). No time to diagnose in depth at the moment, but I just noticed that SEARCHing in virtual mailboxes seems not to create lucene index content of its own but use those of referenced mailboxes? The problem is that no new indexing takes place. Example: with INBOX INBOX.in% all in dovecot-virtual, for a given query I only get matches from those mailboxes searched in previously. Regards, Lutz

2 2

[Dovecot] [PATCH] Bad boundary check in client_find_namespace
by Christoph Bussenius 09 Feb '12

09 Feb '12

Hi, while trying to investigate the bug I reported last week, I found that there is a broken boundary check in client_find_namespace in src/imap/imap-commands-util.c. The code is: /* make sure two hierarchy separators aren't next to each others */ for (p = storage_name+1; *p != '\0'; p++) { if (p[0] == ns->real_sep && p[-1] == ns->real_sep) { client_send_tagline(cmd, "NO Invalid mailbox name."); return NULL; } } The loop iterates over the string storage_name starting from its second byte. However, the string may be "". This is the case if you select the root of a namespace, like in "SELECT shared" or "SELECT shared/user1". In that case, the code will read past the end of the buffer from random memory until it finds a zero byte or a duplicate separator. This would fix it: diff -ru dovecot-2.0.15.orig/src/imap/imap-commands-util.c dovecot-2.0.15/src/imap/imap-commands-util.c --- dovecot-2.0.15.orig/src/imap/imap-commands-util.c 2011-08-02 12:29:37.000000000 +0200 +++ dovecot-2.0.15/src/imap/imap-commands-util.c 2011-09-26 18:33:16.121917759 +0200 @@ -81,7 +81,9 @@ } /* make sure two hierarchy separators aren't next to each others */ - for (p = storage_name+1; *p != '\0'; p++) { + for (p = storage_name; *p != '\0'; p++) { + if (p == storage_name) + continue; if (p[0] == ns->real_sep && p[-1] == ns->real_sep) { client_send_tagline(cmd, "NO Invalid mailbox name."); return NULL; Cheers, Christoph Bußenius -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik TU München +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching

3 2