September 2015 - dovecot

Patch for "doveadm -f table" nit (was Re: Dovecot current number of connections being used.)
by Joseph Tam 08 Sep '15

08 Sep '15

> doveadm who -1 2>/dev/null | wc -l You have to redirect stderr to /dev/null because that's where the first header line is written to. The default format style (table) is inconsistent with the other formats (flow,pager,tab) that write headers and data to stdout. The following patch will pick this nit. This patch will require modifications to scripts that rely on doveadm writing headers to stderr. For example, the above doveadm command could be modified to doveadm who -1 | grep -vc '^username' -------------------------------------------------------------------------------- --- a/dovecot-2.2.16.rc1/src/doveadm/doveadm-print-table.c Wed Mar 18 15:40:40 2015 +++ b/dovecot-2.2.16.rc1/src/doveadm/doveadm-print-table.c Wed Mar 18 15:41:18 2015 @@ -151,3 +151,3 @@ for (i = 0; i < count; i++) { - if (i > 0) fprintf(stderr, " "); + if (i > 0) printf(" "); @@ -155,3 +155,3 @@ DOVEADM_PRINT_HEADER_FLAG_RIGHT_JUSTIFY) == 0) { - fprintf(stderr, "%-*s", (int)headers[i].length, + printf("%-*s", (int)headers[i].length, headers[i].title); @@ -158,3 +158,3 @@ } else { - fprintf(stderr, "%*s", (int)headers[i].length, + printf("%*s", (int)headers[i].length, headers[i].title); @@ -162,3 +162,3 @@ } - fprintf(stderr, "\n"); + printf("\n"); } -------------------------------------------------------------------------------- Joseph Tam <jtam.home(a)gmail.com>

5 9

Re: Null deference pointer in dovecot-2.2.18
by 吴迪 08 Sep '15

08 Sep '15

Dear, use our static analysis tools, I find some bugs (Null deference pointer) for dovecot-2.2.18. Null deference pointer bugs often make program crashes, Please confim them, Thanks! 1. dovecot-2.2.18/src/config/config-request.c 332 'setting_export_section_name(ctx->prefix, def, children[i], i);', pointer 'children' in line 202 assigned NULL and if branch 'case SET_DEFLIST_UNIQUE' not execute, so pointer 'children' is always NULL. It load to a bug of null deference pointer In line 332. The same bugs also appeared in : 1. dovecot-2.2.18/src/lib-index/mail-index-fsck.c line 170 or 174, pointer 'kw_rec ' mybe NULL. 2.dovecot-2.2.18/src/lib-storage/mail-search-args-simplify line 349, pointer 'prev_arg' mybe NULL. best wishes~ Amy

3 2

RFC 5465 (NOTIFY) violation: missing HIGHESTMODSEQ in initial STATUS responses
by Guilhem Moulin 08 Sep '15

08 Sep '15

Quoting RFC 5465 (NOTIFY): “If the NOTIFY command enables MessageNew, MessageExpunge, AnnotationChange, or FlagChange notifications for a mailbox other than the currently selected mailbox, and the client has specified the STATUS indicator parameter, then the server MUST send a STATUS response for that mailbox before NOTIFY's tagged OK. […] If either AnnotationChange or FlagChange are included and the server also supports the CONDSTORE [RFC4551] and/or QRESYNC [RFC5162] extensions, the STATUS response MUST contain UIDVALIDITY and HIGHESTMODSEQ.” — https://tools.ietf.org/html/rfc5465#section-3.1 While unsolicited STATUS responses include HIGHESTMODSEQ indeed, the initial STATUS responses (caused by the presence of the STATUS indicator) do not: ~$ /usr/lib/dovecot/imap * PREAUTH [CAPABILITY IMAP4rev1 … CONDSTORE QRESYNC … NOTIFY SPECIAL-USE] Logged in as guilhem a ENABLE QRESYNC * ENABLED QRESYNC a OK Enabled (0.000 secs). b NOTIFY SET STATUS (SUBSCRIBED (MessageNew MessageExpunge FlagChange)) * STATUS INBOX (MESSAGES 9069 UIDNEXT 109398 UIDVALIDITY 1312585007 UNSEEN 0) […] b OK NOTIFY completed (0.008 secs). [time passes… a new message is delivered to INBOX] * STATUS INBOX (MESSAGES 9070 UIDNEXT 109399 UNSEEN 1 HIGHESTMODSEQ 22216) This defeats the purpose of the STATUS indicator for disconnected clients since they have to issue separate STATUS commands (or a LIST command if LIST-{EXTENDED,STATUS} have been advertized) to find out which mailboxes have got a new HIGHESTMODSEQ. Cheers, -- Guilhem.

2 1

NOTIFY regression: 2.18 no longer notifies of events in INBOX
by Guilhem Moulin 08 Sep '15

08 Sep '15

Here is an example with APPEND $ /usr/lib/dovecot/imap S1: * PREAUTH [CAPABILITY IMAP4rev1 … MOVE NOTIFY SPECIAL-USE] Logged in as guilhem C1: a1 NOTIFY SET (INBOXES (MessageNew MessageExpunge FlagChange)) S1: a1 OK NOTIFY completed (0.000 secs) $ /usr/lib/dovecot/imap S2: * PREAUTH [CAPABILITY IMAP4rev1 … MOVE NOTIFY SPECIAL-USE] Logged in as guilhem C2: a2 APPEND INBOX {1+} C2: x S2: a2 OK [APPENDUID 1384472528 26085] Append completed (0.229 secs). With 2.13 from Debian Jessie, S1 sends a notification for the new message in INBOX, as expected: S1: * STATUS INBOX (MESSAGES 4333 UIDNEXT 26086 UNSEEN 1) However it doesn't with 2.18 from Debian Sid, as if ‘INBOX’ was excluded from the mailbox filter ‘INBOXES’. It does send a notification for ‘virtual/unseen’ instead (as expected), but nothing for ‘INBOX’. S1: * STATUS virtual/unseen (MESSAGES 3 UIDNEXT 15186 UNSEEN 3) This is INBOX-specific because APPENDING the message to another mailbox triggers the notification on both 2.13 and 2.18 as expected: S1: * STATUS test (MESSAGES 2 UIDNEXT 3 UNSEEN 2) S1: * STATUS virtual/unseen (MESSAGES 2 UIDNEXT 15195 UNSEEN 2) I attach the ‘dovecot -n’ output for both 2.13 and 2.18. -- Guilhem.

2 1

"NOTIFY SET (mailboxes INBOX (...))" crashes the IMAP client
by Guilhem Moulin 07 Sep '15

07 Sep '15

Hi list, The NOTIFY extension (RFC 5465) works fine for mailboxes in the "virtual/" namespace, but it crashes the IMAP client when used with a mailbox in the empty "" namespace: ~$ /usr/lib/dovecot/imap * PREAUTH [CAPABILITY IMAP4rev1 … NOTIFY SPECIAL-USE] Logged in as guilhem a NAMESPACE * NAMESPACE (("" "/")("virtual/" "/")) NIL NIL a OK Namespace completed. b NOTIFY SET (mailboxes virtual/all (MessageNew MessageExpunge)) b OK NOTIFY completed (0.001 secs). c NOTIFY SET (mailboxes INBOX (MessageNew MessageExpunge)) imap(guilhem): Panic: file mail-namespace.c: line 679 (mail_namespace_find): assertion failed: (ns != NULL) imap(guilhem): Error: Raw backtrace: … Aborted The "subtree" mailbox filter has the same problem, but the non-parameterized ones ("inboxes", "personal" and "subscribed") work fine. You'll find the output of ‘dovecot -n’ enclosed. Cheers, -- Guilhem.

2 3

question on autch cache parameters
by matthias lay 07 Sep '15

07 Sep '15

Hi list, I have a question on auth caching in 2.2.18. I am using acl_groups for a master user, appended in a static userdb file # snip ############################### master@uma:{SHA}XXXX=::::::userdb_acl_groups=umareadmaster allow_nets=127.0.0.1 # snap ############################### and use this group in a global ACL file. I discovered this only works on first NOT-cached login environment in imap-postlogin script on first login: AUTH_TOKEN=e96b5a32ceb2cafc4460c210ad2e92e3d7ab388c MASTER_USER=master@uma SPUSER=private/pdf LOCAL_IP=127.0.0.1 USER=pdf AUTH_USER=master@uma PWD=/var/run/dovecot USERDB_KEYS=ACL_GROUPS HOME SPUSER MASTER_USER AUTH_TOKEN AUTH_USER SHLVL=1 HOME=/var/data/vmail/private/pdf ACL_GROUPS=umareadmaster IP=127.0.0.1 _=/usr/bin/env on the second cached login it looks like this AUTH_TOKEN=12703b11932f233520f6d4b33559c33aeb1cfc7f MASTER_USER=master@uma SPUSER=private/pdf LOCAL_IP=127.0.0.1 USER=pdf AUTH_USER=master@uma PWD=/var/run/dovecot USERDB_KEYS=HOME SPUSER MASTER_USER AUTH_TOKEN AUTH_USER SHLVL=1 HOME=/var/data/vmail/private/pdf IP=127.0.0.1 _=/usr/bin/env so the ACL_GROUPS is gone. is this intended to be like that. so groups not included in cache and I have to find another approach? anybody else encountered similar problems with some auth Variables and caching? Greetz Matze

4 6

doveadm quota recalc problem for domain
by Samuel 06 Sep '15

06 Sep '15

Hello, I've set up user quota and domain quota for dict SQL and all seems OK. New mail for user increase the user quota and the domain quota. Deleting email count down user quota and domain quota. But in case of SQL failure, I'm trying to rebuild the user quota and domain quota with doveadm quota recalc. The problem is that the command : 'doveadm quota recalc -u foo@bar' sets the domain quota only with user foo@bar even when I try a domain quota recalc : doveadm quota recalc -A or doveadm quota recalc -u *@bar (only one user is taken in account for the domain quota and not all the user). The domain SQL database is broken with value of only one user for the whole domain quota. So how to achieve rebuild domain quota correctly with doveadm ? Thanks a lot. Samuel. Debian Jessie Dovecot : 2.2.13 custom grsec kernel :Linux postfix 3.14.50-grsec-1 ########################## dovecot -n : # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.14.50-grsec-1 x86_64 Debian 8.1 nfs4 auth_mechanisms = plain login auth_verbose = yes dict { quota = mysql:/etc/dovecot/dovecot-dict-sql-user.conf quota_domain = mysql:/etc/dovecot/dovecot-dict-sql-domain.conf } lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_trusted_networks = XXXXXX/24 mail_location = maildir:/maildir/%d/%n/Maildir mail_plugins = " quota" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:user::proxy::quota quota2 = dict:domain:%d:proxy::quota_domain quota_grace = 10M quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO quota_warning = storage=100%% quota-warning 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=80%% quota-warning 80 %u quota_warning4 = -storage=100%% quota-warning below %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve lmtp service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service dict { unix_listener dict { mode = 0666 user = dovecot } } service quota-status { executable = quota-status -p postfix inet_listener { port = 12340 } } service quota-warning { client_limit = 1 executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } ssl_cert = </etc/dovecot/dovecot.pem ssl_key = </etc/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = " quota sieve" } protocol imap { mail_plugins = " quota imap_quota" } ################ root@postfix:/etc/dovecot# cat dovecot-dict-sql-domain.conf |grep -v "#" connect = host=XXXXX dbname=mail user=test password=test map { pattern = priv/quota/storage table = domains username_field = domain value_field = quota } map { pattern = priv/quota/messages table = domains username_field = domain value_field = quota_messages } map { pattern = shared/expire/$user/$mailbox table = expires value_field = expire_stamp fields { username = $user mailbox = $mailbox } } ################### root@postfix:/etc/dovecot# cat dovecot-dict-sql-user.conf |grep -v "#" connect = host=XXXXXXXX dbname=mail user=test password=test map { pattern = priv/quota/storage table = quota username_field = email value_field = bytes } map { pattern = priv/quota/messages table = quota username_field = email value_field = messages } map { pattern = shared/expire/$user/$mailbox table = expires value_field = expire_stamp fields { username = $user mailbox = $mailbox } } ############################

1 0

Mailbox can't be created
by Paulo Matos 06 Sep '15

06 Sep '15

Hi, I have setup postfix master.cf to use dovecot-lda to deliver (using -m) all messages to PreInbox: dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/libexec/dovecot/deliver -f ${sender} -a ${recipient} -d ${user}@${nexthop} -m PreINBOX And enabled lda_mailbox_autocreate. The message I have now it: Aug 27 10:02:29 lda(): Info: msgid=<dae42d0cf57bf4496a3dd53e162f8281(a)localhost.localdomain>: save failed to open mailbox PreINBOX: Mailbox can't be created Aug 27 10:02:29 lda(): Info: msgid=<dae42d0cf57bf4496a3dd53e162f8281(a)localhost.localdomain>: saved mail to INBOX Why wouldn't dovecot itself be able to create mailboxes? I am using virtual domains and users on a mysql database through postfix and mailboxes saved in MailDir format. Configuration of dovecot is: # 2.2.18: /etc/dovecot/dovecot.conf # OS: Linux 4.1.5-x86_64-linode61 x86_64 Fedora release 22 (Twenty Two) ext4 auth_debug = yes auth_mechanisms = plain login digest-md5 cram-md5 auth_verbose = yes auth_verbose_passwords = yes default_internal_user = root imap_client_workarounds = delay-newmail tb-extra-mailbox-sep lda_mailbox_autocreate = yes mail_debug = yes mail_location = maildir:/home/vmail/%d/%n/Maildir maildir_very_dirty_syncs = yes mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } postmaster_address = pmatos protocols = imap quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { mode = 0600 user = vmail } user = $default_internal_user } ssl = required ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes driver = static } protocol lda { auth_socket_path = /var/run/dovecot/auth-master deliver_log_format = msgid=%m: %$ log_path = /home/vmail/dovecot-deliver.log } protocol imap { mail_max_userip_connections = 100 } Regards, -- Paulo Matos

4 14

Considerations for a not so simple set-up ...
by Lars Hanke 05 Sep '15

05 Sep '15

I want to set up a new IMAP server with Dovecot, so far I've been running Cyrus. I'm running dovecot on Debian Jessie. The general idea of the system would be: 1) Virtual users, i.e. login users are disjoint from any mail accounts. 2) Real-Users authenticate using their kerberos tickets. 3) There is a n-to-n mapping of real to virtual users. 4) Maildirs shall be stored on glusterfs. 5) Fallback passwords for cloud access Concerning 1) and 2) I keep reading that this is possible, but I could not find any concise and current description. I'd welcome pointers to howtos. Concerning 3) this should be a matter of ACL, e.g. a single real user (kerberos pricipal) may have several mail-boxes (virtual users), and groups of real users may share the same mail-box. I have a faint idea, how this could be done in Cyrus (but never did that), but I read that these features should exist in Dovecot as well. But I'm completely lost to even figure out a starting point. I appreciate any keywords for further reading. Concerning 4) I read that there are some issues using NFS. Are there any caveats when using gluster? Concerning 5) Users logging in from outside may not have TGT on their notebooks, which by design should run as stand-alone. They should nevertheless be able to access their mail. I currently think of another set of real users, but anything less messy is welcome. Did someone try something similar, already? Thanks for your help, - lars.

1 0

'doveadm expunge' with -A and userdb { driver = passwd }
by Grant 05 Sep '15

05 Sep '15

I'm expunging mail in my crontab like this: doveadm search -u user1 mailbox INBOX.Trash savedbefore 30d doveadm search -u user2 mailbox INBOX.Trash savedbefore 30d doveadm search -u user3 mailbox INBOX.Trash savedbefore 30d ... I'd rather do this: doveadm search -A mailbox INBOX.Trash savedbefore 30d but I'm told not to: "If the -A option is present, the command will be performed for all users. Using this option in combination with system users from userdb { driver = passwd } is not recommended, because it contains also users with a lower UID than the one configured with the first_valid_uid setting." I'm using userdb { driver = passwd-file } but I'm guessing that's the same thing. Am I understanding this correctly? - Grant

3 7