February 2016 - dovecot

[Dovecot] Patch: 2.0 support for URLAUTH, BURL, CATENATE
by Mike Abbott 18 Apr '16

18 Apr '16

Thank you Timo for beginning the port of URLAUTH, BURL, and CATENATE from 1.2 to 2.0. Attached please find two patches that finish the port to 2.0. One patch updates the dovecot-2.0-urlauth branch and is based on the tip of that branch. The other patch adds the full support to dovecot-2.0.beta5. The resulting URLAUTH/BURL/CATENATE code is the same. Please let me know if you have any questions about these patches. Thanks.

4 9

[Dovecot] Replication and public namespaces
by Markus Weippert 11 Apr '16

11 Apr '16

Hi, I'm having some issues with replicating public namespaces. Everything seems to work fine for private namespaces, but while importing some huge mailboxes (many small mails) into a public namespace via imapsync, something goes wrong. The expected mail flow is: old-server (imapsync)> new-server1 (replication)> new-server2 But then, dovecot seems to run into race conditions when the replications process tries to sync the same public mailbox under two or more different users at the same time. As a result, messages get duplicated, new-server2 sends those back to new-server1 which then starts to produce duplicates too. If I don't kill the processes in time and delete the faulty mailbox, they start to produce thousands of mails. In fact, server2 should not export messages at all, since it's not productive yet and does not get any mail except from the replication. The only thing getting logged (only few compared to the huge amount of duplicates produced): "dsync-server(user(a)example.com): Warning: Maildir /...: Expunged message reappeared, giving a new UID" Is there any way to fix this? Regards, Markus doveadm_password = *********** doveadm_port = ***** mail_home = /var/vmail/private/%d/%n mail_location = maildir:~/mail mail_plugins = acl virtual listescape notify replication namespace inbox { inbox = yes location = prefix = separator = / type = private } namespace legacy { alias_for = # ... } namespace ns_public { list = children location = maildir:/var/vmail/public/%d prefix = Public/ separator = / subscriptions = no type = public } namespace virtual { # ... } plugin { mail_replica = tcps:************** } protocols = imap lmtp sieve replication_dsync_parameters = -d -l 60 -N -x virtual -x ns_public -U replication_full_sync_interval = 4 hours replication_max_conns = 20 service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service doveadm { inet_listener { port = 8143 ssl = yes } process_min_avail = 5 user = vmail vsz_limit = 4 G } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } }

3 7

New feature: HTTP API
by aki.tuomi＠dovecot.fi 05 Apr '16

05 Apr '16

Hi all! We are bringing out HTTP API for doveadm-server in master HEAD. This feature is experimental, and can be expected to have bugs in it and might not always work. If you feel like trying it out though, you can enable it in your configuration file: doveadm_api_key = some-string service doveadm { inet_listener http { port = 8080 } } To use the API, you can go to http://host:8080/doveadm. GET request will print out possible commands, along with their parameters. If you have set API KEY, you need to include it in the request as X-API-Key header. This API has been designed to resemble JMAP API since we are planning to implement it in future version. The rule of thumb is that all the commands work as they would on cli, so the parameters are more or less the same. There are few exceptions, like 'field' parameter in some mail/mailbox commands. Commands are executed by POSTin JSON to /doveadm. The JSON format is [ ["command",{"parameter":"value"},"opaque identifier"], ["command",{"parameter":"value"},"opaque identifier"], ["command",{"parameter":"value"},"opaque identifier"], ... ] Be warned though that even if it supports multiple commands, some commands still use i_fatal to terminate the server if they are unhappy. This will mean you will lose the whole result set, and get no output, so at this point it is safest option to run only one command at a time. If you run into problems or discover issues, please report them in the usual manner to us. Kind regards, Aki Tuomi Dovecot Oy

2 3

imapc tries to fetch uid 0
by Nagy, Attila 30 Mar '16

30 Mar '16

Hi, This is a capture from an imapc client session (which is triggered by a message, received via LMTP): * OK [CAPABILITY IMAP4rev1 NAMESPACE] Ready 2 LOGIN "uid" "pass" 2 OK LOGIN succeeded 1 LIST "" "" * LIST (\Noselect) "/" "" 1 OK LIST completed 3 APPEND "INBOX" {568} + Ready for 568 octets of data [...] 3 OK APPEND complete 4 EXAMINE "INBOX" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 4 EXISTS * 0 RECENT * OK [UIDVALIDITY 1455885804] * OK [UIDNEXT 5] 4 OK [READ-ONLY] EXAMINE successful 5 UID FETCH 0 (RFC822.SIZE BODY.PEEK[HEADER.FIELDS (Message-ID)]) 5 BAD Illegal syntax: Bad message number 0 Dovecot 2.2.21 I don't see any problems with the conversation otherwise. Is it safe to assume this is a bug in imapc?

3 3

LMTP proxy does not pass RCPT TO: ... 5xx response back
by Heiko Schlittermann 22 Mar '16

22 Mar '16

Hello, I'm using dovecot 2.2.9 and a director/backend setup. On the director I've the LMTP in proxy mode, mapping the users to one of the backends. The backends to quota check and return the OverQuota message already at RCPT TO time. Here is what I typed, connected to the director Connection to director1 2525 port [tcp/*] succeeded! 220 director1.rz.hs-example.de Dovecot (Ubuntu) ready. LHLO mailhub1.rz.hs-example.de 250-director1.rz.hs-example.de 250-8BITMIME 250-ENHANCEDSTATUSCODES 250 PIPELINING MAIL FROM:<hs(a)schlittermann.de> 250 2.1.0 OK RCPT TO:<heiko(a)hs-example.de> 250 2.1.5 OK And here is, what TCPDUMP sees (cut for clarity): 00:22:23.029251 IP6 2001:638:914:f33::5:1.59466 > 2001:638:914:f33::5:ff.2525: Flags [S], 00:22:23.029376 IP6 2001:638:914:f33::5:ff.2525 > 2001:638:914:f33::5:1.59466: Flags [S.], 00:22:23.029660 IP6 2001:638:914:f33::5:1.59466 > 2001:638:914:f33::5:ff.2525: Flags [.], 00:22:23.051436 IP6 2001:638:914:f33::5:ff.2525 > 2001:638:914:f33::5:1.59466: Flags [P.], .7i`.7jN220 backend1.rz.hs-example.de Dovecot (Ubuntu) ready. 00:22:23.051805 IP6 2001:638:914:f33::5:1.59466 > 2001:638:914:f33::5:ff.2525: Flags [.], 00:22:23.052017 IP6 2001:638:914:f33::5:1.59466 > 2001:638:914:f33::5:ff.2525: Flags [P.], .7jT.7i`LHLO director1.rz.hs-example.de 00:22:23.052034 IP6 2001:638:914:f33::5:ff.2525 > 2001:638:914:f33::5:1.59466: Flags [.], 00:22:23.052114 IP6 2001:638:914:f33::5:ff.2525 > 2001:638:914:f33::5:1.59466: Flags [P.], .7ia.7jT250-backend1.rz.hs-example.de 250-XCLIENT ADDR PORT TTL TIMEOUT 250-8BITMIME 250-ENHANCEDSTATUSCODES 250 PIPELINING 00:22:23.052476 IP6 2001:638:914:f33::5:1.59466 > 2001:638:914:f33::5:ff.2525: Flags [P.], 0EH....d...... ..3........ ..8 ..3.........J ..... .7jT.7iaXCLIENT ADDR=2001:638:914:f33::7:1 PORT=60574 TTL=4 TIMEOUT=30 00:22:23.052540 IP6 2001:638:914:f33::5:ff.2525 > 2001:638:914:f33::5:1.59466: Flags [P.], 0EH...E....~e.....3........ ..8 ..3........ ..J .7ia.7jT220 backend1.rz.hs-example.de Dovecot (Ubuntu) ready. 00:22:23.052815 IP6 2001:638:914:f33::5:1.59466 > 2001:638:914:f33::5:ff.2525: Flags [P.], 0E.....s...... ..3........ ..8 ..3.........J ....E .7jT.7iaLHLO director1.rz.hs-example.de 00:22:23.052870 IP6 2001:638:914:f33::5:ff.2525 > 2001:638:914:f33::5:1.59466: Flags [P.], 0E....h....~......3........ ..8 ..3........ ..J .7ia.7jT250-backend1.rz.hs-example.de 250-8BITMIME 250-ENHANCEDSTATUSCODES 250 PIPELINING 00:22:23.053120 IP6 2001:638:914:f33::5:1.59466 > 2001:638:914:f33::5:ff.2525: Flags [P.], 0E.....qJ..... ..3........ ..8 ..3.........J ....h .7jT.7iaMAIL FROM:<hs(a)schlittermann.de> RCPT TO:<heiko(a)hs-example.de> 00:22:23.091824 IP6 2001:638:914:f33::5:ff.2525 > 2001:638:914:f33::5:1.59466: Flags [.], 0E.........~,.....3........ ..8 ..3........ ..J .7ik.7jT 00:22:23.119918 IP6 2001:638:914:f33::5:ff.2525 > 2001:638:914:f33::5:1.59466: Flags [P.], 0E.........~......3........ ..8 ..3........ ..J * .7ir.7jT250 2.1.0 OK * 552 5.2.2 <heiko(a)hs-example.de> Quota exceeded (mailbox for user is full) 00:22:23.158836 IP6 2001:638:914:f33::5:1.59466 > 2001:638:914:f33::5:ff.2525: Flags [.], 0F4....j...... ..3........ ..8 ..3.........J ..... .7jo.7ir 00:27:23.029008 IP6 2001:638:914:f33::5:1.59466 > 2001:638:914:f33::5:ff.2525: Flags [F.], 0F4....E...... ..3........ ..8 ..3.........J ..... It looks as if the backend tells the director/proxy about the full mailbox (552 5.2.2 <heiko(a)hs-example.de> Quota exceeded (mailbox for user is full)) already before the DATA phase starts, right as the response to the RCPT TO. But the proxy seems to ignore it… Any suggestion? Best regards from Dresden/Germany Viele Grüße aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ -

2 3

subscribe users mailbox with non-valid mutf7 chars
by Matthias Lay 16 Mar '16

16 Mar '16

Hi all, I got a question if its possible to subscribe a mailbox from another user, who contains non mutf7 valid chars in his mailbox path. I have a private namespace which can subscribe to mailboxes of a shared namespace with public/ prefix in this example username in private namespace is johnd public username is "a & p products group" when I try the subscription I get this one: ******************************************** # doveadm mailbox subscribe -u johnd "public/a & p products group/INBOX" doveadm(johnd): Error: Can't public/a & p products group/INBOX mailbox subscribe to: Invalid mailbox name 'public/a & p products group/INBOX': Missing namespace prefix 'public/a &- p products group/' ******************************************** is there a way to work around this? (except for not to use usernames like this) greetz matze

1 1

Timout for LDAP connection
by Gordon Grubert 11 Mar '16

11 Mar '16

Hi, we are using a round robin dns record for connections to our ldap system. This works fine for almost all cases. In particular, for dovecot does this mean, when an ldap server is stopped, dovecot instantly reconnects to another ldap server. But when the network connection to the active ldap server is broken, dovecot sticks to the failed ldap server. Is there any possibility to define a connection timeout? This is our current configuration for the ldap connection: # 2.2.devel (2d8f665): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.devel (0de2a19) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.3 uris = ldaps://LDAP-SERVER dn = BINDDN dnpass = BINDPASS auth_bind = yes ldap_version = 3 base = BASEDN scope = subtree user_attrs= ATTRIBUTES user_filter = USERFILTER pass_filter = PASSFILTER iterate_filter = ITERATEFILTER Best regards, Gordon -- Technischer Leiter & stellv. Direktor Universitätsrechenzentrum (URZ) E.-M.-Arndt-Universität Greifswald Felix-Hausdorff-Str. 12 17489 Greifswald Germany Tel. +49 3834 86 1456 Fax. +49 3834 86 1401

5 14

dovecot local indexes and multiple servers
by alex 05 Mar '16

05 Mar '16

Hi, I've been testing mdbox with local indexes on our nfs based directors + multiple imap/lda/pop3 server setup. The performance is noticeably better than maildir with indexes on nfs, so I'd like to make this work. About the possible direction to other servers the wiki says that it will create a new index 'on the fly'. That is true, but after the index is recreated, Thunderbird (imap) re-downloads all the messages and sees all those messages as unread again! This is probably quite logical as the message status is only kept in the indexes, but this is not very customer friendly. Is there a way to avoid this problem? I'd rather not manually sync those local indexes between our servers. In fact, I was planning on deleting (old) indexes when a user moved to another server or even if they haven't checked their mail for XX days. with up to 10Gb mailboxes the indexes themselves can get up to 20-40Mb for a mailbox. that amounts to quite a lot of storage for 200k+ users. Thanks, Alex.

2 2

Found bug in quota-status + patch
by Thierry Coppey 03 Mar '16

03 Mar '16

Hello, I've found a bug in the quota-status util (Dovecot 2.2.21, and probably below): it always return OK (sufficient quota) because it fails to load the user properly. More specifically, the branch " if (quser == NULL) return 1; " in src/plugins/quota/quota-status.c:59 is always taken. (at least with postgresql backend, test your config with the (adapted) command below: printf "recipient=mailbox(a)example.com\nsize=10000000000\n\n" | nc inet:mailstore.example.com 12340 it should reject, unless you have no quota, or more than 10G). Therefore it is pointless to configure the quota-status service as described in http://wiki2.dovecot.org/Quota because of that bug: the mail would be always accepted from the client and rejected at the end of the (Postfix) processing by lda. It is possible to reject directly at SMTP level when the mailbox is full using Postfix and main.cf: smtpd_recipient_restrictions = ... reject_unauth_destination, ... check_policy_service inet:mailstore.example.com:12340 smtpd_end_of_data_restrictions = check_policy_service inet:mailstore.example.com:12340 However there remain a *small inconsistency* as lda will count the size of message including headers whereas smtpd_end_of_data_restrictions seems not to count them towards the message size (or additional headers might be added by other preprocessing e.g. antispam etc...) I propose the patch below to fix quota-status. Could an expert double-check that it doesn't have memory leaks? (it shouldn't but I'm not yet comfortable enough with Dovecot memory management). Best regards, Thierry C.

3 3

Deleting / Removing users
by John Krug 03 Mar '16

03 Mar '16

i have a list of users that I’ve removed from LDAP and I want to delete their mail storage. sdbox Dovecot 2.2.15.8 I have mail messages in one location and indexes in another. Should I just rm -rf /messages/<username> rm -rf /indexes/<username> Thanks, JK ----------------------------------------------------------------------- John "JK" Krug System Administrator The University of Chicago | Laboratory Schools 1362 East 59th Street | Chicago, IL 60637 Phone: (773) 834-4476 jk(a)ucls.uchicago.edu <mailto:jkallo@ucls.uchicago.edu> | http://helpdesk.ucls.uchicago.edu _______________________________________________

2 3