October 2016 - dovecot

[Bug] Mailbox aliases still broken
by azurit＠pobox.sk 20 Mar '17

20 Mar '17

Hi, about an year ago i was reporting a bug in mailbox aliases, which remains unfixed and unasnwered (probably totally ignored, don't understand why). I thought it was because the bug is old and already fixed but yesterday i upgraded to Dovecot 2.2.24 and problem persists. Here is the original report, everything, except the Dovecot version, is still correct: http://dovecot.org/list/dovecot/2015-June/101176.html Will this be fixed? Thanks for info. azur

4 7

Dict protocol changes string
by Nagy, Attila 13 Mar '17

13 Mar '17

Hi, I use pass and userdb with dict protocol in a similar way: key passdb { key = passdb^MAuth-User: %u^MAuth-Pass: %w^MAuth-Protocol: %s^MClient-IP: %r format = json } (^M is an \r character, inserted with vi CTRL-v + enter) Until 2.2.24 this has worked, but 2.2.25 seems to convert that ASCII 13 into an ASCII 1 and an "r". Python printout from what I get with 2.2.25: 'Lshared/passdb\x01rAuth-User: user\x01rAuth-Pass: pass\x01rAuth-Protocol: pop3\x01rClient-IP: 1.2.3.4' Is this change intentional? Why?

2 4

Sieve Script Replication Gliches (Report #2)
by Reuben Farrelly 26 Feb '17

26 Feb '17

Hi, I've observed some odd behaviour with dsync replication between two hosts, specifically to do with sieve script replication. In short, I have two hosts which replicate in a master-master type setup where almost all of the reads and writes happen to just one of the two hosts. They are both running 2.2.devel (9dc6403), which is close to the latest 2.2 -git . Pigeonhole is running master-0.4 . This is on x86_64 Gentoo. Normal mail replication between Maildir's for all users works fine, however it appears that something recently committed to the code has broken sieve script replication between the two. I am sure this did once work. Replication is via tcps: . Sieve scripts on the lesser-used host are not up to date by an order of days/weeks with the main host and they don't seem to re-replicate - even if the rules don't exist at all on the replica. The symptoms and effects look to be the same as this (unanswered) post from December: http://dovecot.org/list/dovecot/2015-December/102690.html I am not sure how to view the transaction log files, but I am seeing the same symptoms, ie no live replication, and on the lesser-used host almost all the scripts were old and some had the 1970 date on them. Even after forcing a [dsync replication replicate '*'] the scripts are not replicated. As it stands now there are no sieve scripts on one of the two members and the system seems unable to replicate by itself. Secondly, I am also seeing some doubled up outputs if I run 'doveadm sieve list -A': thunderstorm reuben # doveadm sieve list -A reuben rules ACTIVE liam rules ACTIVE kaylene rules ACTIVE reuben rules ACTIVE liam rules ACTIVE kaylene rules ACTIVE ... Has anyone else experienced the replication problem? Are sieve scripts actually replicating in live time for other 2.2.24/2.2.25 users as well? For me I didn't notice this till I went looking so I wonder if other people are experiencing this but just not aware of it yet...? Reuben

4 9

Replication sieve scripts.
by Luescher Claude 26 Feb '17

26 Feb '17

Hello, Following up on old thread: http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html I have the exact same issue with 2.2.10: ii dovecot-antispam 2.0+20130822-2 amd64 Dovecot plugins for training spam filters ii dovecot-core 1:2.2.10-1 amd64 secure POP3/IMAP server - core files ii dovecot-imapd 1:2.2.10-1 amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-ldap 1:2.2.10-1 amd64 secure POP3/IMAP server - LDAP support ii dovecot-lmtpd 1:2.2.10-1 amd64 secure POP3/IMAP server - LMTP server ii dovecot-managesieved 1:2.2.10-1 amd64 secure POP3/IMAP server - ManageSieve server ii dovecot-mysql 1:2.2.10-1 amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 1:2.2.10-1 amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 1:2.2.10-1 amd64 secure POP3/IMAP server - Sieve filters support As even his version should already support sieve replication through dsync mine would have to support it for sure but I would like to hear a confirmation from the developers. The mail sync just works perfectly between the 2 nodes, it is only the sieve scripts which don't get replicated. I have tried to add additional sieve related options to the replicator: service replicator { process_min_avail = 1 } plugin { mail_replica = tcp:1.2.3.4 replication_full_sync_interval = 1 hours #Added (makes no difference) sieve = ~/.dovecot.sieve sieve_dir = ~/sieve_after sieve_extensions = +vnd.dovecot.duplicate sieve_duplicate_period = 1h #Added (makes no difference) } service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } It did not help. If there is one way I would like to get this work without upgrading dovecot. Thank you.

4 12

antispam plugin pipe backend error when moving multiple emails
by Stéphane Cottin 12 Feb '17

12 Feb '17

Hi, I've got some trouble with the dovecot antispam plugin and the pipe backend. I'm using dovecot 2.2.18 with maildirs and zlib compression enabled. When moving 2 or more emails at once from the Junk folder to another one, I always have the following error : "Failed to copy to temporary file" In the server logs : imap(vvv(a)vvv.vvv): Error: read(zlib(/data/Maildir/.test/tmp/1446974366.M123890P936.vvv)) failed: Cached message size smaller than expected (13553 < 13562, box=test, UID=0) The same operation with one email at a time, on the same emails, works as expected. Stéphane

2 2

Reappearing emails
by Ron Cleven 17 Jan '17

17 Jan '17

We have been running Dovecot 2.2.10 with a pair of CentOS 7 boxes with replications for the past year. We have been quite happy with the performance and reliability. Recently we received a report from an Android IMAP user that emails that he deleted without reading would often reappear in his INBOX. At first I thought it might be some quirky incompatibility with his email client, but I found that I could easily reproduce it in Roundcube by dragging an unread email to Trash. I have been monitoring this forum fairly religiously and don't recall seeing any similar reports. I plan to capture the IMAP wire interactions, but before doing that, I thought I would submit this to the IMAP gods to see if anyone had heard of this and/or if there is some newer version that might resolve the problem. This particular customer contacted his email vendor and asked them to analyze the problem. The actual log data is gigantic, so I just pasted in the vendor's narrative below in case it helps. Otherwise, as I said, I will capture a similar interaction at the wire level and follow up here. Vendor's narrative: You had 8 messages in the Inbox, and moved two to Trash. The server notified the app on the push connection (for Inbox) that there are now 6 messages. By the time the app got around to catch up with that (it did the Trash folder first), the server said "oh wait, there are 8 messages, not 6" (in the Inbox), and so the app synced them in, just as reported. app selecting the Inbox, server saying there are 8 messages there app moving UID 729 and 730 to Trash (push) server saying Trash now has 34 messages (push) server saying two messages gone from Inbox, 6 left (push) 6 messages left in Inbox becoming 8 (push) and here they are (the two new ones) Thanks, Ron

3 3

Replication: Can't unsubscribe from shared mailbox
by Karsten Heiken 16 Jan '17

16 Jan '17

Hi, I am running two dovecot servers active/active. Everything runs pretty great, except for the replication of subscriptions in a shared namespace. When I unsubscribe from a folder the subscription instantly re-appears. The timestamp on the subscriptions file is updated, but the entry is still in there. If the other node is shut down, everything works as expected, which leads me to believe that the subscription is re-applied upon a replicator-run. I attached a tcpdump of a dsync run. The only action that was executed was to unsubscribe from the folder shared/weinrot(a)luis.uni-hannover.de/INBOX. Is there any other way I might help debugging this? Thanks, Karsten # 2.2.24 (a82c823): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 2.6.32-44-pve x86_64 Debian 8.5 default_vsz_limit = 512 M dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } imap_max_line_length = 2 M lmtp_rcpt_check_quota = yes mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_gid = 7777 mail_location = maildir:%h/Maildir:LAYOUT=fs:DIRNAME=maiLdir mail_plugins = " zlib quota acl notify replication " mail_shared_explicit_inbox = yes mail_uid = 7777 mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify namespace { hidden = no ignore_on_failure = no inbox = no list = children location = maildir:%%h/Maildir:INDEXPVT=%h/shared/%%u:LAYOUT=fs:DIRNAME=maiLdir prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { hidden = no inbox = yes list = yes location = mailbox 30dTrash { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = proxy::acl mail_replica = tcp:10.6.1.10:24245 quota = maildir:Postfach-Limit quota_grace = 2%% quota_rule = *:storage=8G quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_max_actions = 250 sieve_max_redirects = 200 zlib_save = gz zlib_save_level = 6 } protocols = " imap lmtp sieve pop3 sieve" replication_dsync_parameters = -d -l 300 -U -N service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener auth-userdb { group = vmail user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { port = 24245 } } service imap-login { process_min_avail = 16 service_count = 0 } service imap { executable = imap postlogin process_limit = 30000 } service lmtp { inet_listener lmtp { port = 24 } vsz_limit = 1 G } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 1024 } service pop3-login { process_min_avail = 4 service_count = 0 } service pop3 { process_limit = 2500 } service postlogin { executable = script-login -d rawlog /usr/local/bin/dovecot-postlogin.sh } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 12340 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } shutdown_clients = no ssl_cert = </etc/dovecot/private/fullchain.pem ssl_key = </etc/dovecot/private/privkey.pem syslog_facility = local6 userdb { args = /etc/dovecot/dovecot-ldap.conf.ext default_fields = home=/var/vmail/%Ld/%Ln quota_rule=*:bytes=8589934592 driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = " zlib quota acl notify replication sieve quota" } protocol lda { mail_plugins = " zlib quota acl notify replication sieve" } protocol imap { imap_metadata = yes mail_max_userip_connections = 100 mail_plugins = " zlib quota acl notify replication imap_quota imap_acl" } protocol pop3 { mail_max_userip_connections = 10 mail_plugins = " zlib quota acl notify replication" }

2 2

RFC 3501 violation in FETCH BODY responses
by Guilhem Moulin 18 Dec '16

18 Dec '16

Hi there, Quoting RFC 3501 sec. 7.4.2 “FETCH Response” (data item BODYSTRUCTURE): “A body type of type MESSAGE and subtype RFC822 contains, immediately after the basic fields, the envelope structure, body structure, and size in text lines of the encapsulated message.” According the ABNF (RFC 3501 sec. 9) the envelope structure is that of the ENVELOPE FETCH data item, and the env-{from,sender,reply-to,to,cc, bcc} fields are non-space-separated address lists: body-type-msg = media-message SP body-fields SP envelope SP body SP body-fld-lines envelope = "(" env-from SP … SP env-to SP … ")" env-from = "(" 1*address ")" / nil env-to = "(" 1*address ")" / nil While this is indeed the case for ‘FETCH … (ENVELOPE)’, for ‘FETCH … (BODY)’ dovecot 2.2.25 adds a space between addresses of an address list of the envelope structure of an encapsulated MESSAGE/RFC822 message. See the attached patch to ‘src/lib-imap/test-imap-bodystructure.c’, which currently (2.2.25) fails as follows test-imap-bodystructure.c:122: Assert failed: strcmp(str_c(str), testmsg_body) == 0 test-imap-bodystructure.c:129: Assert failed: strcmp(str_c(str), testmsg_bodystructure) == 0 imap bodystructure parser ............................................ : FAILED because the ‘env-to’ field of the envelope structure of the encapsulated MESSAGE/RFC822 message is printed as ((NIL NIL "sub-to1" "domain.org") (NIL NIL "sub-to2" "domain.org")) while it should be ((NIL NIL "sub-to1" "domain.org")(NIL NIL "sub-to2" "domain.org")) After a quick look at the source, this seems to be due to src/lib-imap/imap-bodystructure.c:imap_write_list, which always separates list items with spaces. In the case of an envelope, only the top-level list should be space-separated. Indeed, not adding a space between items of type IMAP_ARG_LIST in the recursive call makes the test pass again. Cheers, -- Guilhem.

2 1

RE: First steps in Dovecot; IMAP not working
by Richard 04 Dec '16

04 Dec '16

> Date: Tuesday, October 18, 2016 13:12:39 +0200 > From: Moi <anic297(a)mac.com> > > Thank you. It worked and I now have several log files to check. > > In the meantime, I've tried once again to send a message to > "mailtest" from an outside address; this time, I got an error reply: > > This report relates to a message you sent with the following header > fields: > > Message-id: <00e201d2290f$67da8c70$378fa550$(a)mac.com> > Date: Tue, 18 Oct 2016 09:15:21 +0200 > From: Moi > To: 'Mail Test' <mailtest(a)barbu.sytes.net> > Subject: Test > > Your message cannot be delivered to the following recipients: > > Recipient address: mailtest(a)barbu.sytes.net > Reason: Illegal host/domain name found > <end of reply> > > Yet another area with a problem; at least this is now a valid > reason for it to not work. > Is this a misconfiguration of my DNS server? Assuming that "barbu.sytes.net" is the intended hostname (not something made up to obscure the real name), there is an MX-record for that that points to "mail.barbu.sytes.net", but there is no A-record for the "mail." hostname. There is an A-record for "mail.sytes.net", in case that is what you were intending, in which case you'd need to fix the MX on "barbu.sytes.net".

2 1

Defining INDEX target to other location than maildir seems to have no effect.
by Julian Sternberg 02 Dec '16

02 Dec '16

Dovecot Version 2.2.13 Linux Distribution: Debian Jessie CPU Architecture: x64 Filesystem: GlusterFS/NFS, XFS for Base System/Index Files. Two Dovecot/Postfix nodes accessing same GlusterFS/NFS Maildir. Regardless what i choose in mail_location (:INDEX=MEMORY or :INDEX=/var/indexes/%d/%n) all Mailbox index files will still get created within the users mail_location maildir: ~/Maildir My mail_location Maildir directory is shared on GlusterFS mount so i need to get index files away from this share due locking mechanism and faster caching. The weird is, if i set INDEX to /var/indexes, the index files getting created sometimes but then not updated frequently and exist parallel on Maildir which are mostly newer than on /var/indexes. If you delete the dovecot.index* files from Maildir, they will get recreated immediatly on Imap access but not on the alternative set INDEX location, they are getting recreated within Maildir again. Here is doveconf -n Output: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 auth_mechanisms = plain login cram-md5 disable_plaintext_auth = no first_valid_uid = 2000 hostname = censored.hostname.com last_valid_uid = 2000 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * lock_method = dotlock mail_fsync = always mail_gid = 2000 mail_home = /storage/vmail/%d/%n mail_location = maildir:~/Maildir:LAYOUT=fs:INDEX=MEMORY mail_nfs_storage = yes mail_privileged_group = vmail mail_temp_dir = /var/tmp mail_uid = 2000 maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace inbox { inbox = yes location = mailbox Archive { auto = no special_use = \Archive } mailbox Archives { auto = no special_use = \Archive } mailbox "Deleted Items" { auto = no special_use = \Trash } mailbox "Deleted Messages" { auto = no special_use = \Trash } mailbox Drafts { auto = no special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Items" { auto = no special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = create special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } mailbox virtual/All { auto = no special_use = \All } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_rule3 = Sent:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /storage/vmail/%d/%n/sieve/dovecot.sieve sieve_before = /storage/vmail/sieve/dovecot.sieve sieve_dir = /storage/vmail/%d/%n/sieve sieve_global = /storage/vmail/sieve } postmaster_address = postmaster(a)censored.hostname.com protocols = " imap lmtp sieve pop3" quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service imap-login { inet_listener imaps { port = 993 ssl = yes } service_count = 0 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0666 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_ca = </etc/ssl/censored.hostname.com/ssl-bundle-mail.crt ssl_cert = </etc/ssl/censored.hostname.com/censored_hostname.com.crt ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA ssl_key = </etc/ssl/censored.hostname.com/censored.hostname.com.key ssl_protocols = !SSLv2 !SSLv3 ssl_require_crl = no userdb { args = uid=2000 gid=2000 home=/storage/vmail/%d/%n allow_all_users=yes driver = static } protocol lmtp { mail_plugins = " quota sieve" } protocol lda { mail_plugins = " sieve quota" } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = " quota imap_quota" } protocol sieve { mail_max_userip_connections = 10 } protocol pop3 { mail_plugins = " quota" pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv }

3 3