December 2016 - dovecot

Replication sieve scripts.
by Luescher Claude 26 Feb '17

26 Feb '17

Hello, Following up on old thread: http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html I have the exact same issue with 2.2.10: ii dovecot-antispam 2.0+20130822-2 amd64 Dovecot plugins for training spam filters ii dovecot-core 1:2.2.10-1 amd64 secure POP3/IMAP server - core files ii dovecot-imapd 1:2.2.10-1 amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-ldap 1:2.2.10-1 amd64 secure POP3/IMAP server - LDAP support ii dovecot-lmtpd 1:2.2.10-1 amd64 secure POP3/IMAP server - LMTP server ii dovecot-managesieved 1:2.2.10-1 amd64 secure POP3/IMAP server - ManageSieve server ii dovecot-mysql 1:2.2.10-1 amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 1:2.2.10-1 amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 1:2.2.10-1 amd64 secure POP3/IMAP server - Sieve filters support As even his version should already support sieve replication through dsync mine would have to support it for sure but I would like to hear a confirmation from the developers. The mail sync just works perfectly between the 2 nodes, it is only the sieve scripts which don't get replicated. I have tried to add additional sieve related options to the replicator: service replicator { process_min_avail = 1 } plugin { mail_replica = tcp:1.2.3.4 replication_full_sync_interval = 1 hours #Added (makes no difference) sieve = ~/.dovecot.sieve sieve_dir = ~/sieve_after sieve_extensions = +vnd.dovecot.duplicate sieve_duplicate_period = 1h #Added (makes no difference) } service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } It did not help. If there is one way I would like to get this work without upgrading dovecot. Thank you.

4 12

antispam plugin pipe backend error when moving multiple emails
by Stéphane Cottin 12 Feb '17

12 Feb '17

Hi, I've got some trouble with the dovecot antispam plugin and the pipe backend. I'm using dovecot 2.2.18 with maildirs and zlib compression enabled. When moving 2 or more emails at once from the Junk folder to another one, I always have the following error : "Failed to copy to temporary file" In the server logs : imap(vvv(a)vvv.vvv): Error: read(zlib(/data/Maildir/.test/tmp/1446974366.M123890P936.vvv)) failed: Cached message size smaller than expected (13553 < 13562, box=test, UID=0) The same operation with one email at a time, on the same emails, works as expected. Stéphane

2 2

Vacation Sending To Root
by Shaun Forsyth 31 Jan '17

31 Jan '17

Dovecot Version 2.0.9 I am having an issue where all vacation messages are being sent to root. Is any one able to advise. Dovecot Log Shows :- Sieve: msgid=< CAMD5TViw0sSigGm8JY5WOv0ROs1UdCJFnvb9vo-RzcNN74E92Q(a)mail.xxxxxx.com>: sent vacation response to <root> The emails are handed off to dovecot from sendmail using procmail :0 w | $DELIVER The variable $DELIVER is set as DELIVER="/usr/libexec/dovecot/deliver" I do have DROPPRIVS=YES in procmail, however I don't think deliver works without this. Any advice would greatly help. Shaun

2 2

mail_location in userdb inconsistent with imap and pigeonhole
by Ladislav Laska 31 Jan '17

31 Jan '17

Hi! I'm still fighting with lost mails and what I think is a locking problem. As I've lost some pretty important mails already, I decided to mitigate my troubles by moving to maildir format, which should not be susceptible to these problems. I read the docs and wanted to test on my account, before I mess with all the others and possibly break something. The way to do it seems to be to do an override via userdb. I already have a passwd-file authdb, so I uncommented those few lines in configuration, and added this to my passwd-file: krakonos:{SHA256}xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=:1000:100::/home/krakonos:/bin/bash:userdb_mail=maildir:/home/krakonos/Mail Now, I'm not really sure why it has to be prefixed with userdb_, but the example in docs has it, and it seems to work. I've attached the relevant part of a logfile, with some debugging enabled, but in short: Dec 10 18:39:22 ibex dovecot: imap(krakonos): Debug: Added userdb setting: mail=maildir:/home/krakonos/Mail [...] Dec 10 18:39:22 ibex dovecot: imap(krakonos): Debug: maildir++: root=/home/krakonos/Mail, index=, indexpvt=, control=, inbox=/home/krakonos/Mail, alt= [...] Dec 10 18:39:52 ibex dovecot: lda(krakonos): Debug: mbox: INBOX defaulted to /home/krakonos/.mbox/inbox I'm confused why the last line chooses my old mbox over the new location. I'm even more confused by the fact the IMAP works fine, and as I connect from mutt over imap, I can see empty mailboxes (the ~/Mail has only the basic cur/new/tmp structure). However, pigeonhole delivers into the .mbox, and not only if it's for INBOX, but also for spam and others. I read somewhere in the docs that the order is first try userdb, then mail_location and namespaces override the rest. This might be the problem, as I have: mail_location = mbox:~/.mbox namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } Due to inbox=yes this would explain why inbox is using the default path in mail_location. But why even the spam is delivered into the old mbox? And how do I solve the problem anyway, should I just remove the namespace? Anyway, it doesn't explain why the IMAP part and pigeonhole apply different rules and show me different inboxes, which is probably a bug. Thanks! ----------------------------------------------------------------------- Here is the rest of my configuration for reference, the log is attached: ----------------------------------------------------------------------- # 2.2.26.0 (23d1de6): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.4.26-gentoo x86_64 Gentoo Base System release 2.2 auth_username_format = %n hostname = ibex.krakonos.org login_greeting = Dovecot(a)krakonos.org ready. mail_debug = yes mail_location = mbox:~/.mbox mbox_read_locks = dotlock fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = * driver = pam } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute sieve_execute_socket_dir = sieve-execute sieve_extensions = +vnd.dovecot.filter +editheader sieve_filter_bin_dir = /usr/lib/dovecot/sieve-filter sieve_filter_socket_dir = sieve-filter sieve_pipe_bin_dir = /usr/lib/dovecot/sieve-pipe sieve_pipe_socket_dir = sieve-pipe sieve_plugins = sieve_extprograms } postmaster_address = postmaster(a)krakonos.org protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } ssl_cert = </etc/ssl/dovecot/server.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { mail_plugins = sieve } -- S pozdravem Ladislav "Krakonoš" Láska http://www.krakonos.org/

2 2

Reappearing emails
by Ron Cleven 17 Jan '17

17 Jan '17

We have been running Dovecot 2.2.10 with a pair of CentOS 7 boxes with replications for the past year. We have been quite happy with the performance and reliability. Recently we received a report from an Android IMAP user that emails that he deleted without reading would often reappear in his INBOX. At first I thought it might be some quirky incompatibility with his email client, but I found that I could easily reproduce it in Roundcube by dragging an unread email to Trash. I have been monitoring this forum fairly religiously and don't recall seeing any similar reports. I plan to capture the IMAP wire interactions, but before doing that, I thought I would submit this to the IMAP gods to see if anyone had heard of this and/or if there is some newer version that might resolve the problem. This particular customer contacted his email vendor and asked them to analyze the problem. The actual log data is gigantic, so I just pasted in the vendor's narrative below in case it helps. Otherwise, as I said, I will capture a similar interaction at the wire level and follow up here. Vendor's narrative: You had 8 messages in the Inbox, and moved two to Trash. The server notified the app on the push connection (for Inbox) that there are now 6 messages. By the time the app got around to catch up with that (it did the Trash folder first), the server said "oh wait, there are 8 messages, not 6" (in the Inbox), and so the app synced them in, just as reported. app selecting the Inbox, server saying there are 8 messages there app moving UID 729 and 730 to Trash (push) server saying Trash now has 34 messages (push) server saying two messages gone from Inbox, 6 left (push) 6 messages left in Inbox becoming 8 (push) and here they are (the two new ones) Thanks, Ron

3 3

fts-solr: Returning 400 on searches; unescaped braces
by Michael Welsh Duggan 17 Jan '17

17 Jan '17

Using Debian, dovecot-solr 1:2.2.26.0-4, and solr-tomcat 3.6.2+dfsg-9, I am getting 400 errors when doing searches. Here is an example search query from dovecot that failed (captured with wireshark): Frame 23: 338 bytes on wire (2704 bits), 338 bytes captured (2704 bits) on interface 0 Linux cooked capture Internet Protocol Version 6, Src: ::1, Dst: ::1 Transmission Control Protocol, Src Port: 56860, Dst Port: 8080, Seq: 1, Ack: 1, Len: 250 Hypertext Transfer Protocol GET /solr/select?fl=uid,score&rows=2664&sort=uid+asc&q={!lucene+q.op%3dAND}(hdr:test+OR+body:test)&fq=%2Bbox:6d5de009f991854df726000012cf7b9c+%2Buser:md5i HTTP/1.1\r\n Host: localhost:8080\r\n Date: Mon, 19 Dec 2016 00:25:56 GMT\r\n Connection: Keep-Alive\r\n \r\n [Full request URI: http://localhost:8080/solr/select?fl=uid,score&rows=2664&sort=uid+asc&q={!l…] [HTTP request 1/1] [Response in frame: 25] Here is the same query from firefox, which succeeds: Frame 66: 646 bytes on wire (5168 bits), 646 bytes captured (5168 bits) on interface 0 Linux cooked capture Internet Protocol Version 6, Src: ::1, Dst: ::1 Transmission Control Protocol, Src Port: 56862, Dst Port: 8080, Seq: 1, Ack: 1, Len: 558 Hypertext Transfer Protocol GET /solr/select?fl=uid,score&rows=2664&sort=uid+asc&q=%7B!lucene+q.op%3DAND%7D(hdr:test+OR+body:test)&fq=%2Bbox:6d5de009f991854df726000012cf7b9c+%2Buser:md5i HTTP/1.1\r\n Host: localhost:8080\r\n Connection: keep-alive\r\n Cache-Control: max-age=0\r\n Upgrade-Insecure-Requests: 1\r\n User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.75 Safari/537.36\r\n Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\n DNT: 1\r\n Accept-Encoding: gzip, deflate, sdch, br\r\n Accept-Language: en-US,en;q=0.8\r\n \r\n [Full request URI: http://localhost:8080/solr/select?fl=uid,score&rows=2664&sort=uid+asc&q=%7B…] [HTTP request 1/1] [Response in frame: 86] The salient difference seems to be the encoding of the braces. Indeed in the tomcat 8 logs, I find the following which seems to corroborate my hypothesis: java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986 at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:467) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:667) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:789) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1437) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Indeed the braces are listed in fts-backend-solr.c as part of solr_escape_chars, so don't know how the braces are making it through unencoded. -- Michael Welsh Duggan (md5i(a)md5i.com)

2 3

Replication: Can't unsubscribe from shared mailbox
by Karsten Heiken 16 Jan '17

16 Jan '17

Hi, I am running two dovecot servers active/active. Everything runs pretty great, except for the replication of subscriptions in a shared namespace. When I unsubscribe from a folder the subscription instantly re-appears. The timestamp on the subscriptions file is updated, but the entry is still in there. If the other node is shut down, everything works as expected, which leads me to believe that the subscription is re-applied upon a replicator-run. I attached a tcpdump of a dsync run. The only action that was executed was to unsubscribe from the folder shared/weinrot(a)luis.uni-hannover.de/INBOX. Is there any other way I might help debugging this? Thanks, Karsten # 2.2.24 (a82c823): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 2.6.32-44-pve x86_64 Debian 8.5 default_vsz_limit = 512 M dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } imap_max_line_length = 2 M lmtp_rcpt_check_quota = yes mail_attribute_dict = file:%h/Maildir/dovecot-attributes mail_gid = 7777 mail_location = maildir:%h/Maildir:LAYOUT=fs:DIRNAME=maiLdir mail_plugins = " zlib quota acl notify replication " mail_shared_explicit_inbox = yes mail_uid = 7777 mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify namespace { hidden = no ignore_on_failure = no inbox = no list = children location = maildir:%%h/Maildir:INDEXPVT=%h/shared/%%u:LAYOUT=fs:DIRNAME=maiLdir prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { hidden = no inbox = yes list = yes location = mailbox 30dTrash { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = proxy::acl mail_replica = tcp:10.6.1.10:24245 quota = maildir:Postfach-Limit quota_grace = 2%% quota_rule = *:storage=8G quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is full quota_status_success = DUNNO sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_max_actions = 250 sieve_max_redirects = 200 zlib_save = gz zlib_save_level = 6 } protocols = " imap lmtp sieve pop3 sieve" replication_dsync_parameters = -d -l 300 -U -N service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener auth-userdb { group = vmail user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { port = 24245 } } service imap-login { process_min_avail = 16 service_count = 0 } service imap { executable = imap postlogin process_limit = 30000 } service lmtp { inet_listener lmtp { port = 24 } vsz_limit = 1 G } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 1024 } service pop3-login { process_min_avail = 4 service_count = 0 } service pop3 { process_limit = 2500 } service postlogin { executable = script-login -d rawlog /usr/local/bin/dovecot-postlogin.sh } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 12340 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } shutdown_clients = no ssl_cert = </etc/dovecot/private/fullchain.pem ssl_key = </etc/dovecot/private/privkey.pem syslog_facility = local6 userdb { args = /etc/dovecot/dovecot-ldap.conf.ext default_fields = home=/var/vmail/%Ld/%Ln quota_rule=*:bytes=8589934592 driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = " zlib quota acl notify replication sieve quota" } protocol lda { mail_plugins = " zlib quota acl notify replication sieve" } protocol imap { imap_metadata = yes mail_max_userip_connections = 100 mail_plugins = " zlib quota acl notify replication imap_quota imap_acl" } protocol pop3 { mail_max_userip_connections = 10 mail_plugins = " zlib quota acl notify replication" }

2 2

Virtual POP3 namespaces and quota-status
by Aurélien Beaujean 09 Jan '17

09 Jan '17

Hi, I've tried to do configure Virtual POP3 INBOX (dovecot version 2.2.18) as explained here http://wiki.dovecot.org/Plugins/Virtual with a static userdb and nopassword=y so I dont have any database on the dovecot server without the possibility to distinguish POP3 or IMAP in the userdb extra field to change the namespace on the fly. So I've put the namespace virtual informations inside the protocol pop3 { ... }, I did the same for the protocol imap { }: protocol pop3 { mail_plugins = $mail_plugins last_login virtual namespace inbox { prefix = separator = / list = yes } namespace virtual { inbox = yes prefix = virtual/ separator = / location = virtual:/etc/dovecot/virtual:INDEX=MEMORY list = no hidden = yes } namespace real { prefix = RealMails/ separator = / list = no hidden = yes } } protocol imap { mail_plugins = $mail_plugins imap_quota last_login namespace inbox { inbox = yes } } protocol lda { mail_plugins = " quota sieve" namespace inbox { inbox = yes } } passdb { args = nopassword=y uid=vmail gid=vmail home=/var/mail/%2Mn/%2.2Mn/%n driver = static } userdb { args = nopassword=y uid=vmail gid=vmail home=/var/mail/%2Mn/%2.2Mn/%n driver = static } service quota-status { executable = quota-status -p postfix inet_listener { port = 12340 } process_min_avail = 8 } But the service quota-status getting lost with a message: namespace configuration error: inbox=yes namespace missing I didn't find a way to tell quota-status which namespace to use as you can't setup a default one. If I declare a "global" namespace inbox pop3 deamon fails because of duplicated namespace inbox in the configuration files: pop3(xxx): Error: User initialization failed: namespace configuration error: There can be only one namespace with inbox=yes Any idea before I will patch some piece of code to disable the unique restriction of inbox namespace ? Thanks, -- Auré

1 2

dsync for subscription on public/shared folder
by Mike Fröhner 09 Jan '17

09 Jan '17

Hello, I already wrote regarding that, but maybe it sounded too complex. I will write it a bit softer now. The replication through dsnyc works perfectly with one exception. I am having an issue with the replication of the subscriptions file on shared folder. We have a public mailbox defined: namespace public-test { location = maildir:/opt/mail/_public/test prefix = public/test/ separator = / subscriptions = no type = public } There 2 subfolder .Test1 and .Test2 which I would like to subscribe on my user "ldaptestuser" which is part of the group "ldaptestgroup" The dovecot-acl file on /opt/mail/_public/test and /opt/mail/_public/test/.Test{1,2} are the same and are containing: group=ldaptestgroup lrwstipekxa Now if I login through my client (Thunderbird or Roundcubemail) I can see the following debug log: dovecot: imap(ldaptestuser): Debug: acl: acl username = ldaptestuser dovecot: imap(ldaptestuser): Debug: acl: owner = 0 dovecot: imap(ldaptestuser): Debug: acl: group added: domain users dovecot: imap(ldaptestuser): Debug: acl: group added: ldaptestgroup dovecot: imap(ldaptestuser): Debug: acl: group added: all_mailuser I created one new private folder on the mailbox of my test user and also subscribed it. This subscription is successfully replicated. After that I am trying to subscribe the public folder. I can see the folder in my client after I subscribed them. But on the 2nd dovecot server the subscription file is not updated. imap-1 # cat subscription Private1 public/test/Test1 imap-2 # cat subscription Private1 The debug log on both dovecot server tells me: dovecot: doveadm(<ip>,ldaptestuser): Debug: Effective uid=834603987, gid=991, home=/opt/mail/ldaptestuser dovecot: doveadm(<ip>,ldaptestuser): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled dovecot: doveadm(<ip>,ldaptestuser): Debug: maildir++: root=/opt/mail/_public/test, index=, indexpvt=, control=, inbox=, alt= dovecot: doveadm(<ip>,ldaptestuser): Debug: acl: initializing backend with data: vfile dovecot: doveadm(<ip>,ldaptestuser): Debug: acl: acl username = ldaptestuser dovecot: doveadm(<ip>,ldaptestuser): Debug: acl: owner = 0 dovecot: doveadm(<ip>,ldaptestuser): Debug: acl vfile: Global ACLs disabled It looks like the doveadm() cannot find the group or is not checking/initalizing the groups. Anybody any idea if I missconfigured something or is this just a bug? I really would appreciate some help. kind regards, Mike;

1 1

expunging all mailboxes
by Michael Fox 03 Jan '17

03 Jan '17

Evidently, the 'doveadm expunge' command requires a "mailbox" to be defined in the search string. But is there a way tell it all mailboxes for a user? For example, something like: doveadm expunge -u user@domain mailbox ALL savedbefore 30d Thanks, Michael

4 10