October 2017 - dovecot

Set X-Original-To based an ORCPT?
by Peer Heinlein 09 Aug '18

09 Aug '18

Since http://dovecot.org/pipermail/dovecot-cvs/2014-November/025241.html Dovecot's LMTP does support ORCPT. Is it possible to set X-Original-To-Header based on that ORCPT? Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

5 5

[Dovecot] quota-status not working in distributed environment
by Benoit Panizzon 30 Jul '18

30 Jul '18

Hello List Quick overview of our set-up: Postfix / Dovecot (2.2.2.1) / MySQL Cluster on (at the moment) three Servers to create a HA environment where you could easily add additional servers as the demand or load grows. Circular dovecot replication is used so each server uses another one as replication partner and allowing one server to fail. Dovecot Proxy Feature being used, so we can use round-robin DNS and each server can forward the connecting user to the correct 'master' for his mailbox. So far, everything works as expected. Now we want to reject emails to 'full' mailboxes during SMTP to prevent backscatter and use the quota-status policy service from within postfix. That works fine, if the mailbox or it's replica is present on the machine where quota-status is called, but it fails if it's run on a machine where neither the mailbox or the replica is present. In our case, we get a correct SMTP 550 'Mailbox Full' Reject in two cases and a LMTP generated bounce in the later case. Also `doveadm quota get -u user(a)example.com` return the correct quota if run on the two machines which have the mailbox and it's copy locale, but return 0% used if run on the other machine. Is there a way to get quota-status to also use the proxy feature to request the quota information from the correct machine? Or is the postfix policy daemon call to the quota-status socket documented somewhere (it must be, but where?) so we could implement it from within the Milter? (we use the sendmail Milter API from postfix to filter spam and viruses, do sender/recipient rewriting, forward bounce matching, rate limmiting, login/IP statistics to block botnets abusing phished addresses and legal intercept stuff anyway) Kind regards Benoit Panizzon -- I m p r o W a r e A G - ______________________________________________________ Zurlindenstrasse 29 Tel +41 61 826 93 07 CH-4133 Pratteln Fax +41 61 826 93 02 Schweiz Web http://www.imp.ch ______________________________________________________

5 6

SIS: dovecot creates duplicated attachments (sometimes)
by Alexander Moisseev 20 Jul '18

20 Jul '18

SIS deduplication was broken in 2.2.16 and has been fixed with https://github.com/dovecot/core/commit/3b39022ea0513363241cf852b7d454c84158… but still sometimes (just several times in a month or so) dovecot creates duplicated attachments. As you can see in directory listings below all attachments was created at (about) the same time, but one of them has different inode number. It easy to discover using https://github.com/moisseev/doveadm-tools/blob/master/bin/dsisck # dsisck -n mail_uid=vmail mail_attachment_dir=/vmail/attachments ==> Checking SIS... # ln -f ./30/b3/30b367c584a123eee59478adf3e4f4c9e1226545-c56eae04a67c3157287f01003d96bafd ./30/b3/30b367c584a123eee59478adf3e4f4c9e1226545-7ad0a411d17c31572b7a01003d96bafd # mkdir -p -m 700 ./64/8f/hashes && ln ./64/8f/648f5cfa27af6d20c8570fdcaeab997663e15105-55430d2cd4432c571cb600003d96bafd ./64/8f/hashes/648f5cfa27af6d20c8570fdcaeab997663e15105 # ln -f ./6f/3f/6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-b1935817ed753557546700003d96bafd ./6f/3f/6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-72da1f32e38e3557947b00003d96bafd # ln -f ./dd/d9/ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-6bf7c122fefb3157ad3500003d96bafd ./dd/d9/ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-90fd1415e7e53157112800003d96bafd -------------------------------------------------------------- Unexpected objects found: 0 Attachments processed: 43500 Different attachments with similar hashes skipped: 0 Attachments deduplicated: 3 File system blocks freed up: 1800 Attachment deduplication attempts failed: 0 Hash files created: 1 Hash files creation attempts failed: 0 Hash files re-linked: 0 Hash files re-link attempts failed: 0 Orphaned hash files: 0 Invalid hash files (not a regular file): 0 -------------------------------------------------------------- # cd /vmail/attachments # ls -li ./30/b3/ ./30/b3/hashes/ ./30/b3/: total 1996 7464083 -rw------- 1 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-7ad0a411d17c31572b7a01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-bb6eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-bd6eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-bf6eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-c36eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-c56eae04a67c3157287f01003d96bafd 7464084 drwx------ 2 vmail vmail 512 May 10 09:16 hashes ./30/b3/hashes/: total 332 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545 # ls -li ./6f/3f/ ./6f/3f/hashes/ ./6f/3f/: total 644 15088414 -rw------- 1 vmail vmail 326656 May 13 11:23 6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-72da1f32e38e3557947b00003d96bafd 15088422 -rw------- 2 vmail vmail 326656 May 13 11:23 6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-b1935817ed753557546700003d96bafd 15088448 drwx------ 2 vmail vmail 512 May 13 11:23 hashes ./6f/3f/hashes/: total 320 15088422 -rw------- 2 vmail vmail 326656 May 13 11:23 6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe # ls -li ./dd/d9/ ./dd/d9/hashes/ ./dd/d9/: total 748 80548 -rw------- 3 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-68f7c122fefb3157ad3500003d96bafd 80548 -rw------- 3 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-6bf7c122fefb3157ad3500003d96bafd 80547 -rw------- 1 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-90fd1415e7e53157112800003d96bafd 80549 drwx------ 2 vmail vmail 512 May 10 18:19 hashes ./dd/d9/hashes/: total 248 80548 -rw------- 3 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b # doveconf -n # 2.2.24 (a82c823): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.14 (099a97c) # OS: FreeBSD 10.3-RELEASE i386 auth_default_realm = example.com auth_mechanisms = digest-md5 cram-md5 plain apop doveadm_password = # hidden, use -P to show it first_valid_gid = 1000 first_valid_uid = 1000 lda_mailbox_autosubscribe = yes listen = * mail_attachment_dir = /vmail/attachments mail_gid = vmail mail_home = /vmail/%d/%n mail_location = mdbox:~/mdbox mail_plugins = quota zlib acl mail_shared_explicit_inbox = yes mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { list = children location = mdbox:%%h/mdbox:INDEXPVT=~/mdbox/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Archives { special_use = \Archive } mailbox Drafts { special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = scheme=plain username_format=%n /usr/local/etc/dovecot/dovecot.auth/%d.passwd driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/vmail/shared-mailboxes.db antispam_backend = mailtrain antispam_mail_notspam = report_ham antispam_mail_sendmail = /usr/local/libexec/dovecot/dovecot-lda antispam_mail_sendmail_args = -d;spam(a)example.com;-m antispam_mail_spam = report_spam antispam_spam = Junk antispam_trash = Trash;train_ham;train_prob;train_spam quota = dict:User quota::file:%h/dovecot-quota quota_rule = *:storage=2G quota_rule2 = Trash:storage=+10%% quota_rule3 = Spam:storage=+20%% quota_status_nouser = DUNNO quota_status_success = DUNNO sieve_after = /usr/local/etc/dovecot/sieve/sieve.after sieve_before = /usr/local/etc/dovecot/sieve/sieve.before sieve_vacation_min_period = 0 zlib_save = gz zlib_save_level = 3 } postmaster_address = postmaster(a)example.com protocols = imap lmtp sieve pop3 quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { user = vmail } } service config { unix_listener config { mode = 0600 user = vmail } } service imap-login { inet_listener imaps { port = 0 } process_limit = 200 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } service pop3-login { inet_listener pop3s { port = 0 } } service quota-status { client_limit = 1 executable = quota-status -p postfix unix_listener /var/spool/postfix/private/quota-status { user = postfix } } ssl_cert = </etc/ssl/certs/mx.example.com.crt ssl_key = </etc/ssl/private/mx.example.com.key userdb { args = username_format=%n /usr/local/etc/dovecot/dovecot.auth/%d.passwd driver = passwd-file } verbose_proctitle = yes protocol imap { mail_plugins = quota zlib acl antispam imap_quota imap_zlib imap_acl } protocol lmtp { mail_plugins = quota zlib acl sieve }

1 1

offtopic: Solr compatible IMAP client
by Andy Smith 04 Jun '18

04 Jun '18

Hi, not sure if I'm allowed to go this off topic on here, sorry if not! I've just installed Solr FTS for Dovecot and its great, as is Dovecot! ;) It works perfectly for fast searches using Roundcube, but can anyone suggest any Windows and/or Linux mail clients that work well with IMAP searches rather than doing their own searches on local copies of mails? I've had a look at Thunderbird and you can use IMAP search, buy you have to manually select it from an advanced search dialogue and it seems you cannot make this the default behaviour. I also tried OperaMail but it doesn't appear to work at all (only header is searchable without downloading local copies). I'm pretty sure Outlook will only use its own search engine on its local copy of data. thanks a lot, Andy. PS this guide was really helpful for setting up Solr FTS with a recent release of Solr (I'm using 6.5), might be worth including a link somewhere on the Dovecot wiki?: http://mor-pah.net/2016/08/15/dovecot-2-2-with-solr-6-or-5

4 7

Strange "IMAP connection broken (server response)" errors
by Jozsef Kadlecsik 20 Mar '18

20 Mar '18

Hello, We upgraded one of our dovecot servers to debian stretch with dovecot 2.2.27 and since then one of our users has been experiencing random IMAP failures. We enabled raw logging at the server side and it shows normal IMAP commands/responses: 1507292522.222427 * 6 FETCH (FLAGS () BODYSTRUCTURE ("text" "plain" ("charset" "us-ascii") NIL NIL "7bit" 4645 112 NIL NIL NIL NIL)) 1507292522.222653 00000011 OK Fetch completed (0.006 + 0.000 secs). On the client side the user runs alpine and the corresponding debug lines: IMAP DEBUG 14:22:02.216167: 00000011 FETCH 6 (BODYSTRUCTURE FLAGS) 14:22:02.217396 IMAP 14:22:02 10/6 mm_notify bye: {[127.0.0.1]:1555/imap/user="ha4aa"}INBOX: [CLOSED] IMAP connection broken (server response) 14:22:02.217471 IMAP 14:22:02 10/6 mm_log error: [CLOSED] IMAP connection broken (server response) The "[127.0.0.1]:1555/imap/user="ha4aa" part in the log comes from an socat inserted between the client and the server to check independetly the imap session. According to socat, the server response didn't reach the client!: 00000010 OK Fetch completed (0.005 + 0.000 secs).\r > 2017/10/06 14:22:02.216299 length=40 from=845 to=884 00000011 FETCH 6 (BODYSTRUCTURE FLAGS)\r and here ends the socat log, the server response didn't reach the client. The same alpine package is used against older dovecot servers from debian jessie and works flawlessly. The server and client are both KVM guests and there's no any content filtering software between them. There's no corresponding error in the dovecot log file and dovecot didn't regard the connection broken and detected client close: ... Oct 6 14:19:44 mail2 dovecot: imap(ha4aa): Debug: Mailbox INBOX: Opened mail UID=55244 because: MIME part Oct 6 14:22:02 mail2 dovecot: imap(ha4aa): Connection closed in=797 out=73151 We are out of the ideas what can be the reason for the random lost server responses. Any help is appreciated! Best regards, Jozsef -- E-mail : kadlecsik.jozsef(a)wigner.mta.hu PGP key: http://www.kfki.hu/~kadlec/pgp_public_key.txt Address: Wigner Research Centre for Physics, Hungarian Academy of Sciences H-1525 Budapest 114, POB. 49, Hungary

3 3

Non-destructive deduplication
by Tristan Miller 15 Jan '18

15 Jan '18

Greetings. I use Dovecot 2.2.29.1 as my IMAP server. Owing to a bug in my mail client [1], several unique messages (mostly in my Sent folder) have duplicate Message-ID headers. Dovecot itself doesn't seem to be bothered by this, though my mail client is confused by the false duplicates. (It screws up the threading display, and results in data loss when I run the client's deduplication filter.) I would like to change the Message-ID headers in the "duplicate" message files stored in my IMAP server so that they are unique. I realize that this has disadvantages of its own, but I can't think of a better alternative. So this leads me to two questions: 1) Will Dovecot get confused if I simply open the message files in a text editor and manually change their Message-ID headers? If so, how should I go about changing the Message-ID headers? 2) Does there exist any tool that will scan a maildir folder to find messages with duplicate Message-IDs, and automatically rewrite the Message-ID headers so that they are (probably) unique? I could probably kludge together a quick shell script, but if someone's already gone to the trouble of writing such a tool and making it relatively robust, I'd rather use that instead. Regards, Tristan [1] http://www.thewildbeast.co.uk/claws-mail/bugzilla/show_bug.cgi?id=3828 -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Tristan Miller Free Software developer, ferret herder, logologist https://logological.org/ =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

2 3

Managesieve cannot access script store
by dovelist 11 Jan '18

11 Jan '18

Hi, I am trying to get sieve working on a new OpenSuse leap 42.2 install. On my 'old' OpenSuse 13.2 machine it worked fine. The problem is that Managesieve can't access the script store and won't let me create any script. It says permission denied on ~/sieve directory. See log below. I 've activated debug logging, but that doesn't give any clues to me. Also, I've set the directory accessible to all, but Managesieve still complains. > cd ~ > ls -l drwx------ 1 rogier users 8340 5 feb 16:54 Maildir drwxrwxrwx 1 rogier users 24 5 feb 18:38 sieve To rule out client issues (kmail) I tested also with Manual TLS Login as described in: http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Troubleshooting Same result. I am puzzled. I can't find anything wrong in the dovecot configuration. The output of dovecot -n is shown below. Hope someone has a solution. A lot of mail is waiting to get sorted... Best Regards, Rogier The log: feb 05 20:22:18 p150 dovecot[12120]: managesieve-login: Login: user=<rogier>, method=PLAIN, rip=192.168.0.18, lip=192.168.0.20, mpid=12135, TLS, session=<gmb0bs1H5q/AqAAS> feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: Effective uid=1000, gid=100, home=/home/rogier feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: maildir++: root=/home/rogier/Maildir, index=, indexpvt=, control=, inbox=/home/rogier/Maildir, alt= feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: Pigeonhole version 0.4.15 (97b3da0) initializing feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using active Sieve script path: /home/rogier/.dovecot.sieve feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using script storage path: /home/rogier/sieve/ feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Using permissions from /home/rogier/sieve/: mode=0777 gid=-1 feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: Relative path to sieve storage in active link: sieve/ feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Debug: sieve: file storage: sync: Synchronization active feb 05 20:22:18 p150 dovecot[12120]: managesieve(rogier): Error: sieve: file storage: Failed to list scripts: opendir(/home/rogier/sieve) failed: Permission denied Output of dovecot -n: # 2.2.25 (7be1766): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.15 (97b3da0) # OS: Linux 4.4.36-8-default x86_64 openSUSE 42.2 (x86_64) auth_username_format = %Ln base_dir = /var/run/dovecot/ mail_debug = yes mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = file:~/sieve/;active=~/.dovecot.sieve sieve_trace_debug = yes } protocols = imap lmtp sieve service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl = required ssl_cert = </etc/ssl/private/dovecot.crt ssl_dh_parameters_length = 2048 ssl_key = </etc/ssl/private/dovecot.pem ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = passwd } verbose_ssl = yes protocol lmtp { mail_plugins = sieve postmaster_address = postmaster@xxxxxxxxxxxxxx }

5 14

Locks directory change
by Federico Bartolucci 08 Jan '18

08 Jan '18

Hello, it's the first time for me writing to the list, I'm trying to change the location into which the Dovecot's locks are done reserving a special temporary directory on an other partition, then adding to the dovecont.conf the line: mail_location = maildir:~/Maildir:VOLATILEDIR=/tmp_lock/%2.256Nu/%u so that through the VOLATILEDIR directive the locks should be written in this path. We observe though that the locks for many users are still done in the real maildir (NFS mounted filesystem) as if in some situations this instruction is not effective. Anybody knows if are there other things to change or to do or what could be the reason? (for instance to login in a specific way or doing a particular operation). Regards, Federico

3 3

Sieve can't move messages in a public namespace
by Paolo 02 Jan '18

02 Jan '18

Hello, my dovecot version is 2.1.17 I've configured some public namespaces (config attached). When a mail arrives at a certain mail address, that mail is handled by dovecot lmtp server and a sieve script is executed that shuold move the message in one of the namespaces. This is the script: require "fileinto"; if header :contains "From" "example(a)example.com" { fileinto :create "Cond-UFF_FORNITORI/CONTABILITA/Inviata"; } else { fileinto :create "Cond-UFF_FORNITORI/CONTABILITA/Ricevuta"; } The problem is that the sieve script fails with the error: sieve: info: started log at Jun 27 12:42:28. error: msgid=<c9154db4-ff68-52a3-2639-11f09f225f68(a)fcr.re.it>: failed to store into mailbox 'Cond-UFF_FORNITORI/CONTABILITA/Ricevuta': Mailbox doesn't exist: Cond-UFF_FORNITORI/CONTABILITA/Ricevuta. I double checked config, dovecot documentation, sieve documentation, ACL, filesystem paths, filesystem permissions, etc. etc. I'm pretty sure it's all right. So why isn't sieve working? Thanks if you bother to answer & Cheers Paolo

4 6

ManageSieve: authenticate "EXTERNAL" not behaving correctly
by Marc Weustink 30 Nov '17

30 Nov '17

Hi, I've enabled client certificate authentication for imap and managesieve. When I use Thunderbird with the sieve plugin it tries to login, but times out. Initially I reported this to the sieve plugin, but we came to the conclusion that it managesieve is misbehaving. https://github.com/thsmi/sieve/issues/94 Thunderbird (win10-64) 52.4.0 (32bit) Sieve 0.2.3k Dovecot (Ubuntu 16.04.3 LTS) 2.2.33.1-1 (sid) What happens is the following (p=plugin sends m=managesieve sends) p:authenticate "EXTERNAL" "" m:"" The response is unexpected. According to RFC 5804 an empty challenge/response is sent as an empty string. So I would expect: p:authenticate "EXTERNAL" "" m:OK "Logged in." With the use of gnutls-cli I could reproduce (c=I send m=managesieve sends) gnutls-cli --starttls --x509keyfile marc_mail.key --x509certfile marc_mail.crt -p sieve 172.17.1.4 ... m:OK "TLS negotiation successful." c:authenticate "EXTERNAL" "" m:"" c:"" m:OK "Logged in." However if I try the "imap" syntax (rfc4959) I get logged in at once ... m:OK "TLS negotiation successful." c:authenticate "EXTERNAL" "=" m:OK "Logged in." Note that this is an imap only extention, "=" is an invalid base64 encoding. Marc

2 4