January 2017 - dovecot

[Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header
by Charles Marcus 19 Apr '19

19 Apr '19

On 2012-04-05 4:18 AM, Thomas Leuxner <tlx(a)leuxner.net> wrote: > Also with 2.x you may want to use LMTP rather than the LDA Piping. > > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP I am preparing to convert my main client's postfix_courier-imap setup to dovecot 2.1, which currently just uses the postfix virtual delivery agent... Does anyone know if the use of LMTP (or even the dovecot LDA) still loses the x-original-to header that the postfix vda adds and that I rely heavily on (since I use a lot of aliases), and if it does, is there any solution to get the original recipient added back in before final delivery? Everything I'm reading says that LMTP is better, but I really do need this header (or one like it) to be there so I know who the original recipient was (for filtering and other purposes). Thanks, -- Best regards, Charles

8 16

Status of SMTPUTF8?
by Noah Tilton 14 Sep '18

14 Sep '18

I was wondering whether there is a roadmap for adding SMTPUTF8 support to Dovecot? My delivery pattern is Postfix -> Dovecot LMTP and it is choking on utf8 messages. I might be able to volunteer some of my time as a developer. Another thread about this seemed to go unanswered: http://dovecot.org/list/dovecot/2016-September/105474.html http://unix.stackexchange.com/questions/320091/configure-postfix-and-doveco… -Noah

4 3

Set X-Original-To based an ORCPT?
by Peer Heinlein 09 Aug '18

09 Aug '18

Since http://dovecot.org/pipermail/dovecot-cvs/2014-November/025241.html Dovecot's LMTP does support ORCPT. Is it possible to set X-Original-To-Header based on that ORCPT? Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin

5 5

[Dovecot] quota-status not working in distributed environment
by Benoit Panizzon 30 Jul '18

30 Jul '18

Hello List Quick overview of our set-up: Postfix / Dovecot (2.2.2.1) / MySQL Cluster on (at the moment) three Servers to create a HA environment where you could easily add additional servers as the demand or load grows. Circular dovecot replication is used so each server uses another one as replication partner and allowing one server to fail. Dovecot Proxy Feature being used, so we can use round-robin DNS and each server can forward the connecting user to the correct 'master' for his mailbox. So far, everything works as expected. Now we want to reject emails to 'full' mailboxes during SMTP to prevent backscatter and use the quota-status policy service from within postfix. That works fine, if the mailbox or it's replica is present on the machine where quota-status is called, but it fails if it's run on a machine where neither the mailbox or the replica is present. In our case, we get a correct SMTP 550 'Mailbox Full' Reject in two cases and a LMTP generated bounce in the later case. Also `doveadm quota get -u user(a)example.com` return the correct quota if run on the two machines which have the mailbox and it's copy locale, but return 0% used if run on the other machine. Is there a way to get quota-status to also use the proxy feature to request the quota information from the correct machine? Or is the postfix policy daemon call to the quota-status socket documented somewhere (it must be, but where?) so we could implement it from within the Milter? (we use the sendmail Milter API from postfix to filter spam and viruses, do sender/recipient rewriting, forward bounce matching, rate limmiting, login/IP statistics to block botnets abusing phished addresses and legal intercept stuff anyway) Kind regards Benoit Panizzon -- I m p r o W a r e A G - ______________________________________________________ Zurlindenstrasse 29 Tel +41 61 826 93 07 CH-4133 Pratteln Fax +41 61 826 93 02 Schweiz Web http://www.imp.ch ______________________________________________________

5 6

SIS: dovecot creates duplicated attachments (sometimes)
by Alexander Moisseev 20 Jul '18

20 Jul '18

SIS deduplication was broken in 2.2.16 and has been fixed with https://github.com/dovecot/core/commit/3b39022ea0513363241cf852b7d454c84158… but still sometimes (just several times in a month or so) dovecot creates duplicated attachments. As you can see in directory listings below all attachments was created at (about) the same time, but one of them has different inode number. It easy to discover using https://github.com/moisseev/doveadm-tools/blob/master/bin/dsisck # dsisck -n mail_uid=vmail mail_attachment_dir=/vmail/attachments ==> Checking SIS... # ln -f ./30/b3/30b367c584a123eee59478adf3e4f4c9e1226545-c56eae04a67c3157287f01003d96bafd ./30/b3/30b367c584a123eee59478adf3e4f4c9e1226545-7ad0a411d17c31572b7a01003d96bafd # mkdir -p -m 700 ./64/8f/hashes && ln ./64/8f/648f5cfa27af6d20c8570fdcaeab997663e15105-55430d2cd4432c571cb600003d96bafd ./64/8f/hashes/648f5cfa27af6d20c8570fdcaeab997663e15105 # ln -f ./6f/3f/6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-b1935817ed753557546700003d96bafd ./6f/3f/6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-72da1f32e38e3557947b00003d96bafd # ln -f ./dd/d9/ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-6bf7c122fefb3157ad3500003d96bafd ./dd/d9/ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-90fd1415e7e53157112800003d96bafd -------------------------------------------------------------- Unexpected objects found: 0 Attachments processed: 43500 Different attachments with similar hashes skipped: 0 Attachments deduplicated: 3 File system blocks freed up: 1800 Attachment deduplication attempts failed: 0 Hash files created: 1 Hash files creation attempts failed: 0 Hash files re-linked: 0 Hash files re-link attempts failed: 0 Orphaned hash files: 0 Invalid hash files (not a regular file): 0 -------------------------------------------------------------- # cd /vmail/attachments # ls -li ./30/b3/ ./30/b3/hashes/ ./30/b3/: total 1996 7464083 -rw------- 1 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-7ad0a411d17c31572b7a01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-bb6eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-bd6eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-bf6eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-c36eae04a67c3157287f01003d96bafd 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545-c56eae04a67c3157287f01003d96bafd 7464084 drwx------ 2 vmail vmail 512 May 10 09:16 hashes ./30/b3/hashes/: total 332 7464082 -rw------- 6 vmail vmail 337719 May 10 09:16 30b367c584a123eee59478adf3e4f4c9e1226545 # ls -li ./6f/3f/ ./6f/3f/hashes/ ./6f/3f/: total 644 15088414 -rw------- 1 vmail vmail 326656 May 13 11:23 6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-72da1f32e38e3557947b00003d96bafd 15088422 -rw------- 2 vmail vmail 326656 May 13 11:23 6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe-b1935817ed753557546700003d96bafd 15088448 drwx------ 2 vmail vmail 512 May 13 11:23 hashes ./6f/3f/hashes/: total 320 15088422 -rw------- 2 vmail vmail 326656 May 13 11:23 6f3fa3e4d374a9c80d07af54960ce0e7adb2e0fe # ls -li ./dd/d9/ ./dd/d9/hashes/ ./dd/d9/: total 748 80548 -rw------- 3 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-68f7c122fefb3157ad3500003d96bafd 80548 -rw------- 3 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-6bf7c122fefb3157ad3500003d96bafd 80547 -rw------- 1 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b-90fd1415e7e53157112800003d96bafd 80549 drwx------ 2 vmail vmail 512 May 10 18:19 hashes ./dd/d9/hashes/: total 248 80548 -rw------- 3 vmail vmail 250640 May 10 18:19 ddd97aa6f624d4f54968d2c4956fc3a9d796b31b # doveconf -n # 2.2.24 (a82c823): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.14 (099a97c) # OS: FreeBSD 10.3-RELEASE i386 auth_default_realm = example.com auth_mechanisms = digest-md5 cram-md5 plain apop doveadm_password = # hidden, use -P to show it first_valid_gid = 1000 first_valid_uid = 1000 lda_mailbox_autosubscribe = yes listen = * mail_attachment_dir = /vmail/attachments mail_gid = vmail mail_home = /vmail/%d/%n mail_location = mdbox:~/mdbox mail_plugins = quota zlib acl mail_shared_explicit_inbox = yes mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { list = children location = mdbox:%%h/mdbox:INDEXPVT=~/mdbox/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Archives { special_use = \Archive } mailbox Drafts { special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / } passdb { args = scheme=plain username_format=%n /usr/local/etc/dovecot/dovecot.auth/%d.passwd driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/vmail/shared-mailboxes.db antispam_backend = mailtrain antispam_mail_notspam = report_ham antispam_mail_sendmail = /usr/local/libexec/dovecot/dovecot-lda antispam_mail_sendmail_args = -d;spam(a)example.com;-m antispam_mail_spam = report_spam antispam_spam = Junk antispam_trash = Trash;train_ham;train_prob;train_spam quota = dict:User quota::file:%h/dovecot-quota quota_rule = *:storage=2G quota_rule2 = Trash:storage=+10%% quota_rule3 = Spam:storage=+20%% quota_status_nouser = DUNNO quota_status_success = DUNNO sieve_after = /usr/local/etc/dovecot/sieve/sieve.after sieve_before = /usr/local/etc/dovecot/sieve/sieve.before sieve_vacation_min_period = 0 zlib_save = gz zlib_save_level = 3 } postmaster_address = postmaster(a)example.com protocols = imap lmtp sieve pop3 quota_full_tempfail = yes service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { user = vmail } } service config { unix_listener config { mode = 0600 user = vmail } } service imap-login { inet_listener imaps { port = 0 } process_limit = 200 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } service pop3-login { inet_listener pop3s { port = 0 } } service quota-status { client_limit = 1 executable = quota-status -p postfix unix_listener /var/spool/postfix/private/quota-status { user = postfix } } ssl_cert = </etc/ssl/certs/mx.example.com.crt ssl_key = </etc/ssl/private/mx.example.com.key userdb { args = username_format=%n /usr/local/etc/dovecot/dovecot.auth/%d.passwd driver = passwd-file } verbose_proctitle = yes protocol imap { mail_plugins = quota zlib acl antispam imap_quota imap_zlib imap_acl } protocol lmtp { mail_plugins = quota zlib acl sieve }

1 1

iPhone/iPad IMAP connection bursts causes user+IP exceeded
by Joseph Tam 30 Nov '17

30 Nov '17

I frequently see this from my iPhone/iPad IMAP users: Oct 24 21:30:55 server dovecot: imap-login: Login: user=<user>, ... [... repeated 10 times ...] Oct 24 21:32:54 server dovecot: imap-login: Maximum number of connections from user+IP exceeded (mail_max_userip_connections=12): user=<user> Oct 24 21:32:54 server dovecot: imap(user): Logged out ... [... repeated 11 times ...] These bursts of logins/max/logouts would cycling on for a few minutes. Googling this problem seems to turn up lots of similar complaints about iOS mail mail clients. e.g. https://discussions.apple.com/thread/2547839?tstart=0 iOS mail readers do not limit connections limit as other mailreaders can. I could increase mail_max_userip_connections, but that just moves the goal posts. Using the new rawlog feature in 2.2.26 (thanks Dovecot team!), I was able to see that these connection bursts are caused by clients doing global searches. The rawlogs show each mailbox being SELECT'd and searched (e.g. From header string): 1477369968.730450 2 ID ("name" "iPad Mail" "version" "13G36" "os" "iOS" "os-version" "9.3.5 (13G36)") 1477369968.781932 3 SELECT {mailbox} 1477369968.961636 4 UID SEARCH RETURN (COUNT) 1:* NOT DELETED 1477369969.006087 5 UID SEARCH RETURN (ALL) 1:* NOT DELETED 1477369969.052701 6 UID SEARCH RETURN (ALL) {search-term} NOT DELETED 1477369974.624153 7 LOGOUT Questions: 1) How does this affect the user? I heard from one user that it makes global searches unusable because his reader just spins its wheel. I'm not sure whether this is impatience or this results in failed searches. 2) Is there a client-side fix (e.g. connection limiting)? Apple appears to be intransigent on addressing this. 3) Will maintaining search indices (e.g. solr) help with this? Maybe the searches are taking too long and the connections pile up waiting for previous searches to finish. Thanks, Joseph Tam <jtam.home(a)gmail.com>

3 2

Looking for NTLM config example
by Mark Foley 26 Oct '17

26 Oct '17

Now that I am running Thunderbird on Linux and away from Windows/Outlook, I'd like to take another run at setting up NTLM authentication from Thunderbird to my Samba4 AC/DC. With the help of the samba maillist folks I was able to set up NTLM authentication for domain user login. I should be able to do the same for email! But, I need help. I went to http://wiki2.dovecot.org/Authentication/Mechanisms/NTLM and got lost immediately. Are "authenticaion submethods" synonymous with "password schemes"? The 7th line down says, "NTLM password scheme is required for NTLM, NTLM2 and NTLMv2.", but in the referenced link I found no reference to "NTLM password scheme". The links http://wiki2.dovecot.org/Authentication/Mechanisms/NTLM and http://wiki2.dovecot.org/Authentication/PasswordSchemes, tell you what the 4 NTLM authentication submethods are, tells you what password schemes are, tells you what the NTLM client/server handshake is, but doesn't actually tell you how to configure dovecot config files. I'm much more interested in the "how to" than in: "NTLMv2: server and client nonce, MITM can't force downgrade" ... whatever that means. Anyway, probably it's my lack of understanding terminology. I don't even know what a "nonce" is. But, I learn well from examples! Can somone please give me a sample 10-auth.conf for NTML and any other supporting settings or configs I need? My current/working dovecot settings, which have been running perfectly for well over a year now, are: $ dovecot -n # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.10.17 x86_64 Slackware 14.1 auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no info_log_path = /var/log/dovecot_info mail_location = maildir:~/Maildir passdb { driver = shadow } protocols = imap ssl_cert = </etc/ssl/certs/OHPRS/GoDaddy/Apache/2015-08-14/57aa6ed6ae98b4c7.crt ssl_key = </etc/ssl/certs/OHPRS/GoDaddy/my.server.name.key userdb { driver = passwd } verbose_ssl = yes Here's what I've tried so far as 10-auth.conf: disable_plaintext_auth = no auth_use_winbind = yes info_log_path = /var/log/dovecot_info auth_verbose = yes auth_debug_passwords = yes auth_verbose_passwords= plain auth_winbind_helper_path = /usr/bin/ntlm_auth auth_mechanisms = ntlm plain login userdb { driver = passwd args = username_format=%n allow_all_users=yes } Which gives me a dovecot -n of: $ dovecot -n # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.10.17 x86_64 Slackware 14.1 auth_debug_passwords = yes auth_mechanisms = ntlm plain login auth_use_winbind = yes auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no info_log_path = /var/log/dovecot_info mail_location = maildir:~/Maildir protocols = imap ssl_cert = </etc/ssl/certs/OHPRS/GoDaddy/Apache/2015-08-14/57aa6ed6ae98b4c7.crt ssl_key = </etc/ssl/certs/OHPRS/GoDaddy/my.server.name.key userdb { args = username_format=%n allow_all_users=yes driver = passwd } verbose_ssl = yes I configured Thunderbird for NTLM authentication, then tried sending a message, I got the following in /var/log/dovecot_info: Apr 22 01:37:57 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Apr 22 01:37:57 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Apr 22 01:37:57 auth: Debug: Loading modules from directory: /usr/local/lib/dovecot/auth Apr 22 01:37:57 imap-login: Info: Disconnected: Auth process broken (disconnected before auth was ready, waited 0 secs): user=<>, rip=192.168.0.58, lip=98.102.63.107, session=<xFuyOgwx9wDAqAA6> On Thunderbird I got the error, "Sending of the message failed. The Outlgoing server (SMTP) my.server.name does not support the selected authentication method. Please change the 'Autnentication method' in 'Account Settings | Outgoing Server (SMTP)'." Clearly, something is configured wrong, but I've no clue what. Can I get some advice? THX --Mark

8 19

Setting up replication - First steps...
by SH Development 16 Oct '17

16 Oct '17

Just had my first (non-Dovecot) server failure due to dried out caps on the motherboard. Got me to thinking about my single-point-of-failure mail server. Currently running Dovecot and Postfix with no issues, but want to start taking steps just to be safe. I currently run a filesystem backup every 24 hours to a tar file over NFS to another server in our rack. I am backing up: /home/vmail /etc/dovecot /etc/postfix Unfortunately, the vmail directory has grown to 27GB and takes around 7 hours now to backup as described above. Which leads me to start thinking about how quickly I could restore the server from a backup if need be, and that time is at least 7 hours just to copy and untar the files onto another hard drive. I’m sure I could hook up a HD up directly to the backup server, then I could considerably reduce the time, but I’m making the assumption that I won’t always have quick physical access to the location. So I believe my first step is to set up another server, on another IP, different hostname, with Dovecot and Postfix, and simply use the files from the /etc directories of the existing server to configure it (changing the IP and hostname of course). Am I on the right track so far? Next steps involve setting up replication with dsync? If I have successfully setup replication between the two servers, does this mean users can then actually log into either server and have their “stuff” intact? So I could set up DNS failover in case the primary server fails? Would this make the setup acceptable for secondary MX as everything should sync to the primary server when it comes back online? Sorry for thinking out loud, but I want to make sure I’m understanding the bigger picture here. Jeff

4 3

Quota-status service on Director
by Michael Kliewe 27 Sep '17

27 Sep '17

Hello, I'm trying to configure the quota-status service, but it seems I'm not successful with my director setup (2.2.9). I activate the quota-status service like this on my director server: $ cat 91-quota-status.conf ## ## Quota-Status configuration. ## # Load Module quota-status and listen on TCP/IP Port for connections. service quota-status { executable = quota-status -p postfix inet_listener { address = 10.0.1.44 port = 12340 } client_limit = 1 } # Plugin configuration. # Return messages for requests by quota status: success, nouser and overquota. plugin { quota_status_success = DUNNO quota_status_nouser = DUNNO quota_status_overquota = "552 5.2.2 Mailbox is over quota" } After restarting the director service I try to query the quota status service: printf "recipient=user(a)domain.de\nsize=100000\n\n" | nc 10.0.1.44 12340 The output is: action=DEFER_IF_PERMIT Invalid user settings. Refer to server log for more information. In the debug log of the director I see this: Feb 23 03:03:09 director01 dovecot: auth: Debug: userdb out: USER 1 user(a)domain.de mail=mdbox:/mnt/data01/domain.de/user/maildir home=/mnt/data01/domain.de/user proxy=Y master=<hidden> pass=<hidden> uid=5000 gid=1 quota_rule=*:storage=60593 quota_rule2=*:messages=100000 Feb 23 03:03:09 director01 dovecot: quota-status(user(a)domain.de): Error: user user(a)domain.de: Initialization failed: Namespace '': mkdir(/mnt/data01/domain.de/user/maildir/mailboxes) failed: Permission denied (euid=5000(vmail) egid=1(daemon) missing +w perm: /mnt, dir owned by 0:0 mode=0755) So the quota status service tries to access the mailbox of the user ON THE DIRECTOR. But the director has not mounted the mailboxes of the users, that's what the backend dovecots are for (proxy=Y). So the quota-status query is not proxied to the dovecot backend server I would assume. Does that mean I have to start the quota-status service on the dovecot backend servers and access it from the Postfix server directly? Currently the Postfixes can only reach the directors, not the backend servers. Is it possible to use the quota-status service on the director? Thanks for any hints and help Michael

3 4

[Dovecot] [PATCH, RFC] add APOP authentication mechanism
by Andrey Panin 14 Aug '17

14 Aug '17

Hello all, this patch add APOP authentication mechanism to dovecot 1.0-test23. Please take a look. Best regards. -- Andrey Panin | Linux and UNIX system administrator pazke(a)donpac.ru | PGP key: wwwkeys.pgp.net

3 2