- dovecot - dovecot.org

Broken auth-* sockets.
by ygrishin＠pyramidheadgroup.ca 23 Sep '16

23 Sep '16

Hello. I am migrating my servers to Ubuntu and have been having an issue with Dovecot authenticator service. Exim is set up to use dovecot-auth, anticipating the question whether exim binary was compiled with support of Dovecot authenticator, it was: --- # exim -bV Exim version 4.86_2 #1 built 05-Apr-2016 12:21:41 Copyright (c) University of Cambridge, 1995 - 2015 (c) The Exim Maintainers and contributors in ACKNOWLEDGMENTS file, 2007 - 2015 Berkeley DB: Berkeley DB 5.3.28: (September 9, 2013) Support for: crypteq iconv() IPv6 PAM Perl Expand_dlfunc GnuTLS move_frozen_messages Content_Scanning DKIM Old_Demime DNSSEC PRDR OCSP Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch cdb dbm dbmjz dbmnz dnsdb dsearch ldap ldapdn ldapm mysql nis nis0 passwd pgsql sqlite Authenticators: cram_md5 cyrus_sasl dovecot plaintext spa tls Routers: accept dnslookup ipliteral iplookup manualroute queryprogram redirect Transports: appendfile/maildir/mailstore/mbx autoreply lmtp pipe smtp Fixed never_users: 0 Size of off_t: 8 Configuration file is /etc/exim4/exim4.conf --- The problem is sockets Dovecot creates are somehow broken. Exim reports something like: --- 2016-09-21 14:45:26 dovecot_plain authenticator failed for *** ([***]) [***]: 435 Unable to authenticate at present: authentication socket connection error 2016-09-21 14:45:26 dovecot_login authenticator failed for *** ([***]) [***]: 435 Unable to authenticate at present: authentication socket connection error --- and I initially thought it was wrong permissions for the socket, triple checked and they are 100% right. More than that, I temporarily chmodded it "rw" for the world as follows: --- # ls -l /var/dovecot/auth-* srw-rw-rw- 1 Debian-exim root 0 Aug 17 21:20 /var/dovecot/auth-client srw------- 1 dovecot root 0 Aug 17 21:20 /var/dovecot/auth-login srw-rw-rw- 1 Debian-exim root 0 Aug 17 21:20 /var/dovecot/auth-master -rw------- 1 root root 32 Aug 15 19:35 /var/dovecot/auth-token-secret.dat srw-rw-rw- 1 dovecot root 0 Aug 17 21:20 /var/dovecot/auth-userdb srw------- 1 dovecot root 0 Aug 17 21:20 /var/dovecot/auth-worker --- and Exim reports the same problem. What makes me think that it's Dovecot's and not Exim's problem? The fact that the socket is really broken, on the server where it's not working: --- # socat - UNIX-CONNECT:/var/dovecot/auth-client 2016/09/22 17:58:27 socat[15192] E connect(5, AF=1 "/var/dovecot/auth-client", 26): Connection refused --- on the server where it is working (version 2.2.13 is installed there), precisely same command: --- # socat - UNIX-CONNECT:/var/dovecot/auth-client VERSION 1 1 MECH PLAIN plaintext MECH LOGIN plaintext MECH CRAM-MD5 dictionary active MECH DIGEST-MD5 dictionary active mutual-auth MECH APOP private dictionary active SPID 535 CUID 880 COOKIE 0311e84ed191fb63334819b1fc3bf2e3 DONE --- with a different result!!! The system: --- uname -a Linux *** 4.4.0-38-generic #57-Ubuntu SMP Tue Sep 6 15:41:41 UTC 2016 i686 i686 i686 GNU/Linux --- Doveconf: --- # dovecot -n # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-38-generic i686 Ubuntu 16.04.1 LTS ext3 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login cram-md5 digest-md5 apop debug_log_path = /var/log/dovecot-debug.log default_internal_user = dovenull disable_plaintext_auth = no first_valid_uid = 114 log_path = /var/log/dovecot.log login_greeting = IMAP/POP3 server is ready. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_location = maildir:/var/mail/exim/%d/%n mail_log_prefix = "%Us(%u): " namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocols = imap pop3 service auth { unix_listener auth-client { mode = 0660 user = Debian-exim } unix_listener auth-master { mode = 0600 user = Debian-exim } user = root } service imap-login { chroot = login client_limit = 8 inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 10143 } process_limit = 8 process_min_avail = 5 service_count = 1 user = dovenull vsz_limit = 64 M } service imap { drop_priv_before_exec = yes process_limit = 64 vsz_limit = 2 G } service pop3-login { chroot = login client_limit = 8 inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 10110 } process_limit = 8 process_min_avail = 5 service_count = 1 user = dovenull vsz_limit = 64 M } service pop3 { drop_priv_before_exec = yes process_limit = 64 vsz_limit = 2 G } ssl = no ssl_cert = </etc/dovecot/ssl/server.crt ssl_cipher_list = ALL:!LOW:!SSLv2 ssl_key = </etc/dovecot/ssl/server.key userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol lda { auth_socket_path = /var/dovecot/auth-master info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log postmaster_address = postmaster@*** } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } --- Sounds like a broken Dovecot install, doesn't it? Looks like not the case: --- # apt-get check Reading package lists... Done Building dependency tree Reading state information... Done --- What else to check?

2 2

Logging of custom mail headers
by Francesco Fiore 22 Sep '16

22 Sep '16

Hi, I need to configure Dovecot to log some custom mail headers. I red the documentation of the mail_log plugin but I didn't found which field I've to use as value for mail_log_fields parameter. So, I red the plugin code but in mail-log-plugin.c there're defined only "from" and "subject" fields that match some email header. I found three alternatives: 1. use the mail_filter plugin (spawning a new process, ecc.) 2. coding a new plugin using the notify plugin 3. propose a patch to the mail_log plugin that use a new configuration parameter (es. mail_log_headers) and log the specified mail headers Which is the better solution for you? Thanks in advance Francesco

1 1

Sieve vacation won't send second reply after timeout interval.
by David Williams 21 Sep '16

21 Sep '16

Hi I solved this by updating to Dovecot 2.2.24 David

1 0

dsync-server: Configure lock-file location?
by Udo Waechter 21 Sep '16

21 Sep '16

Good day everyone, I have a situation where my mail-folder structure (mdbox) ist like this: My users have the following data: $ doveadm user XYZ(a)xx.com field value home /srv/mail/ mail mdbox:foo/XYZ@xx.com/ I also do replication between 2 nodes. Now, I sometimes (when a lot of replication is happening I get the following errors: dovecot: dsync-local(XYZ(a)xx.com): Error: Couldn't lock /srv/mail//.dovecot-sync.lock: Timed out after 60 seconds This of course not only for XYZ(a)xx.com It seems to me, that my mailbox structure should rather be: $ doveadm user XYZ(a)xx.com field value home /srv/mail/foo mail mdbox:XYZ@xx.com/ in order to get rid of these messages? Unfortunatly I cannot change that. I've searched around and took a look into the code of dsync-server, unfortunatly the path to the lockfile is not configurable. Is there a way to somehow configure the lock-file location for dsync-server? Thanks very much, udo.

1 0

ldap sasl bind and auth_bind
by Matwey V. Kornilov 20 Sep '16

20 Sep '16

Hello, I am using # dovecot --version 2.2.18 # dovecot -n # 2.2.18: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -n > dovecot-new.conf doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:24: 'imaps' protocol can no longer be specified (use protocols=imap). to disable non-ssl imap, use service imap-login { inet_listener imap { port=0 } } doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -n > dovecot-new.conf doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:24: 'imaps' protocol can no longer be specified (use protocols=imap). to disable non-ssl imap, use service imap-login { inet_listener imap { port=0 } } # OS: Linux 4.1.27-27-default x86_64 openSUSE 42.1 (x86_64) first_valid_uid = 1 mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = lmtp imap service imap-login { inet_listener imap { port = 0 } } ssl = required ssl_ca = /etc/pki/trust/anchors/rootCA.pem ssl_cert = </etc/openldap/openldap.crt ssl_dh_parameters_length = 2048 ssl_key = </etc/openldap/openldap.key ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap override_fields = uid=vmail gid=vmail home=/var/spool/mail/%d/%n } I use LDAP for user and passwd databases as the following: uris = ldapi:/// sasl_bind = yes sasl_mech = EXTERNAL auth_bind = yes And I found that only first authentication after dovecot restart is successful, the others always fail with temp. The reason is that dovecot rebinds to anonymous bind after succesful auth bind instead of rebinding to external sasl bind.

1 1

Sieve vacation won't send second reply after timeout interval.
by David Williams 20 Sep '16

20 Sep '16

Hi there I'm having problems getting sieve to send a second vacation response. For testing I've set sieve_vacation_min_period = 500s and the seconds option in the script is 600. I send the first test message and get a reply fine I then wait about 12 minutes send a second message and get no reply. In the log I get "discarded duplicate response". I've restarted dovecot and removed the .dovecot.lda-dupes file before running the last test. I'm using LMTP for delivery to Dovecot. The sieve script is require ["vacation-seconds"]; # rule:[Vacation] if true { vacation :seconds 600 :addresses "username(a)mydomain.com" :subject "vacation" "test"; } and doveconf -n is # 2.2.19: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: Linux 3.18.12-gentoo x86_64 Gentoo Base System release 2.2 ext4 auth_username_format = %Ln disable_plaintext_auth = no imap_capability = +XLIST mail_location = maildir:/srv/mail/%n/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate vacation-seconds imapflags notify namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox "Sent Items" { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = * driver = pam } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve/default.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_extensions = +notify +imapflags +vacation-seconds sieve_vacation_default_period = 1d sieve_vacation_min_period = 500s sieve_vacation_use_original_recipient = yes } postmaster_address = postmaster(a)mydomain.com protocols = imap lmtp sieve service auth { unix_listener auth-userdb { group = mode = 0777 user = } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 10024 } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/ssl/dovecot/server.pem ssl_key = </etc/ssl/dovecot/server.key submission_host = localhost userdb { driver = passwd override_fields = home=/srv/mail/%n } protocol lmtp { mail_plugins = sieve } protocol lda { mail_plugins = sieve } I've changed the identifying info to stop any spammers. Any ideas, suggestions or just another pair of eyes telling me I've done something stupid would be much appreciated. Many thanks David

1 0

[Dovecot] Plugins: virtuals vs acls
by Leho Kraav 20 Sep '16

20 Sep '16

Hi all dovecot-virtual: * all when dovecot-acl files restrict some subset of * for a user, does dovecot respect these ACLs when collecting messages for virtual folder? I'd like a confirmation about what I should be seeing when I start implementing this setup.

2 3

Dovecot and imapproxy
by Grant 20 Sep '16

20 Sep '16

imapproxy sped things up a lot for me when I was using courier. I switched to dovecot a while back and kept using imapproxy. Lately I've been noticing weird TCP Queuing spikes in my munin graphs and discontinuing my use of imapproxy seems to have smoothed them out completely and I haven't noticed a slowdown in webmail. Is imapproxy generally not necessary with dovecot? - Grant

2 1

Re: Dovecot 2.2.25 fails on SSL
by Joseph Tam 19 Sep '16

19 Sep '16

"Andreas M. Kirchwitz" <amk(a)spamfence.net> writes: >> OK, the origin of your problem becomes clearer. You can hardcode these >> paths into the executables by doing something like >> >> env CFLAGS='-I/my'ssl/include' \ >> LDFLAGS='-L/your/ssl/lib -Wl,-rpath,/my/ssl/lib' \ >> configure ... > > Based on your mail I've tried CFLAGS/LDFLAGS again, and > now Dovecot didn't even compile any longer. I don't use the same OS as you, but what errors dis you get? Joseph Tam <jtam.home(a)gmail.com>

1 0

Panic: file auth-request.c
by Chris Wik 19 Sep '16

19 Sep '16

Hello Dovecot list, We've been running a really old CentOS 5 server with the stock dovecot from yum (1.0.7) for years and years with absolutely no problems. If it ain't broke, don't fix it, or something like that. Well it finally broke, but only due to the server no longer being able to handle the load of the increasing user base (many thousands now, with hundreds of concurrent IMAP connections any any given time) So we upgraded to a new CentOS 7 server with SSD RAID, fast CPUs and tons of RAM. No more load problems. We compiled the latest dovecot from source (as the version from CentOS yum repo is already quite old, figure we might as well run the latest version since we were upgrading anyway). Thanks to excellent documentation on dovecot.org and fairly thorough testing, the upgrade was quite smooth. We kept all the message UUIDs intact and tried to match the supported authentication methods etc to the old setup, and we didn't have any problems with clients re-downloading or re-syncing mail. We do however have one problem, which prompted me to join this list. It is the same problem as described in this thread from last month: http://dovecot.org/list/dovecot/2016-July/104899.html Here's the excerpt from our maillog: Sep 17 19:34:57 mail dovecot: auth: Panic: file auth-request.c: line 1049 (auth_request_lookup_credentials): assertion failed: (request->credentials_scheme == scheme) Sep 17 19:34:57 mail dovecot: auth: Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x89470) [0x7fa9cb8af470] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x8954e) [0x7fa9cb8af54e] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fa9cb851f75] -> dovecot/auth() [0x4165bc] -> dovecot/auth() [0x4221fb] -> dovecot/auth() [0x41620b] -> dovecot/auth(auth_request_lookup_credentials_callback+0x58) [0x4162f8] -> dovecot/auth(passdb_handle_credentials+0x6a) [0x4254ba] -> dovecot/auth() [0x425b62] -> dovecot/auth() [0x41c1f8] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fa9cb8c207c] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) [0x7fa9cb8c3377] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fa9cb8c2105] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fa9cb8c22b8] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fa9cb857f33] -> dovecot/auth(main+0x2eb) [0x40ccdb] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fa9c9dc2b15] -> dovecot/auth() [0x40cf15] On the client side, it manifests as an authentication failure, and the user is prompted to re-enter their password. The reports we've had are all from users who have their passwords saved in a local password manager, so the client is definitely sending the correct password. It's not related to a particular mail client. It is also definitely not related to the particular user or password, as the user will authenticate many times successfully and then (seemingly) randomly be hit by this bug and prompted to re-enter their password. We enabled verbose logging and even logging of passwords for failed authentication attempts in an attempt to find a pattern, but so far we have not found one. Similar to the original poster in the above thread, we are using a MySQL backend, and CentOS 7 on x86_64 It happens quite frequently: in the past 6 days it happened 49175 times, according to a grep of the maillog. In the same period there were 294167 successful IMAP logins and 160322 POP3 logins. For now we have downgraded to 2.2.4 and so far have not seen the crash recorded in the maillog. What can we do to help track down and fix this bug? # dovecot -n # 2.2.24 (a82c823): /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.10.0-327.28.3.el7.x86_64 x86_64 CentOS Linux release 7.2.1511 (Core) ext4 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login digest-md5 cram-md5 ntlm rpa apop auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot-debug.log default_client_limit = 5000 default_process_limit = 1000 disable_plaintext_auth = no first_valid_uid = 89 mail_debug = yes mail_fsync = never mail_gid = 89 mail_location = maildir:/var/virtual/%d/%n mail_uid = 89 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } ssl_ca = <*** ssl_cert = <*** ssl_key = <*** userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } verbose_ssl = yes protocol lmtp { mail_fsync = optimized } protocol lda { mail_fsync = optimized } protocol imap { mail_max_userip_connections = 50 } protocol pop3 { mail_max_userip_connections = 20 } -- Chris Wik Anu Internet Services www.anu.net | www.cwik.ch

3 5