Broken auth-* sockets.

[ygrishin at pyramidheadgroup.ca]

Hello.

I am migrating my servers to Ubuntu and have been having an issue with 
Dovecot authenticator service.

Exim is set up to use dovecot-auth, anticipating the question whether 
exim binary was compiled with support of Dovecot authenticator, it was:
---
# exim -bV
Exim version 4.86_2 #1 built 05-Apr-2016 12:21:41
Copyright (c) University of Cambridge, 1995 - 2015
(c) The Exim Maintainers and contributors in ACKNOWLEDGMENTS file, 2007 
- 2015
Berkeley DB: Berkeley DB 5.3.28: (September  9, 2013)
Support for: crypteq iconv() IPv6 PAM Perl Expand_dlfunc GnuTLS 
move_frozen_messages Content_Scanning DKIM Old_Demime DNSSEC PRDR OCSP
Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch cdb dbm 
dbmjz dbmnz dnsdb dsearch ldap ldapdn ldapm mysql nis nis0 passwd pgsql 
sqlite
Authenticators: cram_md5 cyrus_sasl dovecot plaintext spa tls
Routers: accept dnslookup ipliteral iplookup manualroute queryprogram 
redirect
Transports: appendfile/maildir/mailstore/mbx autoreply lmtp pipe smtp
Fixed never_users: 0
Size of off_t: 8
Configuration file is /etc/exim4/exim4.conf
---

The problem is sockets Dovecot creates are somehow broken. Exim reports 
something like:
---
2016-09-21 14:45:26 dovecot_plain authenticator failed for *** ([***]) 
[***]: 435 Unable to authenticate at present: authentication socket 
connection error
2016-09-21 14:45:26 dovecot_login authenticator failed for *** ([***]) 
[***]: 435 Unable to authenticate at present: authentication socket 
connection error
---

and I initially thought it was wrong permissions for the socket, triple 
checked and they are 100% right. More than that, I temporarily chmodded 
it "rw" for the world as follows:

---
# ls -l /var/dovecot/auth-*
srw-rw-rw- 1 Debian-exim root  0 Aug 17 21:20 /var/dovecot/auth-client
srw------- 1 dovecot     root  0 Aug 17 21:20 /var/dovecot/auth-login
srw-rw-rw- 1 Debian-exim root  0 Aug 17 21:20 /var/dovecot/auth-master
-rw------- 1 root        root 32 Aug 15 19:35 
/var/dovecot/auth-token-secret.dat
srw-rw-rw- 1 dovecot     root  0 Aug 17 21:20 /var/dovecot/auth-userdb
srw------- 1 dovecot     root  0 Aug 17 21:20 /var/dovecot/auth-worker
---

and Exim reports the same problem.

What makes me think that it's Dovecot's and not Exim's problem? The fact 
that the socket is really broken, on the server where it's not working:
---
# socat - UNIX-CONNECT:/var/dovecot/auth-client
2016/09/22 17:58:27 socat[15192] E connect(5, AF=1 
"/var/dovecot/auth-client", 26): Connection refused
---

on the server where it is working (version 2.2.13 is installed there), 
precisely same command:
---
# socat - UNIX-CONNECT:/var/dovecot/auth-client
VERSION 1       1
MECH    PLAIN   plaintext
MECH    LOGIN   plaintext
MECH    CRAM-MD5        dictionary      active
MECH    DIGEST-MD5      dictionary      active  mutual-auth
MECH    APOP    private dictionary      active
SPID    535
CUID    880
COOKIE  0311e84ed191fb63334819b1fc3bf2e3
DONE
---
with a different result!!!

The system:
---
  uname -a
Linux *** 4.4.0-38-generic #57-Ubuntu SMP Tue Sep 6 15:41:41 UTC 2016 
i686 i686 i686 GNU/Linux
---

Doveconf:
---
# dovecot -n
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
# OS: Linux 4.4.0-38-generic i686 Ubuntu 16.04.1 LTS ext3
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login cram-md5 digest-md5 apop
debug_log_path = /var/log/dovecot-debug.log
default_internal_user = dovenull
disable_plaintext_auth = no
first_valid_uid = 114
log_path = /var/log/dovecot.log
login_greeting = IMAP/POP3 server is ready.
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c
mail_location = maildir:/var/mail/exim/%d/%n
mail_log_prefix = "%Us(%u): "
namespace inbox {
   inbox = yes
   location =
   mailbox Drafts {
     special_use = \Drafts
   }
   mailbox Junk {
     special_use = \Junk
   }
   mailbox Sent {
     special_use = \Sent
   }
   mailbox "Sent Messages" {
     special_use = \Sent
   }
   mailbox Trash {
     special_use = \Trash
   }
   prefix =
}
passdb {
   driver = pam
}
passdb {
   args = /etc/dovecot/dovecot-sql.conf
   driver = sql
}
protocols = imap pop3
service auth {
   unix_listener auth-client {
     mode = 0660
     user = Debian-exim
   }
   unix_listener auth-master {
     mode = 0600
     user = Debian-exim
   }
   user = root
}
service imap-login {
   chroot = login
   client_limit = 8
   inet_listener imap {
     address = *
     port = 143
   }
   inet_listener imaps {
     address = *
     port = 10143
   }
   process_limit = 8
   process_min_avail = 5
   service_count = 1
   user = dovenull
   vsz_limit = 64 M
}
service imap {
   drop_priv_before_exec = yes
   process_limit = 64
   vsz_limit = 2 G
}
service pop3-login {
   chroot = login
   client_limit = 8
   inet_listener pop3 {
     address = *
     port = 110
   }
   inet_listener pop3s {
     address = *
     port = 10110
   }
   process_limit = 8
   process_min_avail = 5
   service_count = 1
   user = dovenull
   vsz_limit = 64 M
}
service pop3 {
   drop_priv_before_exec = yes
   process_limit = 64
   vsz_limit = 2 G
}
ssl = no
ssl_cert = </etc/dovecot/ssl/server.crt
ssl_cipher_list = ALL:!LOW:!SSLv2
ssl_key = </etc/dovecot/ssl/server.key
userdb {
   driver = passwd
}
userdb {
   args = /etc/dovecot/dovecot-sql.conf
   driver = sql
}
verbose_proctitle = yes
protocol lda {
   auth_socket_path = /var/dovecot/auth-master
   info_log_path = /var/log/dovecot-lda.log
   log_path = /var/log/dovecot-lda-errors.log
   postmaster_address = postmaster@***
}
protocol imap {
   imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
}
protocol pop3 {
   pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
   pop3_uidl_format = %08Xu%08Xv
}
---


Sounds like a broken Dovecot install, doesn't it? Looks like not the 
case:
---
# apt-get check
Reading package lists... Done
Building dependency tree
Reading state information... Done
---

What else to check?