[Dovecot] dovecot: imap-login: Error: net_connect_unix(imap) failed: Resource temporarily unavailable

[Kurt Hockenbury]

We recently switched over mail servers to one running dovecot and postfix 
on RedHat Enterprise 6.  The mail store is NFS on a NetApp filer, the 
index files are on local disk.  We have about 6000 total active users, 
though not all of them access the system at the same time.

All goes well for a while, then we start seeing errors like this in the 
log:

dovecot: imap-login: Error: net_connect_unix(imap) failed: Resource temporarily unavailable
imap-login: Internal login failure (auth failed, 1 attempts)

And at this point, users can't log in.  I don't know what Resource dovecot 
is running out of.

The machine is a 12-core Xeon 2.27Ghz, with 24GB RAM.

I have removed the 1024 nproc limit in 
/etc/security/limits.d/90-nproc.conf, and upped the nproc and nofile 
limits to 16384 and 65535 respectively, both in limits.conf, as well as 
adding ulimit statements in /etc/init.d/dovecot (belt and suspenders).

I have tried adjusting dovecot.conf, but I'm obviously missing something.

dovecot -n follows:

# 2.0.beta6 (3156315704ef): /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-71.14.1.el6.x86_64 x86_64 Red Hat Enterprise Linux 
Server release 6.0 (Santiago)
auth_master_user_separator = *
auth_mechanisms = plain login
default_client_limit = 5000
default_process_limit = 5000
disable_plaintext_auth = no
mail_fsync = always
mail_gid = 501
mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n
mail_nfs_storage = yes
mail_plugins = quota
mail_uid = 501
passdb {
   args = /etc/dovecot/passwd.masterusers
   driver = passwd-file
   master = yes
   pass = yes
}
passdb {
   args = /etc/dovecot/dovecot-ldap.conf
   driver = ldap
}
plugin {
   quota = maildir:User quota
   quota_rule = *:storage=200M
}
service auth {
   client_limit = 32768
   unix_listener /var/spool/postfix/private/auth {
     group = postfix
     mode = 0660
     user = postfix
   }
   user = root
}
service imap-login {
   process_min_avail = 12
   service_count = 0
}
service pop3-login {
   service_count = 0
}
shutdown_clients = no
ssl_cert = </etc/pki/postfix/certs/nexus.pem
ssl_key = </etc/pki/postfix/private/nexus_key.pem
userdb {
   args = /etc/dovecot/dovecot-ldap.conf
   driver = ldap
}
protocol imap {
   mail_plugins = $mail_plugins imap_quota
}

Thanks,
-Kurt Hockenbury