[Dovecot] Connection queue full on my directors
I finally upgraded to v1.2.16 on my backend servers, with ldap instead of previously mysql, and put a couple of dovecot v2.0.8 directors in front of them (used haproxy previously). It seemed to work OK initially, but after a few hours we got bit by "Connection queue full" problems on the directors.
So I now set default_process_limit=512 and also:
service anvil {
client_limit=1027
}since dovecot said that was now required. Things seems to be working, but I'm afraid default_process_limit=512 is too low:
$ ps -ef|grep dovecot/pop3-login|wc -l
192
$ ps -ef|grep dovecot/imap-login|wc -l
472so looks like I'm quite close to hitting this limit for my imap logins.
We will eventually put up some webmail servers (using imapproxy to keep connections alive), that will be connecting over these directors. Then we will expect to have a lot more active imap sessions over these directors. Should we just keep increasing default_process_limit, or per service process_limit, and follow dovecots warnings about client_limits for auth/anvil.. or will we be getting into other problems like running out of file descriptors? Are there any max'es we should expect to run into ?
Current director config: ################################################################### $ ../../sbin/dovecot -n # 2.0.8: /usr/local/dovecot-2.0.8/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.32.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) default_process_limit = 256 director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 director_servers = 192.168.42.15 192.168.42.17 disable_plaintext_auth = no listen = * passdb { args = proxy=y nopassword=y driver = static } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 5515 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } service pop3-login { executable = pop3-login director } ssl_cert =
$ sudo /usr/local/dovecot/bin/doveadm director status mail server ip vhosts users 192.168.42.7 10 315 192.168.42.8 10 292 192.168.42.9 10 311 192.168.42.10 10 316 192.168.42.11 10 292 192.168.42.28 100 1790 192.168.42.29 100 1787
-jf
On 11.1.2011, at 12.03, Jan-Frode Myklebust wrote:
$ ps -ef|grep dovecot/pop3-login|wc -l 192 $ ps -ef|grep dovecot/imap-login|wc -l 472
so looks like I'm quite close to hitting this limit for my imap logins.
Well, switching to high performance mode would work better for proxies: http://wiki2.dovecot.org/LoginProcess
On Tue, Jan 11, 2011 at 04:31:03PM +0200, Timo Sirainen wrote:
Well, switching to high performance mode would work better for proxies: http://wiki2.dovecot.org/LoginProcess
That text does a good job at scaring me away from high perf mode.. I wouldn't want any attackers stealing my user's passwords. Is it really necessary to run this way, when the server otherwise doesn't seem to be affected by the number of running processes ?
BTW: I'm seeing a bit more failed logins than I think there should be..
pop3-login: Disconnected (auth failed, 1 attempts): user=<user@somewhere.net>, method=PLAIN, rip=xx.xx.xx.xxx, lip=xx.xxx.xx.xxx
imap-login: Aborted login (auth failed, 1 attempts): user=<otheruser@nothere.net>, method=PLAIN, rip=xx.xxx.xxx.xx, lip=xx.xxx.xx.xxx, TLSCould these be explained with the fact that we earlier didn't offer STARTTLS, because users might be talking with one of the CNAMEs that doesn't match the certificate name, and with this new config we started offering STARTTLS ?
I'm a bit reluctant to do it, but if we do get too many complaints about failed logins, will it be it possible to disable STARTTLS, while keeping SSL active for the imaps/pops ports on the director ?
Current config and stats:
##################################################################### # 2.0.8: /usr/local/dovecot-2.0.8/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.32.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) director_mail_servers = 192.168.42.7 192.168.42.8 192.168.42.9 192.168.42.10 192.168.42.11 192.168.42.28 192.168.42.29 director_servers = 192.168.42.15 192.168.42.17 disable_plaintext_auth = no listen = * passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 2051 } service auth { client_limit = 4196 } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 5515 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 1024 } service pop3-login { executable = pop3-login director process_limit = 1024 } ssl_cert =
$ sudo /usr/local/dovecot/bin/doveadm director status mail server ip vhosts users 192.168.42.7 10 546 192.168.42.8 10 531 192.168.42.9 10 487 192.168.42.10 10 520 192.168.42.11 10 493 192.168.42.28 100 5119 192.168.42.29 100 4994
$ ps -ef|grep dovec|wc -l 684
-jf
participants (2)
-
Jan-Frode Myklebust
-
Timo Sirainen