cannot login to imap under load
Dear list,
We moved our dovecot installation to a new vm, and ever since there are problems logging in to our imap server during office hours. ( Evenings and weekends are fine. ) Both the new and the old machine are dovecot 2.2.13. Symptoms: Logging in via imap gives:
. OK Pre-login capabilities listed, post-login capabilities have more. a login <loginname> <passwd>
- OK Waiting for authentication master process to respond.. closed
whereas using the same credentials with pop3:
+OK Dovecot ready. user <loginname> +OK pass <passwd> +OK Logged in.
Our mail.err log gives lots of:
dovecot: imap-login: Error: master(imap): Auth request timed out (received 0/12 bytes) dovecot: imap: Error: Login client disconnected too early dovecot: auth: Error: Master request 24000.918 not found dovecot: master: Error: service(imap): fork() failed: Resource temporarily unavailable dovecot: master: Error: service(imap): command startup failed, throttling for 2 secs
Note thate we our users almost exclusively use imap. Normally we would have some 7 or 800 imap processes running and only a few pop3.
Our doveconf -n output:
# 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 4.4.38-93-default x86_64 SUSE Linux Enterprise Server 12 (x86_64) auth_mechanisms = plain login default_client_limit = 2000 default_process_limit = 2000 default_vsz_limit = 512 M disable_plaintext_auth = no imap_client_workarounds = tb-extra-mailbox-sep import_environment = TZ DEBUG_OUTOFMEM DOVECOT_HOSTDOMAIN mail_location = maildir:~/Maildir mail_plugins = " quota" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = maildir:User quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+10%% sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 lmtp sieve service anvil { client_limit = 8003 } service auth { client_limit = 10000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = dovecot mode = 0666 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 4 service_count = 0 } service imap { process_limit = 2048 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_min_avail = 4 service_count = 0 } service pop3 { process_limit = 2048 } ssl_cert = </etc/ssl/certs/hkuwildcardthawte.crt ssl_key = </etc/ssl/certs/hkuwildcardthawte.key ssl_protocols = !SSLv2 !SSLv3 userdb { args = /etc/dovecot/dovecot-ldap-userdb.conf driver = ldap } protocol lmtp { mail_plugins = " quota sieve" } protocol lda { mail_plugins = " quota sieve" } protocol imap { mail_max_userip_connections = 10 mail_plugins = " quota imap_quota" }
Thank you all for any insight, it will be much appreciated! Best regards,
gerard
Hello Gerard !
On Tuesday, March 28, 2017 4:55 PM, Gerard Ranke <gerard.ranke@hku.nl> wrote:> dovecot: master: Error: service(imap): fork() failed: Resource
temporarily unavailable
dovecot: master: Error: service(imap): command startup failed,
throttling for 2 secs
Note thate we our users almost exclusively use imap. Normally we would
have some 7 or 800 imap processes running and only a few pop3.
Could it be an OS (or VM) limit on the number of processes you can create ?
-- Yassine.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tue, 28 Mar 2017, Gerard Ranke wrote:
dovecot: master: Error: service(imap): fork() failed: Resource temporarily unavailable dovecot: master: Error: service(imap): command startup failed, throttling for 2 secs
check out the ulimits for the Dovecot process.
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1
iQEVAwUBWNuOn3z1H7kL/d9rAQIj1AgAxH8id+JVGJ7YBVKQkSOfb2N160UNRkNo hZ/6HLPfI3pBIzypccvvV+rHtv8pxvURjG1fbAoDBaMlmDWau0gMFJwepBunuEYx gBQGtrBvsABV2nv5kagP5V8TJjzLZplk4/vz0YGsOjlz2JhxbgHcLLA2FyQKTXgc TWGpmcfWUDTQgQeOLVJcfJUBtbdH4MV0JuDCaiVcbtDuWYpWPRWPw+7Gp4gL46X1 orzD9T4+C/80oBtnUV2fERW7ITeRJTgQ3bR1tKYFQmMDJNpQL78G5P06bJB1D8ob 43TO1Ylb/vz4B2+WnM34gKRQcorcNENuuCjLC6Cy1mQ3MK7kjjoZ8Q== =HW21 -----END PGP SIGNATURE-----
Hi Steffen,
On 29-03-17 12:38, Steffen Kaiser wrote:
On Tue, 28 Mar 2017, Gerard Ranke wrote:
dovecot: master: Error: service(imap): fork() failed: Resource temporarily unavailable dovecot: master: Error: service(imap): command startup failed, throttling for 2 secs
check out the ulimits for the Dovecot process.
-- Steffen Kaiser
Here they are:
dovecot@mail:~> ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 256942 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 10000 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 256942 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
This looks ok to me, but on startup, I still get:
dovecot[9309]: Warning: fd limit (ulimit -n) is lower than required under max. load (1024 < 10000), because of service auth { client_limit }
Strange thing is that dovecot still complains about the fd limit being 1024, while I set it to 10000. And how can a ulimit be too low 'because of service auth'? I don't get that at all. Thanks for your interest!
gerard
Could it be that dovecot is being started from a container ? -- Yassine
On Wednesday, March 29, 2017 12:08 PM, Gerard Ranke <gerard.ranke@hku.nl> wrote:Hi Steffen,
On 29-03-17 12:38, Steffen Kaiser wrote:
On Tue, 28 Mar 2017, Gerard Ranke wrote:
dovecot: master: Error: service(imap): fork() failed: Resource temporarily unavailable dovecot: master: Error: service(imap): command startup failed, throttling for 2 secs
check out the ulimits for the Dovecot process.
-- Steffen Kaiser
Here they are:
dovecot@mail:~> ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 256942 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 10000 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 256942 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
This looks ok to me, but on startup, I still get:
dovecot[9309]: Warning: fd limit (ulimit -n) is lower than required under max. load (1024 < 10000), because of service auth { client_limit }
Strange thing is that dovecot still complains about the fd limit being 1024, while I set it to 10000. And how can a ulimit be too low 'because of service auth'? I don't get that at all. Thanks for your interest!
gerard
participants (3)
-
chaouche yacine
-
Gerard Ranke
-
Steffen Kaiser