cannot login to imap under load

[Gerard Ranke]

Dear list,

We moved our dovecot installation to a new vm, and ever since there are
problems logging in to our imap server during office hours. ( Evenings
and weekends are fine. ) Both the new and the old machine are dovecot
2.2.13. Symptoms:
Logging in via imap gives:

. OK Pre-login capabilities listed, post-login capabilities have more.
a login <loginname> <passwd>
* OK Waiting for authentication master process to respond..
closed

whereas using the same credentials with pop3:

+OK Dovecot ready.
user <loginname>
+OK
pass <passwd>
+OK Logged in.

Our mail.err log gives lots of:

dovecot: imap-login: Error: master(imap): Auth request timed out
(received 0/12 bytes)
dovecot: imap: Error: Login client disconnected too early
dovecot: auth: Error: Master request 24000.918 not found
dovecot: master: Error: service(imap): fork() failed: Resource
temporarily unavailable
dovecot: master: Error: service(imap): command startup failed,
throttling for 2 secs

Note thate we our users almost exclusively use imap. Normally we would
have some 7 or 800 imap processes running and only a few pop3.

Our doveconf -n output:

# 2.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 4.4.38-93-default x86_64 SUSE Linux Enterprise Server 12
(x86_64)
auth_mechanisms = plain login
default_client_limit = 2000
default_process_limit = 2000
default_vsz_limit = 512 M
disable_plaintext_auth = no
imap_client_workarounds = tb-extra-mailbox-sep
import_environment = TZ DEBUG_OUTOFMEM DOVECOT_HOSTDOMAIN
mail_location = maildir:~/Maildir
mail_plugins = " quota"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date ihave
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
}
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  quota = maildir:User quota
  quota_rule = *:storage=1G
  quota_rule2 = Trash:storage=+10%%
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
}
protocols = imap pop3 lmtp sieve
service anvil {
  client_limit = 8003
}
service auth {
  client_limit = 10000
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
  unix_listener auth-userdb {
    group = dovecot
    mode = 0666
  }
}
service imap-login {
  inet_listener imap {
    port = 143
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
  process_min_avail = 4
  service_count = 0
}
service imap {
  process_limit = 2048
}
service pop3-login {
  inet_listener pop3 {
    port = 110
  }
  inet_listener pop3s {
    port = 995
    ssl = yes
  }
  process_min_avail = 4
  service_count = 0
}
service pop3 {
  process_limit = 2048
}
ssl_cert = </etc/ssl/certs/hkuwildcardthawte.crt
ssl_key = </etc/ssl/certs/hkuwildcardthawte.key
ssl_protocols = !SSLv2 !SSLv3
userdb {
  args = /etc/dovecot/dovecot-ldap-userdb.conf
  driver = ldap
}
protocol lmtp {
  mail_plugins = " quota sieve"
}
protocol lda {
  mail_plugins = " quota sieve"
}
protocol imap {
  mail_max_userip_connections = 10
  mail_plugins = " quota imap_quota"
}


Thank you all for any insight, it will be much appreciated!
Best regards,

gerard