Number of imap-login processes always keeps growing, never goes down

Jasper Siepkes jasper at siepkes.nl
Sun Jan 26 15:32:34 EET 2020


Hi all!

I've bumped into an issue which Dovecot which has me a bit stumped; All of a
sudden (ie. no obvious changes made to the config as far as I know) Dovecot
seems to never stop 'imap-login' processes. This continues to the point where
it will hit the 'process_limit'. For a 50-ish user install the limit was set to
100 but I increased it to 512. The only thing this changes it that Dovecot
takes a little longer to hit the limit.

I started having this problem with Dovecot version 2.3.2.1. After which I
updated to the latest version of Dovecot (2.3.9.2) to see if that would fix my
problem. However I'm still experiencing the same issue.

For a bit of context; We use Dovecot with Open-Xchange where users login via
OIDC, get a token and then use that token with Dovecot with the 'oauthbearer'
auth method. However users can also login via a username / password combo stored
in LDAP for clients that don't support 'oauthbearer' (ie. about all the mail
clients ;-). We run Dovecot on SmartOS (ie. Illumos, a Solaris derivative).

Oddly enough 'doveadm' is under the impression there are only 3 users loggedin
(when there are 512 imap-login processes). I think 'doveadm who' doesn't show
users who are authenticated via an token (oauthbearer) because I only see 3
users and I don't see my own user which is logged in via Open-Xchange by using
oauthbearer as auth method. I've anonimized the output but you get the idea:

----8<--------------------
# doveadm who
username              # proto (pids)        (ips)
foo1 at foo.nl 2 imap  (78393 78391) (52.XXX.XXX.XXX)
foo2 at foo.nl 2 imap  (72548 72547) (52.XXX.XXX.XXX)
foo3 at foo.nl 2 imap  (480893 481231) (54.XXX.XXX.XXX 54.XXX.XXX.XXX)
----8<--------------------

I don't know if the above issue is related to my problem but the oauthbearer
sessions not showing up seems like a bug? Meanwhile there are 512 Dovecot
imap-login processes:

----8<--------------------
# ptree | grep imap-login | wc -l
513
----8<--------------------

I realise that the 'imap-login' process also works as a IMAP proxy and that
it stays alive during the entire session of the client since it handles the TLS
part of the connection. However there are nowhere near 512 connections active. As
far as I know IMAP doesn't support multiplexing (ie. multiple IMAP sessions in a
single TCP connection) so with 512 imap-login proccess I would expect to see
a lot more connections then I'm seeing:

----8<--------------------
# netstat -a
TCP: IPv4
   Local Address        Remote Address    Swind  Send-Q Rwind  Recv-Q    State
-------------------- -------------------- ------ ------ ------ ------ -----------
      *.ssh                *.*                 0      0 1048576      0 LISTEN
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 52.XXX.XXX.XXX.53136  178176      0 1049580      0 ESTABLISHED
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 52.XXX.XXX.XXX.53188  126848      0 1049580      0 ESTABLISHED
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 83.XXX.XXX.XXX.64845  132352      0 1048960      0 ESTABLISHED
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 10.100.2.2.7937       16384      0 1049800      0 CLOSE_WAIT
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 10.100.2.3.4983       16384      0 1049800      0 CLOSE_WAIT
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 10.100.2.2.34369      16384      0 1049800      0 CLOSE_WAIT
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 10.100.2.3.15041      16384      0 1049800      0 CLOSE_WAIT
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 10.100.2.2.49044      16384      0 1049800      0 CLOSE_WAIT
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 10.100.2.3.6340       16384      0 1049800      0 CLOSE_WAIT
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 10.100.2.2.11331      16384      0 1049800      0 CLOSE_WAIT
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 52.XXX.XXX.XXX.49920  94976      0 1049580      0 ESTABLISHED
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.58569 10.100.3.84.ldap     1049792      0 1049800      0 ESTABLISHED
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.33749 10.100.3.84.ldap     1049792      0 1049800      0 ESTABLISHED
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps 52.XXX.XXX.XXX.49966  46464      0 1049580      0 ESTABLISHED
      *.4190               *.*                 0      0 1048576      0 LISTEN
      *.24                 *.*                 0      0 1048576      0 LISTEN
      *.imap2              *.*                 0      0 1048576      0 LISTEN
      *.imaps              *.*                 0      0 1048576      0 LISTEN
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps ec2-54-210-254-232.compute-1.amazonaws.com.32892 107904      0 1049800      0 ESTABLISHED
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.ssh 10.101.2.14.59256     64128     35 1049880      0 ESTABLISHED
dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl.imaps ec2-54-167-34-137.compute-1.amazonaws.com.3890 549632      0 1049800      0 ESTABLISHED

TCP: IPv6
   Local Address                     Remote Address                 Swind  Send-Q Rwind  Recv-Q    State      If
--------------------------------- --------------------------------- ------ ------ ------ ------ ----------- -----
      *.ssh                             *.*                              0      0 1048576      0 LISTEN

Active UNIX domain sockets
Address          Type       Vnode            Conn             Local Address                           Remote Address
----8<--------------------

This is the running Dovecot config:

----8<--------------------
# doveconf -n -c /etc/dovecot/dovecot.conf
# 2.3.9.2 (cf2918cac): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.9 (db4e9a2f)
# OS: SunOS 5.11 i86pc
# Hostname: dovecot-1.inst.sp-prod.nl1.cns.supersecretcorp.nl
auth_failure_delay = 3 secs
auth_mechanisms = plain oauthbearer xoauth2
base_dir = /var/run/dovecot/
disable_plaintext_auth = no
first_valid_uid = 1000
hostname = imap.supersecretcorp.nl
instance_name = dovecot-1
listen = *
lmtp_rcpt_check_quota = yes
log_path = /dev/stderr
mail_attachment_dir = /var/lib/dovecot/attachments
mail_attachment_fs = sis-queue /var/lib/dovecot/attachments/queue:posix
mail_attachment_hash = %{sha256}
mail_attribute_dict = file:~/mdbox/dovecot-attributes
mail_gid = vmail
mail_home = /var/lib/dovecot/vmail/%d/%n
mail_location = mdbox:~/mdbox
mail_plugins = " quota notify"
mail_server_admin = mailto:it at ask.supersecretcorp.nl
mail_temp_dir = /var/lib/dovecot/tmp
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext
metric imap_command_fetch_ok {
  event_name = imap_command_finished
  filter {
    name = FETCH
    tagged_reply_stat = OK
  }
}
metric imap_command_list {
  event_name = imap_command_finished
  filter {
    name = LIST
    tagged_reply_state = OK
  }
}
metric imap_command_search {
  event_name = imap_command_finished
  filter {
    name = SEARCH
    tagged_reply_stat = OK
  }
}
metric imap_command_select {
  event_name = imap_command_finished
  filter {
    name = SELECT
    tagged_reply_state = OK
  }
}
metric imap_select_no {
  event_name = imap_command_finished
  filter {
    name = SELECT
    tagged_reply_state = NO
  }
}
metric imap_select_no_notfound {
  event_name = imap_command_finished
  filter {
    name = SELECT
    tagged_reply = NO*Mailbox doesn't exist:*
  }
}
metric storage_http_gets {
  categories = storage
  event_name = http_request_finished
  filter {
    method = get
  }
}
namespace inbox {
  hidden = no
  inbox = yes
  list = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox "Sent objects" {
    special_use = \Sent
  }
  mailbox Spam {
    special_use = \Junk
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
  separator = /
  subscriptions = yes
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-oauth2.conf.ext
  driver = oauth2
  mechanisms = oauthbearer xoauth2
}
passdb {
  args = /etc/dovecot/dovecot-ldap-passdb.conf.ext
  driver = ldap
}
plugin {
  push_notification_driver = ox:url=http://#hidden_use-P_to_show#@open-xchange.svc.sp-prod.nl1.cns.supersecretcorp.nl:8009/preliminary/http-notify/v1/notify user_from_metadata
  quota = count:User quota
  quota_rule2 = Trash:storage=+25M
  quota_vsizes = yes
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_max_actions = 32
  sieve_max_redirects = 4
  sieve_max_script_size = 1M
  sieve_quota_max_scripts = 500
  sieve_quota_max_storage = 10M
  sieve_vacation_send_from_recipient = yes
}
postmaster_address = postmaster at supersecretcorp.nl
protocols = imap lmtp sieve
service auth-worker {
  user = $default_internal_user
}
service auth {
  unix_listener auth-userdb {
    group = vmail
    mode = 0770
    user = dovecot
  }
  user = $default_internal_user
}
service imap-login {
  process_limit = 512
  process_min_avail = 1
  service_count = 1
}
service imap {
  process_limit = 512
}
service lmtp {
  inet_listener lmtp {
    address = *
    port = 24
  }
  user = vmail
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
  process_min_avail = 0
  service_count = 1
}
service managesieve {
  process_limit = 256
}
service stats {
  unix_listener stats-reader {
    group = vmail
    mode = 0660
    user = vmail
  }
  unix_listener stats-writer {
    group = vmail
    mode = 0660
    user = vmail
  }
}
ssl_client_ca_dir = /opt/local/etc/openssl/certs
ssl_dh = # hidden, use -P to show it
ssl_min_protocol = TLSv1.2
submission_host = postfix.svc.sp-prod.nl1.cns.supersecretcorp.nl:25
userdb {
  args = /etc/dovecot/dovecot-ldap-userdb.conf.ext
  driver = ldap
}
protocol imap {
  mail_plugins = " quota notify imap_quota"
  ssl_cert = </etc/dovecot/pki/imap.supersecretcorp.nl.crt
  ssl_key = # hidden, use -P to show it
}
protocol submission {
  ssl_cert = </etc/dovecot/pki/smtp.supersecretcorp.nl.crt
  ssl_key = # hidden, use -P to show it
}
protocol lmtp {
  mail_plugins = quota sieve notify push_notification
  postmaster_address = postmaster at supersecretcorp.nl
}
remote 10.100.2.0/23 {
  protocol imap {
    imap_metadata = yes
  }
}
----8<--------------------

Hoping anyone can offer any insights!

Kind regards,

Jasper



More information about the dovecot mailing list