Hitting wall at 2048 IMAP connections

[Jiri Bourek]

On 02/03/2015 03:35 PM, Ron Cleven wrote:
> We are gradually rolling out Dovecot (IMAP only, no POP3) to our
> customer base.  We are replicating between a pair of CentOS 7 boxes. All
> has been working wonderfully.  However, to be sure our rollout continues
> to go smoothly, we put together a simple benchmark client program to
> fire up X persistent IMAP connections (using hundreds of mailboxes) that
> login, list the folders, select the INBOX, periodically FETCH an email
> from the INBOX, and otherwise sit in an IDLE loop.  As long as we keep
> the number of concurrent IMAP connections under 2k, everything works
> fine.  2,000 processes generate a minimal load on the server.  However,
> 2048 is a "hard limit" for number of IMAP processes on a box the way
> things are configured right now.  The number (2048) is a bit too magical
> to be anything but a misconfiguration of either some kernel limit or a
> dovecot configuration limit.  I have been unable to figure out where
> else to look.  I have pasted in my dovecot config below along with the
> kernel  setting for max_user_instances.
>
> Hoping someone can tell me what stupid mistake I have made or what else
> to check.
>
> Just an aside, we front-end this with a separate set of proxy servers
> that provide an SSL front-end, hence, no SSL in the dovecot config.
>

You didn't mention but did you try checking process_limit and 
client_limit for various Dovecot services in the output of doveconf | 
less ? If I remember correctly, as long as imap process lives, it is a 
client of anvil process. When anvil reaches it's maximum, it's no longer 
possible to run more imap processes and therefore no longer possible to 
connect more clients.

I think I hit some similar problem in the past, but in my case Dovecot 
complained during startup and warned that anvil's client_limit is too 
low compared to imap process_limit and that it could lead to problems. 
So I just increased the limit and was done with it.

That said, this was some time ago so process and setting names can be 
completely wrong. Hope someone more knowledgeable corrects me in that case