dovecot config for 1500 simultaneous connection

[Christian Balzer]

On Fri, 10 Feb 2017 07:59:52 -0500 KT Walrus wrote:

> > 1500 IMAP sessions will eat up about 3GB alone.  
> 
> Are you saying that Dovecot needs 2MB of physical memory per IMAP session?
> 
That depends on the IMAP session, read the mailbox size and index size,
etc.
Some are significantly larger:
---
    PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                       
1033864 mail      20   0 97600  67m  54m S     0  0.1   0:01.15 imap                                          
---

But yes, as somebody who has mailbox servers with 55k+ session the average
is around 1.6MB. 

> If I want to support a max 100,000 IMAP sessions per server, I should configure the server to have at least 200GBs of SWAP?
>
You will want:
1. 256GB of real RAM, swap is for chums.
2. Understanding how to tune Dovecot and more importantly the overall
system to such a task (see that PID up there?).
3. Be willing to deal with stuff like top and ps taking ages to start/run
and others like atop actually killing dovecot (performance wise, not
literally) when doing their obviously flawed cleanup on exit. Some things
clearly do NOT scale well.

My current goal is to have 100k capable servers that work well, 200k in a
failover scenario, but that won't be particular enjoyable.

Christian

> > On Feb 10, 2017, at 3:58 AM, Christian Balzer <chibi at gol.com> wrote:
> > 
> > On Fri, 10 Feb 2017 01:13:20 +0530 Rajesh M wrote:
> >   
> >> hello
> >> 
> >> could somebody with experience let me know the dovecot config file settings to handle around 1500 simultaneous connections over pop3 and 1500 connection over imap simultaneously.
> >>   
> > 
> > Be very precise here, you expect to see 1500 as the result of 
> > "doveadm who |grep pop3 |wc -l"?
> > 
> > Because that implies an ungodly number of POP3 connects per second, given
> > the typically short duration of these.
> > 
> > 1500 IMAP connections (note that frequently a client will have more than
> > the INBOX open and thus have more than one session and thus process on the
> > server) are a much easier proposition, provided they are of the typical
> > long lasting type.
> > 
> > So can you put a number to your expected logins per second (both protocols)?
> >   
> >> my server
> >> 
> >> server configuration
> >> hex core processor, 16 gb ram 1 X 600 gb 15 k rpm for main drive and 2 X 2000 
> >> gb hdd for data (No raid)
> >>   
> > No RAID and no other replication like DRBD?
> > Why would you even bother?
> > 
> > How many users/mailboxes in total with what quota? 
> > 
> > 1500 IMAP sessions will eat up about 3GB alone.
> > You will want more memory, simply to keep all relevant SLAB bits (inodes,
> > dentries) in RAM. 
> > 
> > If you really have several hundreds logins/s,  you're facing several
> > bottlenecks:
> > 1. Login processes themselves (easily fixed by high performance mode)
> > 2. Auth processes (that will depend on your backends, method mostly)
> > 3. Dovecot master process (spawning mail processes)
> > 
> > The later is a single-threaded process, so it will benefit from a faster
> > CPU core.
> > It can be dramatically improved by enabling process re-usage, see:
> > http://wiki.dovecot.org/PerformanceTuning
> > 
> > However that also means more memory usage.
> > 
> > 
> > 
> > Christian
> >   
> >> 
> >> thanks
> >> rajesh
> >>   
> > 
> > [snip]
> > -- 
> > Christian Balzer        Network/Systems Engineer                
> > chibi at gol.com   	Global OnLine Japan/Rakuten Communications
> > http://www.gol.com/  
> 


-- 
Christian Balzer        Network/Systems Engineer                
chibi at gol.com   	Global OnLine Japan/Rakuten Communications
http://www.gol.com/