concerning dovecot settings for high volume server

[Tony Morehen]

On 9/12/2015 10:51 PM, Rajesh M wrote:
> ----- Original Message -----
> From: Tony Morehen [mailto:tmorehen at ajmconsulting.ca]
> To: dovecot at dovecot.org
> Sent: Sat, 12 Sep 2015 17:57:27 -0400
> Subject: Re: concerning dovecot settings for high volume server
>
> You may be running up against Linux system/user limits.  Run
> $ cat /proc/sys/kernel/pid_max
> and
> $ ulimit -a
> That should give some insight into your problem.
>
> On 12/09/2015 2:53 PM, Rajesh M wrote:
>> hi
>>
>> centos 6 64 bit
>>
>> hex core processor with hyperthreading ie display shows 12 cores
>> 16 gb ram
>> 600 gb 15000 rpm drive
>>
>> we are having around 4000 users on a server
>>
>>
>> i wish to allow 1500 pop3 and 1500 imap connections simultaneously.
>>
>> need help regarding the settings to handle the above
>>
>> imap-login, pop3-login
>> imap pop3 service settings
>>
>> i recently i got an error
>> imap-login: Error: read(imap) failed: Remote closed connection (process_limit reached?)
>>
>>
>> my current dovecot config file
>>
>> # 2.2.7: /etc/dovecot/dovecot.conf
>> # OS: Linux 2.6.32-431.23.3.el6.x86_64 x86_64 CentOS release 6.5 (Final)
>> auth_cache_negative_ttl = 0
>> auth_cache_ttl = 0
>> auth_mechanisms = plain login digest-md5 cram-md5
>> default_login_user = vpopmail
>> disable_plaintext_auth = no
>> first_valid_gid = 89
>> first_valid_uid = 89
>> log_path = /var/log/dovecot.log
>> login_greeting = ready.
>> mail_max_userip_connections = 50
>> mail_plugins = " quota"
>> managesieve_notify_capability = mailto
>> managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave
>> namespace {
>>     inbox = yes
>>     location =
>>     prefix =
>>     separator = .
>>     type = private
>> }
>> passdb {
>>     args = cache_key=%u webmail=127.0.0.1
>>     driver = vpopmail
>> }
>> plugin {
>>     quota = maildir:ignore=Trash
>>     quota_rule = ?:storage=0
>> }
>> protocols = imap pop3
>> service imap-login {
>>     client_limit = 256
>>     process_limit = 400
>>     process_min_avail = 4
>>     service_count = 0
>>     vsz_limit = 512 M
>> }
>> service pop3-login {
>>     client_limit = 1000
>>     process_limit = 400
>>     process_min_avail = 12
>>     service_count = 0
>>     vsz_limit = 512 M
>> }
>> ssl_cert = </var/qmail/control/servercert.pem
>> ssl_dh_parameters_length = 2048
>> ssl_key = </var/qmail/control/servercert.pem
>> userdb {
>>     args = cache_key=%u quota_template=quota_rule=*:backend=%q
>>     driver = vpopmail
>> }
>> protocol imap {
>>     imap_client_workarounds = delay-newmail
>>     mail_plugins = " quota imap_quota"
>> }
>> protocol pop3 {
>>     pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
>>     pop3_fast_size_lookups = yes
>>     pop3_lock_session = no
>>     pop3_no_flag_updates = yes
>> }
>>
>>
>> thanks very much,
>>
>> rajesh
>
>
> this is the first time i got this message since past over an year.
> the error went away as soon as i restarted dovecot..
> surprisingly this happened in the night ie off-office hours.
>
> here are the values i got
>
> also could you provide me the settings for the limits if wish to handle around 1500 simultaneous connections of pop3 and imap each ?
>
> the current values are as follows
>
> [root at ns1 log]# cat /proc/sys/kernel/pid_max
> 49152
>
> [root at ns1 log]# ulimit -a
> core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 127047
> max locked memory       (kbytes, -l) 64
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 1024
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) 10240
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 127047
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
>
>
> thanks
> rajesh

I'm no expert but it is my understanding that the binding limit would be 
the lower of pid_max or max user processes ie 49152.  From that you 
would subtract 300 (reserved system pids) and the number of non-dovecot 
processes.

Dovecot itself uses about 6 base processes (/usr/sbin/dovecot -F; 
dovecot/anvil; dovecot/log; dovecot/config; dovecot/auth; 
dovecot/ssl-params) plus one processes for each logged-in pop3 and imap 
mailbox.  That's one process for each pop3 account but potentially 
multiple imap mailboxes per imap account if the imap client is using 
idle to monitor multiple imap mailboxes for changes.

Assuming 10 mailboxes per simultaneous imap account, that would be 15000 
processes, well below max_pid.  I'd say that your problem does not 
appear to arise from system limits.  Perhaps someone else could point 
you in a different direction.