Occasional service disruptions
Hello,
On a server with (Postfix and) Dovecot 2.3.18 (on a VM running CentOS 7
- 1 CPU, 5 GB RAM) with the config you will see below, we are facing occasional (infrequent) service disruptions: IMAP service seems unavailable to some users.
Jun 6 12:01:25 vweb2 roundcube: <1eecb0d4> IMAP Error: Login failed for imaptester against vmail2.noa.gr from 195.251.202.xxx. Could not connect to ssl://vmail2.noa.gr:993: Connection rejected in /var/webs/webmail/rcube/program/lib/Roundcube/rcube_imap.php on line 211 (POST /?_task=login&_action=login)
At that time there was no associated logged event in dovecot log. (Other users are logging in and out.)
However, I see some warnings (I list the two of them closest to the above event):
Jun 06 12:01:22 imap(user1)<29639><Vr0atcPg5M3BXBCl>: Warning: Inotify instance limit for user 500 (UID vmail) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances ... Jun 06 12:01:26 imap(user2)<29793><rZuSt8PgztoKyVSG>: Warning: Inotify instance limit for user 500 (UID vmail) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances
(In above log excerpts I've only modified real usernames.)
Restarting Dovecot returns things back to normal.
I have tried to use "service_count = 100" in all configured services, to see how it goes.
Most of the config is inherited from the past (older versions) and is not optimized. For example one can observe different "process_limit" values for different services, for no apparent reason I am aware of.
Could anyone suggest changes and/or additions to the OS and/or Dovecot to resolve this issue?
Any additional suggestions will also be welcome.
Thanks in advance for your kind assistance.
Here is the config (I've only changed postmaster address):
=======================================================================
protocols = imap pop3 sieve lmtp
login_greeting = Dovecot NOA ICXC-NIKA
log_path = /var/log/dove.log
mail_location = maildir:~/Maildir/
mail_gid = 500 mail_uid = 500
auth_mechanisms = plain login auth_username_format = %Ln
auth_verbose = no auth_debug = no mail_debug = no
disable_plaintext_auth = no
mail_plugins = quota mail_log notify
protocol imap { imap_client_workarounds = "delay-newmail" mail_plugins = quota imap_quota mail_log notify mail_max_userip_connections = 400
namespace inbox { mailbox Trash { autoexpunge = 15d } } }
protocol pop3 { mail_max_userip_connections = 3 mail_plugins = quota notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv
namespace inbox { mailbox Trash { autoexpunge = 15d } } }
protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota notify sieve postmaster_address = xxxxxxxxx@noa.gr sendmail_path = /usr/lib/sendmail }
protocol lmtp { auth_socket_path = /var/run/dovecot/auth-master postmaster_address = xxxxxxxxx@noa.gr mail_plugins = quota notify sieve sendmail_path = /usr/lib/sendmail }
protocol sieve { managesieve_max_line_length = 65536 mail_max_userip_connections = 10 managesieve_logout_format = bytes=%i/%o
managesieve_max_compile_errors = 10 }
userdb { args = /etc/dovecot/dovecot-usrdb-ldap.conf driver = ldap }
passdb { args = /etc/dovecot/dovecot-passdb-ldap.conf driver = ldap }
plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change save mailbox_create mail_log_fields = uid box msgid size flags vsize from subject
quota = maildir:User quota quota_rule = *:storage=15G quota_rule2 = Trash:storage=+3%% quota_warning = storage=75%% quota-warning 75 %u quota_warning2 = storage=90%% quota-warning 90 %u
sieve = file:~/sieve;active=~/.dovecot.sieve sieve_max_script_size = 0 sieve_max_actions = 0 sieve_max_redirects = 2 }
service quota-warning { executable = script /opt/mail1.sh user = vmail unix_listener quota-warning { user = vmail } }
service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = root }
service imap-login { service_count = 100 vsz_limit = 64 M process_limit = 500 }
service pop3-login { service_count = 100 vsz_limit = 64 M }
service managesieve-login { inet_listener sieve { port = 4190 }
service_count = 100 process_min_avail = 0 vsz_limit = 64M }
service managesieve { process_limit = 1024 }
service imap { executable = imap postlogin process_limit = 2048 }
service pop3 { executable = pop3 postlogin }
service postlogin { executable = script-login -d rawlog unix_listener postlogin { } }
service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } }
ssl = yes ssl_cert =
namespace inbox {
separator = . prefix = inbox = yes
mailbox Drafts { special_use = \Drafts auto = subscribe } mailbox Junk { special_use = \Junk auto = subscribe } mailbox Trash { special_use = \Trash auto = subscribe } mailbox Sent { special_use = \Sent auto = subscribe } }
=======================================================================
Nick
On 2022-06-06 11:38 a.m., Nikolaos Milas wrote:
Jun 06 12:01:22 imap(user1)<29639><Vr0atcPg5M3BXBCl>: Warning: Inotify instance limit for user 500 (UID vmail) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances ... Jun 06 12:01:26 imap(user2)<29793><rZuSt8PgztoKyVSG>: Warning: Inotify instance limit for user 500 (UID vmail) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances
Restarting Dovecot returns things back to normal.
This might help: https://doc.dovecot.org/configuration_manual/os/
increasing inotify settings, as the error message suggests. E.g. in /etc/sysctl.conf
On 6/6/2022 7:09 μ.μ., Oscar del Rio wrote:
increasing inotify settings, as the error message suggests. E.g. in /etc/sysctl.conf
Thanks Oscar,
I did that.
Any other suggestions regarding process_limit and service_count values?
Are there any rules of thumb for determining optimal values for these and/or other critical configuration parameters?
Cheers, Nick
On 2022-06-06 12:59 p.m., Nikolaos Milas wrote:
Any other suggestions regarding process_limit and service_count values?
Are there any rules of thumb for determining optimal values for these and/or other critical configuration parameters?
The limits are explained here: https://doc.dovecot.org/configuration_manual/service_configuration/
In my case, I've only had to adjust process_limit for service.imap and service.imap-login, similar to yours.
ok i do NOT use roundcube however trying to help
regarding :
Jun 06 12:01:22 imap(user1)<29639><Vr0atcPg5M3BXBCl>: Warning: Inotify instance limit for user 500 (UID vmail) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances
this to me indicates that all users are logging in under one user name / process at least according to dovecot?
usually when an imap connection is started it starts its own pid per mailbox opened process (at least for me it does)
both dovecot & cyrus work this way
number of simitanulus connections is usually handled by the client (thunderbird, outlook whatever)
roundcube would be considered a client thus the overflow in connections if it is opening everthing under one connection / user
it gets complicated but i would start by checking if different users are actually being logged in
if so then try closing the connection via roundqube and see if the connection drops off on the dovecot server.
# dovecot.who username # proto (pids) (ips)
epower@scom.ca 4 imap (20263 74767 74743 75194) (174.114.171.16)
installers@tomkudla.ca 7 imap (28281 28280 69830 69832 69834 69836 69838) (167.94.196.10)
ditchburn@scom.ca 1 imap (41136) (65.39.148.2)
reception@clancyca.com 1 imap (41133) (65.39.148.2)
ed@scom.ca 4 imap (36344 25879 89306 89308) (204.237.48.37)
rcooke@tnky.ca 6 imap (91131 23791 8700 16087 91176 91179) (172.97.128.227) carol@scom.ca 1 imap (88120) (216.58.34.142)
paul@scom.ca 1 imap (36202) (69.60.225.80)
ditchburn@clancyca.com 1 imap (40942) (65.39.148.2)
the max_user_instances is meant to control how many connections per user thus 500 is way more for multiple clients per user to log in with
i have several customers (like myself) that open connections from multiple locations without issues.
is it possible that roundcube is opening the same user multiple times and not closing the connection after a while?
I get the above logged in list above from
doveadm mailbox status -t all -u $1 '*'
something to consider.
Happy Wednesday !!! Thanks - paul
Paul Kudla
Scom.ca Internet Services http://www.scom.ca 004-1009 Byron Street South Whitby, Ontario - Canada L1N 4S3
Toronto 416.642.7266 Main 1.866.411.7266 Fax 1.888.892.7266 Email paul@scom.ca
On 6/6/2022 11:38 AM, Nikolaos Milas wrote:
Hello,
On a server with (Postfix and) Dovecot 2.3.18 (on a VM running CentOS 7
- 1 CPU, 5 GB RAM) with the config you will see below, we are facing occasional (infrequent) service disruptions: IMAP service seems unavailable to some users.
Jun 6 12:01:25 vweb2 roundcube: <1eecb0d4> IMAP Error: Login failed for imaptester against vmail2.noa.gr from 195.251.202.xxx. Could not connect to ssl://vmail2.noa.gr:993: Connection rejected in /var/webs/webmail/rcube/program/lib/Roundcube/rcube_imap.php on line 211 (POST /?_task=login&_action=login)
At that time there was no associated logged event in dovecot log. (Other users are logging in and out.)
However, I see some warnings (I list the two of them closest to the above event):
Jun 06 12:01:22 imap(user1)<29639><Vr0atcPg5M3BXBCl>: Warning: Inotify instance limit for user 500 (UID vmail) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances ... Jun 06 12:01:26 imap(user2)<29793><rZuSt8PgztoKyVSG>: Warning: Inotify instance limit for user 500 (UID vmail) exceeded, disabling. Increase /proc/sys/fs/inotify/max_user_instances
(In above log excerpts I've only modified real usernames.)
Restarting Dovecot returns things back to normal.
I have tried to use "service_count = 100" in all configured services, to see how it goes.
Most of the config is inherited from the past (older versions) and is not optimized. For example one can observe different "process_limit" values for different services, for no apparent reason I am aware of.
Could anyone suggest changes and/or additions to the OS and/or Dovecot to resolve this issue?
Any additional suggestions will also be welcome.
Thanks in advance for your kind assistance.
Here is the config (I've only changed postmaster address):
=======================================================================
protocols = imap pop3 sieve lmtp
login_greeting = Dovecot NOA ICXC-NIKA
log_path = /var/log/dove.log
mail_location = maildir:~/Maildir/
mail_gid = 500 mail_uid = 500
auth_mechanisms = plain login auth_username_format = %Ln
auth_verbose = no auth_debug = no mail_debug = no
disable_plaintext_auth = no
mail_plugins = quota mail_log notify
protocol imap { imap_client_workarounds = "delay-newmail" mail_plugins = quota imap_quota mail_log notify mail_max_userip_connections = 400
namespace inbox { mailbox Trash { autoexpunge = 15d } } }
protocol pop3 { mail_max_userip_connections = 3 mail_plugins = quota notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv
namespace inbox { mailbox Trash { autoexpunge = 15d } } }
protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota notify sieve postmaster_address = xxxxxxxxx@noa.gr sendmail_path = /usr/lib/sendmail }
protocol lmtp { auth_socket_path = /var/run/dovecot/auth-master postmaster_address = xxxxxxxxx@noa.gr mail_plugins = quota notify sieve sendmail_path = /usr/lib/sendmail }
protocol sieve { managesieve_max_line_length = 65536 mail_max_userip_connections = 10 managesieve_logout_format = bytes=%i/%o
managesieve_max_compile_errors = 10 }
userdb { args = /etc/dovecot/dovecot-usrdb-ldap.conf driver = ldap }
passdb { args = /etc/dovecot/dovecot-passdb-ldap.conf driver = ldap }
plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change save mailbox_create mail_log_fields = uid box msgid size flags vsize from subject
quota = maildir:User quota quota_rule = *:storage=15G quota_rule2 = Trash:storage=+3%% quota_warning = storage=75%% quota-warning 75 %u quota_warning2 = storage=90%% quota-warning 90 %u
sieve = file:~/sieve;active=~/.dovecot.sieve sieve_max_script_size = 0 sieve_max_actions = 0 sieve_max_redirects = 2 }
service quota-warning { executable = script /opt/mail1.sh user = vmail unix_listener quota-warning { user = vmail } }
service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = root }
service imap-login { service_count = 100 vsz_limit = 64 M process_limit = 500 }
service pop3-login { service_count = 100 vsz_limit = 64 M }
service managesieve-login { inet_listener sieve { port = 4190 }
service_count = 100 process_min_avail = 0 vsz_limit = 64M }
service managesieve { process_limit = 1024 }
service imap { executable = imap postlogin process_limit = 2048 }
service pop3 { executable = pop3 postlogin }
service postlogin { executable = script-login -d rawlog unix_listener postlogin { } }
service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } }
ssl = yes ssl_cert =
namespace inbox {
separator = . prefix = inbox = yes
mailbox Drafts { special_use = \Drafts auto = subscribe } mailbox Junk { special_use = \Junk auto = subscribe } mailbox Trash { special_use = \Trash auto = subscribe } mailbox Sent { special_use = \Sent auto = subscribe } }
=======================================================================
Nick
participants (3)
-
Nikolaos Milas
-
Oscar del Rio
-
Paul Kudla (SCOM.CA Internet Services Inc.)