System load spike on dovecot reload

[dovecot at mtfbwy.cz]

Hi,

enabling 'login_proxy_max_disconnect_delay' on IMAP proxy did the trick. 
I should have mentioned I use proxy servers, sorry

Thanks,

Dave


Dne 22.4.2017 v 06:25 Christian Balzer napsal(a):
> Hello,
>
> On Fri, 21 Apr 2017 10:43:47 +0200 dave at evilcigi.eu wrote:
>
>> Hi everyone,
>>
>> I'm running dovecot with quite a lot of users and lots of active imap
>> connections (like 20'000). I'm using different user IDs for users, so I
>> need to have imap {service_count=1} - i.e. I have a lots of imap
>> processes running.
>>
> We peaked out at 65k imap processes before upgrading to a version where
> imap-hibernate more or less works, but we're using a common ID.
> ---
> dovecot   119157  0.1  0.0  59364 52216 ?        S    Apr01  48:25 dovecot/imap-hibernate [15137 connections]
> ---
>
> The service_count parameter in this context is not doing what you think it
> does, I have it at 200 these days and that will allow imap (or pop3)
> processes to be recycled (they are labeled with "idling" when waiting for a
> new client), not having one imap process serve multiple clients.
> ---
> mail      591307  0.0  0.0  29876  4712 ?        S    Apr20   0:00 dovecot/imap [idling]
> mail      735323  0.0  0.0  27396  4196 ?        S    13:20   0:00 dovecot/pop3 [idling]
> ---
>
> The advantage (for me at least) is that the dovecot master process doesn't
> have to to spin up a new mail processes each time during logins.
>
> Since this process is quite single-threaded, it becomes a bottleneck
> eventually.
>    
>> Everything works fine, until I reload dovecot configuration. When that
>> happen, every client is forced to relogin in the same time and that
>> causes a huge system load spike (2-3000 5 min load).
>>
> Unless you're making a change that affects the dovecot master process,
> restarting everything isn't needed and you should set
> "shutdown_clients = no".
> You could still kick users with "dovecot kick" at a leisurely pace, but
> security problems with the mail processes are rare.
>
>> I was thinking that it would be great, if dovecot wouldn't kick all the
>> users in the same time during reload, but somehow gradually, during
>> specified interval. I'm aware of the shutdown_clients directive that
>> could help, but I don't like it -
> I've very much gotten to like it, once things got huge and busy.
>
>> I do want the clients get disconnected
>> on dovecot shutdown and also I want them to relogin in reasonably short
>> time after reload.
>>
>> Is something like that possible with dovecot or does it make sense to
>> implement that in the future versions?
>>
> Run a dovecot proxy (if you have single box with all these users on it,
> Mr. Murphy would like a word with you) and set
> "login_proxy_max_disconnect_delay" to something that suits you.
>
> Christian