Too many wait in auth process

[itanguy@univ-brest.fr]

>>>
>>> service auth-worker {
>>>    client_limit = 1
>>>    idle_kill = 0
>>>    process_limit = 6000000
>>>    process_min_avail = 0
>>>    service_count = 1
>>>    vsz_limit = 18446744073709551615 B
>>> }
>> What dovecot version is this? with 2.3.17 or later you should 
>> probably use service_count=0 here.
>>
>> That would prevent auth-worker process from dying after each 
>> authentication and then need for new process to be spawned for each 
>> authentication.
>
> Yes, it is 2.3.17.
> I give a try, it's slighty better. There is a little fewer stalled 
> auth processes.
> But I didn't manage to go more than 2000 clients although in 
> production it's more than 8000 connections.
> Maybe, it's because I didn't find how to make persistent connections 
> with imaptest and there was too many login/logout. I use delay to make 
> client during around 5 seconds
>
> So I increase this delay up to 120s, this slow down login/logout and 
> decrease processes stuck in wait auth queue.
>
> I think I will go this way to simulate normal load on this server.
> But that doesn't simulate a reboot of service while clients are 
> connected.
>
> Thank you all,
> Ismaël
>

Hello,

I made some little progress in my benchmarks.
I have found how to use imaptest to get IDLE command and make persistent 
connections, using profile.

I have ended yesterday to have 8000 persistents clients on the bench server.
My target is 60000 persistents clients for 250k mailboxes.

The server has 12 procs (24 cores) and 192 Go RAM, fs is zfs.
Increasing clients over 8000 make stalled all connections.
Login slows down drastically, but after login, IMAP commands stay fast.

I'm wondering how to go further.
I believe that I have to tune imap-login service.
I'm seeing 60 Go RAM used in my tests, I suppose that's login process 
and authentication UNIX socket.
Monitoring alerts also about some minor page faults, it could be related.

Conf for now :

service auth-worker {
   client_limit = 1                  # because only the master auth process connects to auth worker
   process_limit = 18000     # should be a bit higher than auth_worker_max_count setting
   service_count = 0             # prevent auth-worker process from dying after each authentication
   process_min_avail = 96     # number of CPU cores * 4
}
service imap-login {
   client_limit = 200
   process_limit = 3000
   process_min_avail = 96
   service_count = 0
   vsz_limit = 1G
}
// using High-performance mode :https://doc.dovecot.org/admin_manual/login_processes/

I'll try today differents settings for this imap-login step, while 
trying to increase number of clients.

If you have any hints to achieve that, I thank you

Ismaël Tanguy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20220218/12546b6d/attachment-0001.htm>