Are you sure the problem is authentication / pgsql? You could test with looping "doveadm auth lookup $user" rapidly. Of course for different users to avoid them coming from cache. Or if you can reproduce it that way, try if the same happens for repeating the same user so it does come from cache.
i test in condition when 90% of imap connection is already established. auth cache is enabled so i guess tests with the same user are not relevant.
-------- less loaded server ps waux|grep imap-login|wc -l 24977
netstat -n | grep ":993" | grep -ic established 24868
find /srv/vmail/*/ -depth -maxdepth 1 -type d |grep -v "/$"|awk -F '/' '{print $5"@"$4}' > /tmp/userlist
cat /tmp/userlist |wc -l 13285
time $(for i in $(cat /tmp/userlist); do doveadm auth lookup $i; done) passdb: command not found
real 5m43.336s user 1m4.634s sys 2m32.215s
echo "13285 / 343" |bc 38 (per second)
--------- high loaded server ps waux|grep imap-login|wc -l 52264
netstat -n | grep ":993" | grep -ic established 52773
cat /tmp/userlist |wc -l 37727
time $(for i in $(cat /tmp/userlist); do doveadm auth lookup $i; done) passdb:: command not found
real 26m26.193s user 3m55.742s sys 18m21.957s
echo "37727 / 1586" |bc 23 (per second)
Less than the 40\25 query per second with successive requests - it's much slower than I would like to see but not awful. However, with parallel requests, things seem to get a lot worse. Most requests die by timeout. Unfortunately, I can't measure this now on busy servers without risking affecting clients.
And logs look like this
Jan 27 18:51:46 imapserver dovecot: imap-login: Error: Timeout while finishing login (waited 44 secs): user=<aaa@bbb>, method=PLAIN, rip=xx.xx.xx.xx, lip=xx.xx.xx.xx, TLS: Connection closed, session=<3BnDhrQs6MANO0fO> Jan 27 18:51:46 imapserver dovecot: imap-login: Error: Timeout while finishing login (waited 40 secs): user=<bbb@ccc>, method=PLAIN, rip=xx.xx.xx.xx, lip=xx.xx.xx.xx, TLS: Connection closed, session=<0f7IhrQsRuIDEJ9E> Jan 27 18:51:46 imapserver dovecot: imap-login: Error: Timeout while finishing login (waited 41 secs): user=<ccc@ddd>, method=PLAIN, rip=xx.xx.xx.xx, lip=xx.xx.xx.xx, TLS: Connection closed, session=<f8TFhrQshMMDDlat> Jan 27 18:51:46 imapserver dovecot: imap-login: Error: Timeout while finishing login (waited 40 secs): user=<ddd@eee>, method=PLAIN, rip=xx.xx.xx.xx, lip=xx.xx.xx.xx, TLS: Connection closed, session=<lknKhrQspOMDEJ9E> Jan 27 18:51:47 imapserver dovecot: imap-login: Error: Timeout while finishing login (waited 40 secs): user=<eee@fff>, method=PLAIN, rip=xx.xx.xx.xx, lip=xx.xx.xx.xx, TLS: Connection closed, session=<4dLJhrQsXOAS2fSG>
On Mon, Feb 3, 2025 at 8:04 PM Timo Sirainen <timo@sirainen.com> wrote:
On 3. Feb 2025, at 7.05, Anatoliy Zhestov via dovecot <dovecot@dovecot.org> wrote:
Hi. We have a performance problem with imap authentication through postgresql. Our servers(modoboa based) have a big amount of permanent imap connections(5000-50000). Current performance is about 3000 successful authentications per hour. No visible reasons for such low speed. Accordingly, after a network failure
or
server restart, all clients try to reconnect, but restoring the connection pool takes hours and even tens of hours. Judging by the logs after the restart, a huge number of auth requests closed by timeout after 70-90 seconds. The postgresql database is not overloaded at the restore connections process and the postgresql connection pool (100) does not overflow. Manually started sql auth queries work fast, tables have indexes. So I guess there is a bottleneck somewhere in dovecot auth service or postgresql driver.
Are you sure the problem is authentication / pgsql? You could test with looping "doveadm auth lookup $user" rapidly. Of course for different users to avoid them coming from cache. Or if you can reproduce it that way, try if the same happens for repeating the same user so it does come from cache.