- dovecot - dovecot.org

Thunderbird IMAP Problem
by wertstoffe＠nurfuerspam.de 26 Feb '25

26 Feb '25

Hi there! I upgraded to dovecot 2.4 from dovecot repo on my debian bookworm. I modified the config for the needs for 2.4. My Fairemail could connect to dovecot and receive emails via imap. But thunderbird did not work. I activated the debug-logging for thunderbird in shell and got this |2025-02-26 07:46:15.775740 UTC - [Parent 677248: IMAP]: D/IMAP ImapThreadMainLoop entering [this=770954ef3f00] 2025-02-26 07:46:15.776145 UTC - [Parent 677248: IMAP]: D/IMAP ImapThreadMainLoop entering [this=770957d74600] 2025-02-26 07:46:15.776820 UTC - [Parent 677248: Main Thread]: I/IMAP exceeded connection cache limit:imap://my.user@imap.domain.tld:143/folderstatus>/Synoptic/Rechnungen 2025-02-26 07:46:15.776853 UTC - [Parent 677248: Main Thread]: I/IMAP queuing url:imap://my.user@imap.domain.tld:143/folderstatus>/Synoptic/Rechnungen 2025-02-26 07:46:15.776869 UTC - [Parent 677248: Main Thread]: I/IMAP considering playing queued url:imap://my.user@imap.domain.tld:143/folderstatus>/Synoptic/Rechnungen 2025-02-26 07:46:15.776881 UTC - [Parent 677248: Main Thread]: I/IMAP creating protocol instance to play queued url:imap://my.user@imap.domain.tld:143/folderstatus>/Synoptic/Rechnungen 2025-02-26 07:46:15.776897 UTC - [Parent 677248: Main Thread]: I/IMAP exceeded connection cache limit:imap://my.user@imap.domain.tld:143/folderstatus>/Synoptic/Rechnungen 2025-02-26 07:46:15.776909 UTC - [Parent 677248: Main Thread]: I/IMAP failed creating protocol instance to play queued url:imap://my.user@imap.domain.tld:143/folderstatus>/Synoptic/Rechnungen 2025-02-26 07:46:15.777125 UTC - [Parent 677248: Main Thread]: I/IMAP 770954ef3f00:imap.domain.tld:NA:SetupWithUrlCallback: clearing IMAP_CONNECTION_IS_OPEN 2025-02-26 07:46:15.777217 UTC - [Parent 677248: Main Thread]: I/IMAP 770954ef3f00:imap.domain.tld:NA:SetupSinkProxy: got m_imapMailFolderSink 2025-02-26 07:46:15.777302 UTC - [Parent 677248: Main Thread]: I/IMAP 770957d74600:imap.domain.tld:NA:SetupWithUrlCallback: clearing IMAP_CONNECTION_IS_OPEN 2025-02-26 07:46:15.777317 UTC - [Parent 677248: IMAP]: I/IMAP 770954ef3f00:imap.domain.tld:NA:ProcessCurrentURL: entering 2025-02-26 07:46:15.777348 UTC - [Parent 677248: IMAP]: I/IMAP 770954ef3f00:imap.domain.tld:NA:ProcessCurrentURL:imap://my.user@imap.domain.tld:143/discoverallboxes: = currentUrl 2025-02-26 07:46:15.777362 UTC - [Parent 677248: Main Thread]: I/IMAP 770957d74600:imap.domain.tld:NA:SetupSinkProxy: got m_imapMailFolderSink 2025-02-26 07:46:15.777404 UTC - [Parent 677248: IMAP]: I/IMAP 770957d74600:imap.domain.tld:NA:ProcessCurrentURL: entering 2025-02-26 07:46:15.777420 UTC - [Parent 677248: IMAP]: I/IMAP 770957d74600:imap.domain.tld:NA:ProcessCurrentURL:imap://my.user@imap.domain.tld:143/select%3E/INBOX: = currentUrl 2025-02-26 07:46:15.822761 UTC - [Parent 677248: IMAP]: I/IMAP 770954ef3f00:imap.domain.tld:NA:CreateNewLineFromSocket: * OK [CAPABILITY +SPECIAL-USE STARTTLS AUTH=PLAIN] Dovecot ready. 2025-02-26 07:46:15.823122 UTC - [Parent 677248: IMAP]: I/IMAP 770957d74600:imap.domain.tld:NA:CreateNewLineFromSocket: * OK [CAPABILITY +SPECIAL-USE STARTTLS AUTH=PLAIN] Dovecot ready. 2025-02-26 07:46:15.823329 UTC - [Parent 677248: IMAP]: D/IMAP SetConnectionStatus(0x80004005) 2025-02-26 07:46:15.823509 UTC - [Parent 677248: IMAP]: D/IMAP URL failed with code 0x80004005 (imap://my.user@imap.domain.tld:143/discoverallboxes) 2025-02-26 07:46:15.823573 UTC - [Parent 677248: IMAP]: D/IMAP SetConnectionStatus(0x80004005) 2025-02-26 07:46:15.823816 UTC - [Parent 677248: IMAP]: D/IMAP URL failed with code 0x80004005 (imap://my.user@imap.domain.tld:143/select%3E/INBOX) 2025-02-26 07:46:15.824515 UTC - [Parent 677248: IMAP]: I/IMAP 770954ef3f00:imap.domain.tld:NA:ProcessCurrentURL: aborting queued urls 2025-02-26 07:46:15.824529 UTC - [Parent 677248: IMAP]: I/IMAP 770957d74600:imap.domain.tld:NA:ProcessCurrentURL: aborting queued urls 2025-02-26 07:46:15.824723 UTC - [Parent 677248: IMAP]: I/IMAP 770954ef3f00:imap.domain.tld:NA:TellThreadToDie: close socket connection 2025-02-26 07:46:15.824748 UTC - [Parent 677248: IMAP]: D/IMAP ImapThreadMainLoop leaving [this=770954ef3f00] 2025-02-26 07:46:15.824760 UTC - [Parent 677248: IMAP]: I/IMAP 770957d74600:imap.domain.tld:NA:TellThreadToDie: close socket connection 2025-02-26 07:46:15.824785 UTC - [Parent 677248: IMAP]: D/IMAP ImapThreadMainLoop leaving [this=770957d74600] Dovecot-log shows only this two lines on a login-try with thunderbird. ||Feb 26 08:52:16 mail dovecot[204586]: imap-login: Login aborted: Connection closed (no auth attempts in 0 secs) (no_auth_attempts): user=<>, rip=172.30.0.1, lip=172.30.0.9, session=<ryQk2gYvMp+sHgAB> Feb 26 08:52:16 mail dovecot[204586]: imap-login: Login aborted: Connection closed (no auth attempts in 0 secs) (no_auth_attempts): user=<>, rip=172.30.0.1, lip=172.30.0.9, session=<zR8k2gYvIp+sHgAB> Other logins had a user and auth attempts... only from thunderbird it does not work. I also use dovecot-login for my exim4. And smtp-login was also not able. Not even directly via openssl s_client. I could connect, send ehlo, and when i sent AUTH LOGIN <base64-username-passwor-string> and then it hangs. I went back to dovecot 2.3, and thunderbird could login again, also smtp-login works. There seems something wrong in dovecot 2.4. Have ldap-authentication. My bad, i deinstalled dovecot 2.4 and now i can not create a conf with doveconf to add it here. Hope you can help me. Jakob | Hi there! I upgraded to dovecot 2.4 from dovecot repo on my debian bookworm. I modified the config for the needs for 2.4. My Fairemail could connect to dovecot and receive emails via imap. But thunderbird did not work. I activated the debug-logging for thunderbird in shell and got this 2025-02-26 07:46:15.775740 UTC - [Parent 677248: IMAP]: D/IMAP ImapThreadMainLoop entering [this=770954ef3f00] 2025-02-26 07:46:15.776145 UTC - [Parent 677248: IMAP]: D/IMAP ImapThreadMainLoop entering [this=770957d74600] 2025-02-26 07:46:15.776820 UTC - [Parent 677248: Main Thread]: I/IMAP exceeded connection cache limit:imap://my.user@imap.domain.tld:143/folderstatus>/ Synoptic/Rechnungen 2025-02-26 07:46:15.776853 UTC - [Parent 677248: Main Thread]: I/IMAP queuing url:imap://my.user@imap.domain.tld:143/folderstatus>/Synoptic/Rechnungen 2025-02-26 07:46:15.776869 UTC - [Parent 677248: Main Thread]: I/IMAP considering playing queued url:imap://my.user@imap.domain.tld:143/ folderstatus>/Synoptic/Rechnungen 2025-02-26 07:46:15.776881 UTC - [Parent 677248: Main Thread]: I/IMAP creating protocol instance to play queued url:imap://my.user@imap.domain.tld:143/ folderstatus>/Synoptic/Rechnungen 2025-02-26 07:46:15.776897 UTC - [Parent 677248: Main Thread]: I/IMAP exceeded connection cache limit:imap://my.user@imap.domain.tld:143/folderstatus>/ Synoptic/Rechnungen 2025-02-26 07:46:15.776909 UTC - [Parent 677248: Main Thread]: I/IMAP failed creating protocol instance to play queued url:imap://my.user@imap.domain.tld: 143/folderstatus>/Synoptic/Rechnungen 2025-02-26 07:46:15.777125 UTC - [Parent 677248: Main Thread]: I/IMAP 770954ef3f00:imap.domain.tld:NA:SetupWithUrlCallback: clearing IMAP_CONNECTION_IS_OPEN 2025-02-26 07:46:15.777217 UTC - [Parent 677248: Main Thread]: I/IMAP 770954ef3f00:imap.domain.tld:NA:SetupSinkProxy: got m_imapMailFolderSink 2025-02-26 07:46:15.777302 UTC - [Parent 677248: Main Thread]: I/IMAP 770957d74600:imap.domain.tld:NA:SetupWithUrlCallback: clearing IMAP_CONNECTION_IS_OPEN 2025-02-26 07:46:15.777317 UTC - [Parent 677248: IMAP]: I/IMAP 770954ef3f00: imap.domain.tld:NA:ProcessCurrentURL: entering 2025-02-26 07:46:15.777348 UTC - [Parent 677248: IMAP]: I/IMAP 770954ef3f00: imap.domain.tld:NA:ProcessCurrentURL:imap://my.user@imap.domain.tld:143/ discoverallboxes: = currentUrl 2025-02-26 07:46:15.777362 UTC - [Parent 677248: Main Thread]: I/IMAP 770957d74600:imap.domain.tld:NA:SetupSinkProxy: got m_imapMailFolderSink 2025-02-26 07:46:15.777404 UTC - [Parent 677248: IMAP]: I/IMAP 770957d74600: imap.domain.tld:NA:ProcessCurrentURL: entering 2025-02-26 07:46:15.777420 UTC - [Parent 677248: IMAP]: I/IMAP 770957d74600: imap.domain.tld:NA:ProcessCurrentURL:imap://my.user@imap.domain.tld:143/ select%3E/INBOX: = currentUrl 2025-02-26 07:46:15.822761 UTC - [Parent 677248: IMAP]: I/IMAP 770954ef3f00: imap.domain.tld:NA:CreateNewLineFromSocket: * OK [CAPABILITY +SPECIAL-USE STARTTLS AUTH=PLAIN] Dovecot ready. 2025-02-26 07:46:15.823122 UTC - [Parent 677248: IMAP]: I/IMAP 770957d74600: imap.domain.tld:NA:CreateNewLineFromSocket: * OK [CAPABILITY +SPECIAL-USE STARTTLS AUTH=PLAIN] Dovecot ready. 2025-02-26 07:46:15.823329 UTC - [Parent 677248: IMAP]: D/IMAP SetConnectionStatus(0x80004005) 2025-02-26 07:46:15.823509 UTC - [Parent 677248: IMAP]: D/IMAP URL failed with code 0x80004005 (imap://my.user@imap.domain.tld:143/discoverallboxes) 2025-02-26 07:46:15.823573 UTC - [Parent 677248: IMAP]: D/IMAP SetConnectionStatus(0x80004005) 2025-02-26 07:46:15.823816 UTC - [Parent 677248: IMAP]: D/IMAP URL failed with code 0x80004005 (imap://my.user@imap.domain.tld:143/select%3E/INBOX) 2025-02-26 07:46:15.824515 UTC - [Parent 677248: IMAP]: I/IMAP 770954ef3f00: imap.domain.tld:NA:ProcessCurrentURL: aborting queued urls 2025-02-26 07:46:15.824529 UTC - [Parent 677248: IMAP]: I/IMAP 770957d74600: imap.domain.tld:NA:ProcessCurrentURL: aborting queued urls 2025-02-26 07:46:15.824723 UTC - [Parent 677248: IMAP]: I/IMAP 770954ef3f00: imap.domain.tld:NA:TellThreadToDie: close socket connection 2025-02-26 07:46:15.824748 UTC - [Parent 677248: IMAP]: D/IMAP ImapThreadMainLoop leaving [this=770954ef3f00] 2025-02-26 07:46:15.824760 UTC - [Parent 677248: IMAP]: I/IMAP 770957d74600: imap.domain.tld:NA:TellThreadToDie: close socket connection 2025-02-26 07:46:15.824785 UTC - [Parent 677248: IMAP]: D/IMAP ImapThreadMainLoop leaving [this=770957d74600] Dovecot-log shows only this two lines on a login-try with thunderbird. Feb 26 08:52:16 mail dovecot[204586]: imap-login: Login aborted: Connection closed (no auth attempts in 0 secs) (no_auth_attempts): user=<>, rip=172.30.0.1, lip=172.30.0.9, session=<ryQk2gYvMp+sHgAB> Feb 26 08:52:16 mail dovecot[204586]: imap-login: Login aborted: Connection closed (no auth attempts in 0 secs) (no_auth_attempts): user=<>, rip=172.30.0.1, lip=172.30.0.9, session=<zR8k2gYvIp+sHgAB> Other logins had a user and auth attempts... only from thunderbird it does not work. I also use dovecot-login for my exim4. And smtp-login was also not able. Not even directly via openssl s_client. I could connect, send ehlo, and when i sent AUTH LOGIN <base64-username- passwor-string> and then it hangs. I went back to dovecot 2.3, and thunderbird could login again, also smtp-login works. There seems something wrong in dovecot 2.4. Have ldap-authentication. My bad, i deinstalled dovecot 2.4 and now i can not create a conf with doveconf to add it here. Hope you can help me. Jakob

2 1

dovecot Panic
by Jakob 24 Feb '25

24 Feb '25

Hi! I face in short intervals (every few minutes) such error-messages in dovecot Feb 24 01:36:26 mail dovecot[699699]: imap(user.name)<711331><0D3Ch9guy6UufflE>: Panic: file mail-index-view.c: line 165 (view_lookup_full): assertion failed: (seq > 0 && seq <= mail_in dex_view_get_messages_count(view)) Feb 24 01:36:26 mail dovecot[699699]: imap(user.name)<711331><0D3Ch9guy6UufflE>: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42) [0x7fb1e519f662] -> /usr/l ib/dovecot/libdovecot.so.0(backtrace_get+0x1e) [0x7fb1e519f77e] -> /usr/lib/dovecot/libdovecot.so.0(+0x1043db) [0x7fb1e51ac3db] -> /usr/lib/dovecot/libdovecot.so.0(+0x104471) [0x7fb1e51ac471 ] -> /usr/lib/dovecot/libdovecot.so.0(+0x566a0) [0x7fb1e50fe6a0] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x4fe26) [0x7fb1e52cae26] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x102116) [0x7fb1e537d116] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x10514c) [0x7fb1e538014c] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x105ff5) [0x7fb1e5380ff5] -> /usr/lib/dovecot/libdovec ot-storage.so.0(+0x101d14) [0x7fb1e537cd14] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_keyword_indexes+0x6b) [0x7fb1e535027b] -> /usr/lib/dovecot/libdovecot-storage.so.0(inde x_mail_get_keywords+0x6d) [0x7fb1e53502fd] -> /usr/lib/dovecot/modules/lib20_mail_log_plugin.so(+0x2ef9) [0x7fb1e4ba4ef9] -> /usr/lib/dovecot/modules/lib15_notify_plugin.so(notify_contexts_m ail_copy+0x5f) [0x7fb1e4cbd50f] -> /usr/lib/dovecot/modules/lib15_notify_plugin.so(+0x24f8) [0x7fb1e4cbe4f8] -> /usr/lib/dovecot/modules/lib01_acl_plugin.so(+0xf863) [0x7fb1e4eab863] -> /usr /lib/dovecot/libdovecot-storage.so.0(+0x6508f) [0x7fb1e52e008f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_move+0x5e) [0x7fb1e52e024e] -> dovecot/imap [user.name 46.125.249.68 UID MOVE](+0x13ef8) [0x55766c74aef8] -> dovecot/imap [user.name 46.125.249.68 UID MOVE](command_exec+0xa4) [0x55766c7595b4] -> dovecot/imap [user.name 46.125.249.68 UID MOVE](+0x20 52f) [0x55766c75752f] -> dovecot/imap [user.name 46.125.249.68 UID MOVE](+0x205da) [0x55766c7575da] -> dovecot/imap [user.name 46.125.249.68 UID MOVE](+0x208ed) [0x55766c7578ed] -> dovecot/imap [user.name 46.125.249.68 UID MOVE](client_handle_input+0x1b5) [0x55766c757ab5] -> dovecot/imap [user.name 46.125.249.68 UID MOVE](client_input+0x70) [0x55766c758020] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x69) [0x7fb1e51c2609] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x132) [0x7fb1e51c3cf2] -> /usr/lib/dovecot/libdov ecot.so.0(io_loop_handler_run+0x50) [0x7fb1e51c26b0] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7fb1e51c2870] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f b1e51348e3] Feb 24 01:36:26 mail dovecot[699699]: imap(user.name)<711331><0D3Ch9guy6UufflE>: Fatal: master: service(imap): child 711331 killed with signal 6 (core dumped) I run this # dpkg -l|grep dovecot ii dovecot-core 2:2.3.21.1-2+debian11 amd64 secure POP3/IMAP server - core files ii dovecot-imapd 2:2.3.21.1-2+debian11 amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-ldap 2:2.3.21.1-2+debian11 amd64 secure POP3/IMAP server - LDAP support ii dovecot-lmtpd 2:2.3.21.1-2+debian11 amd64 secure POP3/IMAP server - LMTP server ii dovecot-lucene 2:2.3.21.1-2+debian11 amd64 secure POP3/IMAP server - Lucene support ii dovecot-managesieved 2:2.3.21.1-2+debian11 amd64 secure POP3/IMAP server - ManageSieve server ii dovecot-pop3d 2:2.3.21.1-2+debian11 amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 2:2.3.21.1-2+debian11 amd64 secure POP3/IMAP server - Sieve filters support ii dovecot-solr 2:2.3.21.1-2+debian11 amd64 secure POP3/IMAP server - Solr support on debian bookworm. The client, the user uses is maybe apples mail-app on ios... but I'm not sure. And i don't know, what he is doing. Can not interpret this message. Best regards Jakob

1 0

User authentication for shared mailbox
by sveyret＠gmail.com 23 Feb '25

23 Feb '25

Hi, I’m using Dovecot 2.3.21.1. I have mailboxes for the family members and some shared mailboxes, all of them available with IMAP protocol. Users are authenticating using PAM, and shared mailboxes appear as sub-folders of user accounts. Because it is easier to use with some clients, I’m looking for a way to use the shared mailboxes as independent accounts. This would mean that, on a client, I would like to create an account to access to `family@mydomain`, but using my own username and password. I thought of a way to do this by tricking the password, setting it, for example, to `myname+mysecretpassword`, then split it in dovecot configuration in order to override the username and password. But not only didn’t I found a way to split the string, but also, if I understand correctly, setting another user to PAM module will override the name of the mailbox I want to open. What would you suggest me to do in order to get this desired behavior? Thank you.

2 2

Mail locations must use the same hierarchy separator
by Mike 21 Feb '25

21 Feb '25

Hello, First post, and yes Ive searched the archives and didn't find an answer. Sorry if this is old hat but... I am trying build a new mailserver (iredmail) and import messages from the old server (old old iredmail), and convert from dbox to maildir in the process. I have previously used doveadm sync successfully several years/versions ago with a master user and imapc:. Now however, Im getting errors that don't make sense to me. doveadm -D -c /etc/dovecot/dovecot.conf sync -1 -R -u example(a)example.com imapc: results in: dsync(example(a)example.com): Info: imapc(mail.example.com:993): Connected to 192.0.2.1:993 (local 192.0.2.1:51732) dsync(example(a)example.com): Error: Mail locations must use the same hierarchy separator (specify namespace prefix="" { separator } explicitly) I have tried all sorts of chicanery including testing every combination of "" and / and . for each of prefix and separator in dovecot.conf, no dice. I am refraining from providing full debug dump/config files since this seems excessive if this is a config file issue, but will do so upon your advice if requested. Thank you. Hello, First post, and yes Ive searched the archives and didn't find an answer. Sorry if this is old hat but... I am trying build a new mailserver (iredmail) and import messages from the old server (old old iredmail), and convert from dbox to maildir in the process. I have previously used doveadm sync successfully several years/versions ago with a master user and imapc:. Now however, Im getting errors that don't make sense to me. doveadm -D -c /etc/dovecot/dovecot.conf sync -1 -R -u example(a)example.com imapc: results in: dsync(example(a)example.com): Info: imapc(mail.example.com:993): Connected to 192.0.2.1:993 (local 192.0.2.1:51732) dsync(example(a)example.com): Error: Mail locations must use the same hierarchy separator (specify namespace prefix="" { separator } explicitly) I have tried all sorts of chicanery including testing every combination of "" and / and . for each of prefix and separator in dovecot.conf, no dice. I am refraining from providing full debug dump/config files since this seems excessive if this is a config file issue, but will do so upon your advice if requested. Thank you.

2 1

Unable to debug sieve
by Roberto Puzzanghera 20 Feb '25

20 Feb '25

I'm trying to enable sieve debug with no success on dovecot 4.0. I would like to know what it is doing when taking decisions if running a script or not and what it does when a script is executed These settings do not enable the trace sieve_trace_debug = yes sieve_trace_dir = /tmp (I also tried any other world writable dir and also ~/.sieve while testing trace for personal scripts) nor this one is enabling the debug log_debug = category=sieve sieve is working perfectly for personal but not for global scripts. Any hint appreciated. Thanks in advance, Roberto

1 0

autoexpunge
by Doug Hardie 20 Feb '25

20 Feb '25

I have the following in 15-mailboxes.config: mailbox Trash { special_use = \Trash autoexpunge = 30 days } I thought that would empty the deleted emails after 30 days. However, I find that in .Deleted Messages/cur there are over 18K messages dating back to the 90's. What do I need to set to make the autoexpunge work? -- Doug

6 9

iOS Mail.app Network error POSIX 54: is there a workaround?
by Gerben Wierda 19 Feb '25

19 Feb '25

I understand that this error that appears on iOS 18 devices is an iOS bug. Is this indeed the case and is there anything I can do about it on th server side as a workaround? Thanks, Gerben

1 0

Preventing message deletion
by dovecot＠iotti.biz 19 Feb '25

19 Feb '25

Hi all How may I disable message deletions via IMAP for some or all of my users? I read on the net that a possible solution would be to use the ACL IMAP plugin. But the examples I found were not so clear to me. We do not use shared mailboxes or namespaces. Only simple virtual users, each with it's own private mailbox. I would like to use the global acl file both for administration simplicity and because as I understood, global acl enrties take preference over any user setting. In the ACL page example there is a line: * user=foo lrw But in the comment it tells that doing so, every user' mailbox would be shared with the foo user, with the lrw permissions. Which is not what I would want. I don't need to share anything, just to restrict what the user, foo here, can do on his mailbox. Thank you, Luigi

2 4

IMAP dovecot\postgres low authentication performance
by Anatoliy Zhestov 19 Feb '25

19 Feb '25

Hi. We have a performance problem with imap authentication through postgresql. Our servers(modoboa based) have a big amount of permanent imap connections(5000-50000). Current performance is about 3000 successful authentications per hour. No visible reasons for such low speed. Accordingly, after a network failure or server restart, all clients try to reconnect, but restoring the connection pool takes hours and even tens of hours. Judging by the logs after the restart, a huge number of auth requests closed by timeout after 70-90 seconds. The postgresql database is not overloaded at the restore connections process and the postgresql connection pool (100) does not overflow. Manually started sql auth queries work fast, tables have indexes. So I guess there is a bottleneck somewhere in dovecot auth service or postgresql driver. I couldn't find any settings in the documentation that are directly responsible for regulating the number of connections to PostgreSQL from the auth service or performance of the driver. Is there any way to manage this? Does it make sense to use pgbouncer in front of the base? What else can be a bottleneck in our configuration and how to regulate the number of possible simultaneous authentications via PostgreSQL? I would be happy for any advice on how to increase the performance to at least 100 authentications per second. Thanks in advance for your help. Anatoliy Zhestov. ---------------- configs of server with 48 core, 184 gb mem --------------- dovecot 2.3.16 # 2.3.16 (7e2e900c1a): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.16 (09c29328) # OS: Linux 5.15.0-126-generic x86_64 Ubuntu 22.04.5 LTS # Hostname: imap.ourcompany.net auth_cache_negative_ttl = 0 auth_cache_size = 20 M auth_cache_ttl = 3 hours auth_master_user_separator = * auth_mechanisms = plain login default_client_limit = 131072 default_process_limit = 131072 dict { quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } mail_location = maildir:~/Maildir mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mai lbox date index ihave duplicate mime foreverypart extracttext editheader imapflags notify vnd.dovecot.pipe vnd.dovecot.execute namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } passdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql master = yes pass = yes } plugin { quota = dict:User quota::proxy::quota sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_execute_bin_dir = /usr/lib/dovecot/sieve-execute sieve_execute_socket_dir = . sieve_extensions = +notify +imapflags +editheader +body +vnd.dovecot.execute +vnd.dovecot.pipe sieve_pipe_bin_dir = /usr/lib/dovecot/sieve-execute sieve_pipe_socket_dir = . sieve_plugins = sieve_extprograms } protocols = " imap lmtp sieve" service anvil { client_limit = 393219 } service auth { client_limit = 524288 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-radicale { group = radicale mode = 0666 user = radicale } unix_listener auth-userdb { user = vmail } } service config { vsz_limit = 1 G } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap { executable = imap postlogin process_limit = 131072 vsz_limit = 1 G } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } vsz_limit = 512 M } service pop3 { executable = pop3 postlogin } service postlogin { executable = script-login /usr/local/bin/postlogin.sh service_count = 1000 process_min_avail = 5 user = modoboa } service send-to-click-frontapp1 { executable = script /usr/lib/dovecot/sieve-execute/clickhouse_log_frontapp1.sh unix_listener send-to-click-frontapp1 { group = vmail mode = 0666 user = vmail } user = dovenull vsz_limit = 2 G } service send-to-click { executable = script /usr/lib/dovecot/sieve-execute/clickhouse_log.sh unix_listener send-to-click { group = vmail mode = 0666 user = vmail } user = dovenull vsz_limit = 2 G } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } vsz_limit = 2 G } ssl_cert = </etc/letsencrypt/live/imap.ourcompany.net/fullchain.pem ssl_cipher_list = EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS:!RC4 ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1 stats_writer_socket_path = userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota sieve postmaster_address = postmaster(a)imap.ourcompany.net } protocol imap { mail_max_userip_connections = 50 mail_plugins = quota imap_quota } /etc/dovecot/dovecot-sql-master.conf.ex driver = pgsql connect = host=127.0.0.1 port=5432 dbname=modoboa user=modoboa password=XXXXXXXXXXXXXXXXX password_query = SELECT email AS user, password FROM core_user WHERE email='%u' and is_active and master_user /etc/dovecot/dovecot-sql.conf.ext driver = pgsql connect = host=127.0.0.1 port=5432 dbname=modoboa user=modoboa password=XXXXXXXXXXXXXXXXX user_query = SELECT '/srv/vmail/%d/%n' AS home, 1003 as uid, 1003 as gid, '*:bytes=' || mb.quota || 'M' AS quota_rule FROM admin_mailbox mb INNER JOIN admin_domain dom ON mb.domain_id=dom.id INNER JOIN core_user u ON u.id=mb.user_id WHERE (mb.is_send_only IS NOT TRUE OR '%s' NOT IN ('imap', 'pop3', 'lmtp')) AND mb.address='%n' AND dom.name='%d' password_query = SELECT email AS user, password, '/srv/vmail/%d/%n' AS userdb_home, 1003 AS userdb_uid, 1003 AS userdb_gid, CONCAT('*:bytes=', mb.quota, 'M') AS userdb_quota_rule FROM core_user u INNER JOIN admin_mailbox mb ON u.id=mb.user_id INNER JOIN admin_domain dom ON mb.domain_id=dom.id WHERE (mb.is_send_only IS NOT TRUE OR '%s' NOT IN ('imap', 'pop3')) AND email='%u' AND is_active AND dom.enabled iterate_query = SELECT email AS user FROM core_user /usr/local/bin/postlogin.sh #!/bin/sh PATH="/usr/bin:/usr/local/bin:/bin" psql -c "UPDATE core_user SET last_login=now() WHERE username='$USER'" > /dev/null exec "$@" /usr/lib/dovecot/sieve-execute/clickhouse_log_frontapp1.sh #!/bin/bash cd /tmp logger -i -t SIEVE_TEST "$1,$2,$3,$4" clickhouse-client -h XXXXXXXXXXXXXXXXX -d default -u some_user --password XXXXXXXXXXXXXXXXX \ --param_origin_from="$1" \ --param_origin_to="$2" \ --param_message_id="$3" \ --param_forward="$4" \ -q "INSERT INTO frontapp1_redirects (origin_from_email,origin_to_email,message_id,forward_email) VALUES ({origin_from:String},{origin_to:String},{message_id:String},{forward:String})" /usr/lib/dovecot/sieve-execute/clickhouse_log.sh #!/bin/bash cd /tmp logger -i -t SIEVE_TEST "$1,$2,$3,$4" clickhouse-client -h XXXXXXXXXXXXXXXXX -d default -u some_user --password XXXXXXXXXXXXXXXXX \ --param_origin_from="$1" \ --param_origin_to="$2" \ --param_message_id="$3" \ --param_forward="$4" \ -q "INSERT INTO frontapp_redirects (origin_from_email,origin_to_email,message_id,forward_email) VALUES ({origin_from:String},{origin_to:String},{message_id:String},{forward:String})"

6 30

auth_userdb_iterate
by Luigi Trovato 19 Feb '25

19 Feb '25

Hello, I have a doubt: is the lua function "auth_userdb_iterate" expected to always return the whole list of users no matter what, or it should evaluate if a specific virtual domain is provided as req argument and only list the users in that specific domain? And should the returned user list be in the format "username(a)domain.tld" or "username" ? -- Ciao, Luigi

1 1