January 2024 - dovecot

recipient delimiter option in dovecot
by André Rodier 17 Jan '24

17 Jan '24

Hello, all. Postfix allows me to configure multiple characters to be used as recipient delimiter. However, I don't see that in dovecot, it seems I can only use one character in recipient_delimiter ? Am I right ? Does dovecot would only let me use email+box(a)domain.com, or can I use other forms, for instance email~box(a)domain.com and email&box(a)domain.com ? Thanks. André -- 𝓐𝓡 - 𝐴𝑛𝑑𝑟𝑒 𝑅𝑜𝑑𝑖𝑒𝑟

2 1

Virtual folder with custom flags
by Kirill Korinsky 15 Jan '24

15 Jan '24

Greetings, I can't figure out how doesn it possible to create a virtual folder which contains mails which has falgs: \Flagged or any customers flag. For example I do have a test mail: mx2$ doas doveadm fetch -u kirill(a)korins.ky "mailbox flags" mailbox-guid fb9e821c7b778b658896000079e31c01 uid 67096 mailbox: Archive flags: \Flagged \Seen $NotJunk NotJunk $MailFlagBit0 $MailFlagBit2 Invoice mx2$ and virtual folder: mx2$ cat /etc/dovecot/virtual/Flagged/dovecot-virtual * -INBOX/Trash -INBOX/Trash/* -INBOX/Junk -INBOX/Junk/* flagged mx2$ If I unset flag \Flagged from that mail, it dissapear from the folder. So, how should I change definition to keep it here if it has user's specified flags like Invoice or ASAP or whatsoever? -- wbr, Kirill

2 1

Quota clone plugin - initial population of data
by John Fawcett 15 Jan '24

15 Jan '24

Hi while responding on a previous thread, I noticed in the documentation: https://doc.dovecot.org/configuration_manual/quota_clone_plugin/ "For example you might want to access everybody’s quota via Redis ... but you don’t store the Redis database permanently so it could become empty once in a while. " It aslo seems that the quota clone plugin propagates only quota changes. Putting both of these points together, you'll going to have missing quotas in redis if it is restarted. It will gradually get populated but only on basis of mailbox changes. Maybe there should be an initial load of all the quotas when dovecot starts, or could that be a performance issue? Maybe even doveadm quota recalc should propagate or contain an option to propage quotas so that even if it can't be done on dovecot startup, at least it can be done on demand by the administrator. John

1 0

ARM support
by Brad Koehn 15 Jan '24

15 Jan '24

Hey there, I was wondering if you’re planning to support ARM in your packages and/or Docker images. Right now I’m stuck on whatever ships with Debian for ARM64 architectures. Otherwise, suggestions how would I go about building them myself? In a modern build world, it’s fairly straightforward, especially with Docker to handle all the emulation. Thanks for such a great product! I’ve used Dovecot on my little server for over twenty years. — Brad Koehn Hey there, I was wondering if you’re planning to support ARM in your packages and/or Docker images. Right now I’m stuck on whatever ships with Debian for ARM64 architectures. Otherwise, suggestions how would I go about building them myself? In a modern build world, it’s fairly straightforward, especially with Docker to handle all the emulation. Thanks for such a great product! I’ve used Dovecot on my little server for over twenty years. — Brad Koehn

8 15

Custom map for quota
by Matt 15 Jan '24

15 Jan '24

Hi, I'm trying to setup quota with the following layout CREATE TABLE "domain" ( "id" INTEGER, "name" TEXT, "active" INTEGER DEFAULT 0, PRIMARY KEY("id"), UNIQUE("name") ) CREATE TABLE "address" ( "id" INTEGER, "localpart" TEXT NOT NULL, "domain_id" INTEGER NOT NULL, PRIMARY KEY("id"), UNIQUE("localpart","domain_id") ) CREATE TABLE "mailbox" ( "id" INTEGER, "address_id" INTEGER, "active" INTEGER DEFAULT 0, "password" TEXT, "quota_bytes" INTEGER DEFAULT 0, PRIMARY KEY("id") ) As I like to update the quota and I like to use a dict/map I'm not sure if I can update this following: map { pattern = priv/quota/storage table = maibox value_field = dummy fields { value_field = quota_bytes id = mailbox_id } mailbox_id will be queried using password_query using: "mailbox.id as mailbox_id" Using such map I hope to be able to use: SELECT quota_bytes FROM mailbox WHERE id = '$mailbox_id' INSERT INTO mailbox (quota_bytes) VALUES ('$value') ON DUPLICATE KEY UPDATE quota_bytes='$value' Can someone help me with this ? Hi, I'm trying to setup quota with the following layout CREATE TABLE "domain" ( "id" INTEGER, "name" TEXT, "active" INTEGER DEFAULT 0, PRIMARY KEY("id"), UNIQUE("name") ) CREATE TABLE "address" ( "id" INTEGER, "localpart" TEXT NOT NULL, "domain_id" INTEGER NOT NULL, PRIMARY KEY("id"), UNIQUE("localpart","domain_id") ) CREATE TABLE "mailbox" ( "id" INTEGER, "address_id" INTEGER, "active" INTEGER DEFAULT 0, "password" TEXT, "quota_bytes" INTEGER DEFAULT 0, PRIMARY KEY("id") ) As I like to update the quota and I like to use a dict/map I'm not sure if I can update this following: map { pattern = priv/quota/storage table = maibox value_field = dummy fields { value_field = quota_bytes id = mailbox_id } mailbox_id will be queried using password_query using: "mailbox.id as mailbox_id" Using such map I hope to be able to use: SELECT quota_bytes FROM mailbox WHERE id = '$mailbox_id' INSERT INTO mailbox (quota_bytes) VALUES ('$value') ON DUPLICATE KEY UPDATE quota_bytes='$value' Can someone help me with this ?

3 3

Panic: file index-search-result.c: line 132 (index_search_result_update_flags): assertion failed: (result->search_args->args == &search_arg)
by Markus Doits 14 Jan '24

14 Jan '24

Hello dovecot community, I hope you can help me with the following panic: --- Panic: file index-search-result.c: line 132 (index_search_result_update_flags): assertion failed: (result->search_args->args == &search_arg) Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x7f839a045846] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x7f839a045962] -> /usr/lib/dovecot/libdovecot.so.0(+0x10d81b) [0x7f839a05281b] -> /usr/lib/dovecot/libdovecot.so.0(+0x10d8b7) [0x7f839a0528b7] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e2e2) [0x7f8399fa32e2] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x518a4) [0x7f839a1768a4] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x274e) [0x7f8399ea4a9e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x5c) [0x7f839a192f9c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x39) [0x7f839a193039] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x37) [0x7f839a210467] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0xc6c5) [0x7f8399e9f6c5] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x145e2) [0x7f8399ee65e2] -> /usr/lib/dovecot/modules/lib01_acl_plugin.so(+0x10cb9) [0x7f8399f31cb9] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x64) [0x7f839a1935a4] -> dovecot/imap(imap_status_get+0x98) [0x55c63c180748] -> dovecot/imap(cmd_status+0x19e) [0x55c63c170a7e] -> dovecot/imap(command_exec+0xa4) [0x55c63c177444] -> dovecot/imap(+0x22392) [0x55c63c175392] -> dovecot/imap(+0x22444) [0x55c63c175444] -> dovecot/imap(client_handle_input+0x1bd) [0x55c63c17589d] -> dovecot/imap(client_input+0x74) [0x55c63c175e64] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6d) [0x7f839a06903d] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x13a) [0x7f839a06a7aa] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x54) [0x7f839a0690e4] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x7f839a0692a0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x17) [0x7f8399fd9ad7] -> dovecot/imap(main+0x570) [0x55c63c166d20] -> /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f8399c29d90] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f8399c29e40] -> dovecot/imap(_start+0x25) [0x55c63c166de5] Fatal: master: service(imap): child 793983 killed with signal 6 (core dumped) --- Happening on ubuntu 22.04, dovecot 2.3.21 from [this ubuntu ppa](https://launchpad.net/~patrickdk/+archive/ubuntu/production/+packages?… dovecot-core:amd64 1:2.3.21-0a~jammy dovecot-dev:amd64 1:2.3.21-0a~jammy dovecot-imapd:amd64 1:2.3.21-0a~jammy dovecot-ldap:amd64 1:2.3.21-0a~jammy dovecot-lmtpd:amd64 1:2.3.21-0a~jammy dovecot-sieve:amd64 1:2.3.21-0a~jammy dovecot-solr:amd64 1:2.3.21-0a~jammy --- I have this virtual mailbox configured: namespace { prefix = _/ separator = / location = virtual:~/Maildir/virtual:LAYOUT=maildir++:INDEX=~/Maildir/virtual } When I remove this mailbox, the error goes away - so it seems to be something with the virtual mailbox. As the fts provider I use [fts_flatcurve](https://slusarz.github.io/dovecot-fts-flatcurve), maybe this has something to do with it? Any help appreciated how to debug this further and maybe fix it. Thanks!

3 3

execute: /user/bin/checkpassword /user/libexec/dovecot/checkpassword-reply
by Steve Button 14 Jan '24

14 Jan '24

Hi, I have recently moved a dovecot installation from 2.2.36 on RHEL7.9 onto 2.3.20 on Amazon Linux 2023 and I'm now seeing the error message in the title relating to /usr/bin/checkpassword (which doesn't exist on either). This is a project that I've inherited, and I've never used dovecot previously. I've been searching around for a couple of days trying to figure this out. Running out of ideas. Here's a slightly redacted snippet from dovecot.log Jan 04 17:23:22 auth: Debug: checkpassword(redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): Performing passdb lookup Jan 04 17:23:22 auth: Debug: checkpassword(redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): execute: /usr/bin/checkpassword /usr/libexec/dovecot/checkpassword-reply Jan 04 17:23:22 auth: Fatal: execv(/usr/bin/checkpassword) failed: No such file or directory Jan 04 17:23:22 auth: Debug: checkpassword(redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): Received input: Jan 04 17:23:22 auth: Debug: checkpassword(redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): exit_status=84 Jan 04 17:23:22 auth: Error: checkpassword(redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): Child 106455 exited with status 84 Jan 04 17:23:22 auth: Debug: checkpassword(redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): Finished passdb lookup Jan 04 17:23:22 auth: Debug: auth(redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): Auth request finished Jan 04 17:23:24 auth: Debug: client passdb out: FAIL 1 user=redacted(a)redacted.redacted.local code=temp_fail Jan 04 17:23:24 imap-login: Info: Disconnected: Connection closed (auth service reported temporary failure): user=redacted(a)redacted.redacted.local, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, session=<6uwL/SEOzpt/AAAB> Jan 04 17:23:24 auth: Debug: auth client connected (pid=106456) Jan 04 17:23:24 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=R5kq/SEO0Jt/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=39888 As part of the build process we overwrite the config files with ones which are presumably from a previous version. In particular there's an auth.conf file which contains :- !include auth-checkpassword.conf.ext and in turn that file contains the stanza :- passdb { driver = checkpassword args = /usr/bin/checkpassword } which is where I guess the problem lies. The thing is the config files are almost identical on both systems. Here's the dovecot -n output... [root@server dovecot]# dovecot -n # 2.3.20 (xyz675d): /etc/dovecot/dovecot.conf # OS: Linux 6.1.61-85.141.amzn2023.x86_64 x86_64 ext4 # Hostname: server...1a97d auth_debug = yes auth_verbose = yes disable_plaintext_auth = no first_valid_gid = 0 first_valid_uid = 0 listen = * log_path = /var/project/log/dovecot/dovecot.log mail_location = maildir:/var/mail/vhosts/%d/%n mail_privileged_group = mail mbox_write_locks = fcntl passdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } passdb { args = /usr/bin/checkpassword driver = checkpassword } protocols = imap service auth { unix_listener auth-userdb \{ group = postfix mode = 0600 user = postfix } } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = # hidden, use -P to show it userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static } userdb { driver = prefetch } Hi, I have recently moved a dovecot installation from 2.2.36 on RHEL7.9 onto 2.3.20 on Amazon Linux 2023 and I'm now seeing the error message in the title relating to /usr/bin/checkpassword (which doesn't exist on either). This is a project that I've inherited, and I've never used dovecot previously. I've been searching around for a couple of days trying to figure this out. Running out of ideas. Here's a slightly redacted snippet from dovecot.log Jan 04 17:23:22 auth: Debug: checkpassword (redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): Performing passdb lookup Jan 04 17:23:22 auth: Debug: checkpassword (redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): execute: /usr/ bin/checkpassword /usr/libexec/dovecot/checkpassword-reply Jan 04 17:23:22 auth: Fatal: execv(/usr/bin/checkpassword) failed: No such file or directory Jan 04 17:23:22 auth: Debug: checkpassword (redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): Received input: Jan 04 17:23:22 auth: Debug: checkpassword (redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): exit_status=84 Jan 04 17:23:22 auth: Error: checkpassword (redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): Child 106455 exited with status 84 Jan 04 17:23:22 auth: Debug: checkpassword (redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): Finished passdb lookup Jan 04 17:23:22 auth: Debug: auth (redacted(a)redacted.redacted.local,127.0.0.1,<6uwL/SEOzpt/AAAB>): Auth request finished Jan 04 17:23:24 auth: Debug: client passdb out: FAIL 1 user=redacted(a)redacted.redacted.local code=temp_fail Jan 04 17:23:24 imap-login: Info: Disconnected: Connection closed (auth service reported temporary failure): user=redacted(a)redacted.redacted.local, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, session=<6uwL/SEOzpt/AAAB> Jan 04 17:23:24 auth: Debug: auth client connected (pid=106456) Jan 04 17:23:24 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=R5kq/SEO0Jt/AAAB lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=39888 As part of the build process we overwrite the config files with ones which are presumably from a previous version. In particular there's an auth.conf file which contains :- !include auth-checkpassword.conf.ext and in turn that file contains the stanza :- passdb { driver = checkpassword args = /usr/bin/checkpassword } which is where I guess the problem lies. The thing is the config files are almost identical on both systems. Here's the dovecot -n output... [root@server dovecot]# dovecot -n # 2.3.20 (xyz675d): /etc/dovecot/dovecot.conf # OS: Linux 6.1.61-85.141.amzn2023.x86_64 x86_64 ext4 # Hostname: server...1a97d auth_debug = yes auth_verbose = yes disable_plaintext_auth = no first_valid_gid = 0 first_valid_uid = 0 listen = * log_path = /var/project/log/dovecot/dovecot.log mail_location = maildir:/var/mail/vhosts/%d/%n mail_privileged_group = mail mbox_write_locks = fcntl passdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } passdb { args = /usr/bin/checkpassword driver = checkpassword } protocols = imap service auth { unix_listener auth-userdb \{ group = postfix mode = 0600 user = postfix } } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = # hidden, use -P to show it userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static } userdb { driver = prefetch }

2 1

Unconfigurable 500 ms debounce on IMAP IDLE notifications
by Alex 14 Jan '24

14 Jan '24

Hello all, I have been investigating the reason for high end-to-end email delivery delay (>1 second to send a message to echo bot and receive a reply back) on a Postfix+Dovecot email server setup used to run the tests for Delta Chat core [1]. It appeared that most of the time was spent between the submission SMTP service accepting the mail for delivery and receiving an IDLE wakeup on the other side. Looking into the Dovecot source code, I found that all notifications from the storage to callbacks set up by IDLE are delayed by a debounce constant NOTIFY_DELAY_MSECS [2]. It appears to have been added in 2009 [3], but even before that Dovecot was only delivering notifications when a second-resolution timer is changed by at least 1, so IDLE notifications probably were delayed by half a second on average even before that. Commit message says "If the notification is done immediately, IDLE may not notice the change because it's not finished yet." This sounds like a workaround for some internal Dovecot bug. I have nevertheless tried to compile Dovecot with this constant reduced from 500 ms to 1 ms [4] and it resulted in exceptionally fast delivery without any problems caused to the Delta Chat core online test suite which is using this Postfix+Dovecot server accounts. We are running all the tests against this server and it appears to work fine. Round trip time for sending a message to an echo bot and receiving a reply back is reduced from 1.1 s to 0.2 s, which is easily visible in a chat app [5]. I wonder if this debounce delay was added as a workaround for an internal Dovecot bug or a client bug and is not necessary anymore. It does not seem to be useful for well-behaving clients because if necessary the client may debounce notifications on its side and postpone interrupting IDLE with "DONE". I would actually like to disable this delay completely on our server setup [6], as for chat it is common to receive multiple messages in a short period of time, e.g. when sending a message to a group and receiving multiple read receipts from currently active users. [1] https://github.com/deltachat/deltachat-core-rust/ [2] https://github.com/dovecot/core/blob/93a53a9d590f0220de28600f36a969cc38c314… [3] https://github.com/dovecot/core/commit/56fb5d09955b6097f77b341717fd9b70e9f1… [4] https://github.com/deltachat/chatmail/issues/72#issuecomment-1806645153 [5] https://delta.chat/assets/blog/nine-echo.mp4 [6] https://github.com/deltachat/chatmail Best regards, Alex

2 6

Lots of "dovecot: auth: Error: auth client 0 disconnected with 1 pending requests: EOF" errors in error.log
by Jay Sladojevic 14 Jan '24

14 Jan '24

I've recently upgraded to dovecot 2.3 and now my error.log is full of this message: dovecot: auth: Error: auth client 0 disconnected with 1 pending requests: EOF In earlier versions, this error was logged at a warning level. What is the rationale behind the elevation to error log level? May I suggest to revert this change? (fyi: the code in question is in /src/auth/auth-client-connection.c) static void auth_client_disconnected(struct auth_client_connection **_conn) { struct auth_client_connection *conn = *_conn; unsigned int request_count; int err; *_conn = NULL; if (conn->input->stream_errno != 0) err = conn->input->stream_errno; else if (conn->output->stream_errno != 0) err = conn->output->stream_errno; else err = 0; request_count = conn->request_handler == NULL ? 0 : auth_request_handler_get_request_count(conn->request_handler); if (request_count > 0) { e_error(conn->event, "auth client %u disconnected with %u " "pending requests: %s", conn->pid, request_count, err == 0 ? "EOF" : strerror(err)); } auth_client_connection_destroy(&conn); }

1 0

Redirects 550'd So why no SRS method
by Laura Steynes 05 Jan '24

05 Jan '24

Happy New Year! Now to Aki, Timo, Corr, why, when setting up mail forwarding, is sieve not automatically configuring rules for SRS, it has this ability for long time, yet dovecots sieve based forwarding just creates a plain old redirect "forward" resulting in,in 2024, 99% of forwarded emails getting rejected for failing SPF because it's not enabling the require or the rules needed for successful sender rewriting. Is this an oversight? and I suppose the bigger question is, when is it planned to be corrected to using SRS method? Happy New Year! Now to Aki, Timo, Corr, why, when setting up mail forwarding, is sieve not automatically configuring rules for SRS, it has this ability for long time, yet dovecots sieve based forwarding just creates a plain old redirect "forward" resulting in,in 2024, 99% of forwarded emails getting rejected for failing SPF because it's not enabling the require or the rules needed for successful sender rewriting. Is this an oversight? and I suppose the bigger question is, when is it planned to be corrected to using SRS method?

3 2