- dovecot - dovecot.org

2.4.4 fts-flatcurve: fts optimize silently destroys a folder's index when the index is on NFS
by Ihor Rusyn 11 Aug '26

11 Aug '26

Hello, Summary ======= On Dovecot 2.4.4 with fts_flatcurve and mail_index_path on NFS, "doveadm fts optimize" leaves behind an empty shard directory. That directory remains the shard every subsequent write opens, so from that point on the folder can no longer index new mail. The operation reports success and logs nothing at the time. The damage is permanent and, on a folder that receives no new mail, invisible. Two separate defects are involved, and both are reported below: (1) fts optimize deletes a shard directory whose files it still holds open. On a local filesystem the unlink succeeds invisibly; NFS cannot express "unlink but keep open" and silly-renames instead, so the directory survives. (2) unlink_directory() reports success when rmdir() failed, which is what makes (1) silent rather than loud. Environment =========== Dovecot 2.4.4 CE (8b687aa65c) container image sha256:d6b2f80db2e656a9250583712a82397e2752b35ad7e4e2e71c85fdc0d510137d base Debian 13 trixie fts_flatcurve (Xapian glass) libxapian30 1.4.29-3 (libxapian.so.30.14.1) substring_search = yes, min_term_size = 2 optimize_limit at its default of 10 dovecot_storage_version = 2.3.0 mail_index_path on NFSv3; mail storage on a separate NFS export NFS client is the host, not the container: Debian 12 bookworm, kernel 6.1.0-51-amd64 x86_64 nfs-common 1:2.6.2-4+deb12u1 mount options: rw,noatime,vers=3,rsize=32768,wsize=32768,hard, nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys, local_lock=none Impact ====== An operator does not have to invoke optimize to be exposed: with the default optimize_limit = 10, flatcurve runs it itself once a folder accumulates ten shards. Every active folder therefore reaches this eventually. Already-indexed mail stays searchable, so the folder looks healthy. Only indexing of new mail fails. With fts_search_add_missing = yes, the first SEARCH after new mail arrives attempts to index it, fails, and the client receives NO [SERVERBUG]. I hit this while migrating a host from 2.3 to 2.4: 114 mailboxes were left in this state in a single run, and it surfaced only because a monitoring mailbox began answering SERVERBUG. Defect 1: fts optimize leaves an empty shard directory on NFS ============================================================= Steps to reproduce ------------------ 1. Configure fts_flatcurve with the index path on an NFSv3 mount. 2. Take a mailbox with mail in INBOX (a few hundred messages is enough) and build the index: doveadm index -u <user> INBOX Confirm one shard exists: ls <index>/.INBOX/fts-flatcurve/ -> current.<id> 3. Run: doveadm fts optimize -u <user> INBOX (Alternatively, let it fire on its own by driving the folder past optimize_limit shards.) 4. While it runs, list the shard directory repeatedly - .nfsXXXX entries appear inside current.<id>. 5. After it finishes, note the exit status is 0 and no error was logged. List the directory again: index.<n> was created as expected, and current.<id> is still there, now empty. 6. Deliver one new message, then: doveadm index -u <user> INBOX Expected vs actual ------------------ Expected: after optimize, only the compacted index.<n> remains, and subsequent indexing writes to a new shard. Actual: the stale empty current.<id> remains and is reopened for writing, so step 6 fails: Cannot open DB (RW, current.<id>): .../current.<id>/iamglass: Failed to open glass revision file for reading (No such file or directory) cmd index: Mailbox INBOX: Precache for UID=59 failed and IMAP SEARCH on that folder returns NO [SERVERBUG]. What I observed while diagnosing -------------------------------- strace of "doveadm fts optimize" on NFSv3 shows the old shard's files being unlinked while handles on them are still open, so the directory cannot be removed: unlink(...) = 0 (x7, every file in the shard) rmdir(...) = -1 ENOTEMPTY (Directory not empty) rename(...) = 0 (optimize -> index.N, proceeds regardless) Once the process exits, the NFS client reaps the .nfsXXXX files and the directory is simply empty. What does not help ------------------ NFSv4.1, mounting with nolock, "doveadm fts rescan", "doveadm index", and full re-indexing all take the same write path into the same empty shard and fail identically. Removing the empty directory (or the folder's whole fts-flatcurve tree) and re-indexing restores the folder; setting optimize_limit = 0 avoids the situation but leaves the index spread across many shards. Defect 2: unlink_directory() returns success when rmdir() failed ================================================================ The ENOTEMPTY above never reaches the caller. unlink_directory() in src/lib/unlink-directory.c returns 1 when the final rmdir() fails, the same value it returns on success: if (rmdir(dir) < 0 && errno != ENOENT) { *error_r = t_strdup_printf("rmdir(%s) failed: %m", dir); return errno == ENOENT ? 0 : 1; } The "errno != ENOENT" guard already establishes that errno is not ENOENT, so the ternary always yields 1. Callers cannot distinguish "directory removed" from "directory still there", and the error message it composed is discarded. Any condition that makes rmdir() fail with something other than ENOENT reproduces this, so the return value is wrong independently of NFS - every caller of unlink_directory() is affected. The code path is unchanged in current main. -- Best regards, Ihor Ru Hello, Summary ======= On Dovecot 2.4.4 with fts_flatcurve and mail_index_path on NFS, "doveadm fts optimize" leaves behind an empty shard directory. That directory remains the shard every subsequent write opens, so from that point on the folder can no longer index new mail. The operation reports success and logs nothing at the time. The damage is permanent and, on a folder that receives no new mail, invisible. Two separate defects are involved, and both are reported below: (1) fts optimize deletes a shard directory whose files it still holds open. On a local filesystem the unlink succeeds invisibly; NFS cannot express "unlink but keep open" and silly-renames instead, so the directory survives. (2) unlink_directory() reports success when rmdir() failed, which is what makes (1) silent rather than loud. Environment =========== Dovecot 2.4.4 CE (8b687aa65c) container image sha256:d6b2f80db2e656a9250583712a82397e2752b35ad7e4e2e71c85fdc0d510137d base Debian 13 trixie fts_flatcurve (Xapian glass) libxapian30 1.4.29-3 (libxapian.so.30.14.1) substring_search = yes, min_term_size = 2 optimize_limit at its default of 10 dovecot_storage_version = 2.3.0 mail_index_path on NFSv3; mail storage on a separate NFS export NFS client is the host, not the container: Debian 12 bookworm, kernel 6.1.0-51-amd64 x86_64 nfs-common 1:2.6.2-4+deb12u1 mount options: rw,noatime,vers=3,rsize=32768,wsize=32768,hard, nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys, local_lock=none Impact ====== An operator does not have to invoke optimize to be exposed: with the default optimize_limit = 10, flatcurve runs it itself once a folder accumulates ten shards. Every active folder therefore reaches this eventually. Already-indexed mail stays searchable, so the folder looks healthy. Only indexing of new mail fails. With fts_search_add_missing = yes, the first SEARCH after new mail arrives attempts to index it, fails, and the client receives NO [SERVERBUG]. I hit this while migrating a host from 2.3 to 2.4: 114 mailboxes were left in this state in a single run, and it surfaced only because a monitoring mailbox began answering SERVERBUG. Defect 1: fts optimize leaves an empty shard directory on NFS ============================================================= Steps to reproduce ------------------ 1. Configure fts_flatcurve with the index path on an NFSv3 mount. 2. Take a mailbox with mail in INBOX (a few hundred messages is enough) and build the index: doveadm index -u <user> INBOX Confirm one shard exists: ls <index>/.INBOX/fts-flatcurve/ -> current.<id> 3. Run: doveadm fts optimize -u <user> INBOX (Alternatively, let it fire on its own by driving the folder past optimize_limit shards.) 4. While it runs, list the shard directory repeatedly - .nfsXXXX entries appear inside current.<id>. 5. After it finishes, note the exit status is 0 and no error was logged. List the directory again: index.<n> was created as expected, and current.<id> is still there, now empty. 6. Deliver one new message, then: doveadm index -u <user> INBOX Expected vs actual ------------------ Expected: after optimize, only the compacted index.<n> remains, and subsequent indexing writes to a new shard. Actual: the stale empty current.<id> remains and is reopened for writing, so step 6 fails: Cannot open DB (RW, current.<id>): .../current.<id>/iamglass: Failed to open glass revision file for reading (No such file or directory) cmd index: Mailbox INBOX: Precache for UID=59 failed and IMAP SEARCH on that folder returns NO [SERVERBUG]. What I observed while diagnosing -------------------------------- strace of "doveadm fts optimize" on NFSv3 shows the old shard's files being unlinked while handles on them are still open, so the directory cannot be removed: unlink(...) = 0 (x7, every file in the shard) rmdir(...) = -1 ENOTEMPTY (Directory not empty) rename(...) = 0 (optimize -> index.N, proceeds regardless) Once the process exits, the NFS client reaps the .nfsXXXX files and the directory is simply empty. What does not help ------------------ NFSv4.1, mounting with nolock, "doveadm fts rescan", "doveadm index", and full re-indexing all take the same write path into the same empty shard and fail identically. Removing the empty directory (or the folder's whole fts-flatcurve tree) and re-indexing restores the folder; setting optimize_limit = 0 avoids the situation but leaves the index spread across many shards. Defect 2: unlink_directory() returns success when rmdir() failed ================================================================ The ENOTEMPTY above never reaches the caller. unlink_directory() in src/lib/unlink-directory.c returns 1 when the final rmdir() fails, the same value it returns on success: if (rmdir(dir) < 0 && errno != ENOENT) { *error_r = t_strdup_printf("rmdir(%s) failed: %m", dir); return errno == ENOENT ? 0 : 1; } The "errno != ENOENT" guard already establishes that errno is not ENOENT, so the ternary always yields 1. Callers cannot distinguish "directory removed" from "directory still there", and the error message it composed is discarded. Any condition that makes rmdir() fail with something other than ENOENT reproduces this, so the return value is wrong independently of NFS - every caller of unlink_directory() is affected. The code path is unchanged in current main. -- Best regards, Ihor Ru

1 0

Tribulations trying to pin-down viable sieve script constraints in connection with imap events.
by sjh_dovecot_org＠shic.co.uk 10 Aug '26

10 Aug '26

Hi Dovecot experts... I've recently started what I expected to be a tiny project to migrate from a legacy ad-hoc procmail by embracing Sieve scripting. I have two different frustrations - depending upon which approach I adopt to trigger spam-training actions. *Option 1*: Trigger actions when messages are moved to/from Spam folder. This mostly works... but has has a niggle I can't see how to overcome. I don't have only one 'Spam' folder - I have a hierarchy of folders - with names like "Spam.2026.July" and "Spam.Unsubscribed.2023". My 'learn ham' sieve script uses "imap.mailbox" from environment to determine if the email is being moved out of the Spam hierarchy. This works well. Unfortunately, when triggering 'learn spam' - the value of imap.mailbox is always the destination mailbox 'Spam' - not the source mailbox from which the spammy email has been moved. I can't see any way to trigger a spam reporting action, within Sieve scripts, if (and only if) a message is moved from a folder that does not have a name matching 'Spam.*' into the folder 'Spam'. This restriction is required so that adjustments to filed spam messages do not result in a single email being wrongly presented multiple times for learning as a spam example. This multiple reporting might otherwise happen if, for example, a message was moved between 'Spam' and 'Spam.2025.July" ; "Spam.2025.August" and "Spam.ForReview.2025" several times. Have I overlooked a reliable way to determine the source folder, in a Sieve script, when a message is moved (copied) to the Spam folder? *Option 2*: Trigger actions when the Thunderbird 'Junk' and 'Not Junk' buttons are pressed I accept it would be preferable to avoid depending upon client-specific flags... but I do expect Thunderbird to be the mail client... and marking messages as Junk/Spam in Thunderbird does promise a neat user interface experience. I've read RFC 6785 (also RFC 5228 and RFC 5232). I understand that the "imap.changedflags" environment value is set both when flags have been added and when removed... and that it is expected that hasflag should be used to determine if the imap.changedflag flag was added or removed. This makes perfect sense, in principle... but the actual behaviour I see is peculiar. With a sieve script /var/dovecot-sieve/scripts/flags_changed.sieve (compiled with sievec) like this: require ["environment", "imap4flags", "imapsieve", "variables", "vnd.dovecot.debug"]; if header :matches "subject" "*" { debug_log "FLAG-EMAIL Subject: ${1}"; } if environment :matches "imap.changedflags" "*" { debug_log "FLAG-EMAIL changedflags : ${1}"; } if hasflag :matches "*" { debug_log "FLAG-EMAIL hasflag : ${1}"; } and configuration in /etc/dovecot/conf.d/90-sieve.conf like this (N.B. user sieve script does not interact with flags... type=after and type=before behave similarly): sieve_script flags_changed { type = after cause = flag path = /var/dovecot-sieve/scripts/flags_changed.sieve } I experimented with the "Junk" and "Not Junk" buttons for an email with a distinctive subject. The following nested bullet-list summarises the values I found for various scenarios. My experiment followed this order of scenarios - there should have been no flag changes, for the email in question, between each of the scenarios in the test sequence. * Scenario 1 - Fresh, not-deterimed-to-be-spam email arrives in Inbox... - Sieve script invocation 1 * imap.changedflags => NonJunk * hasflag => NonJunk - Notes: As expected. * Scenario 2 - Same email is marked as read by Thunderbird after it has been displayed onscreen for a while... - Sieve script invocation 2 * imap.changedflags => \Seen * hasflag => \seen - Notes: Flags are case insensitive - but odd lower case distinction. What happened to NonJunk flag from Scenario 1? * Scenario 3 - Junk button pressed to manually mark this email as spam... - Sieve script invocation 3 * imap.changedflags => Junk * hasflag => \seen - Sieve script invocation 4 * imap.changedflags => NonJunk * hasflag => \seen - Notes: Why neither Junk nor NonJunk flags set for hasflag? * Scenario 4 - "Not Junk" button is pressed to manually mark this email as ham... - Sieve script invocation 5 * imap.changedflags => NonJunk * hasflag => \seen - Sieve script invocation 6 * imap.changedflags => Junk * hasflag => \seen - Sieve script invocation 7 * imap.changedflags => \Seen * hasflag => NonJunk Perhaps it is relevant... where I have captured the value from hasflag... that only one token is ever reported - even where I'd expect multiple flags present. I would have expected at least to see both 'Junk' and 'NotJunk' alongside '\seen' for hasflag at appropriate stages in the sequence of scenarios. Is there a problem with how I've used the :match idiom (with "*") intending to capture all the flags associated with the message? (If so, what approach should I have used instead?) Is there a bug which prevents using the hasflag feature to disambiguate between a flag reported for imap.changedflags having been added versus having been removed? If not, where have I gone wrong? Versions... Server-side: Debian Trixie: dovecot-core/stable,stable-security,now 1:2.4.1+dfsg1-6+deb13u6 amd64 [installed] dovecot-imapd/stable,stable-security,now 1:2.4.1+dfsg1-6+deb13u6 amd64 [installed] dovecot-sieve/stable,stable-security,now 1:2.4.1+dfsg1-6+deb13u6 amd64 [installed] Client-side: Windows 11 Thunderbird 140.12.1esr (64-bit) Steve NonJunk \seen 5 Junk \seen 6 \Seen NonJunk 7 Weird... Is NonJunk reported for hasflag because it is now the only flag associated with the message (after \Seen flag removed)?

3 5

[patch] proposed patch for managing timeouts on doveadm iteration with -A
by John Fawcett 10 Aug '26

10 Aug '26

Hi having further investigated the timeouts mentioned in the previous thread, I am submitting this patch as a potential solution. Basically with -A doveadm parameter, the 155 second timeout is applied to the whole end to end transaction. With -F parameter the 155 second timeout is applied to each single user operation. Changing the logic of -A to follow -F looks a bit invasive. So this patch does more or less the same thing, i.e. giving each user operation 155 seconds timeout, done in a different way. Whenever another username is returned, the timeout is reset. PS I got help from Claude on this. best regards John

3 5

Bug report: Pigeonhole Sieve delivery plugin fails to load in Dovecot 2.4 — undefined symbol mail_deliver_ctx_get_log_var_expand_table
by Developer Sub N Grill 08 Aug '26

08 Aug '26

Hello Dovecot Team, I am reporting what appears to be a packaging/ABI issue affecting the Pigeonhole Sieve delivery plugin in Dovecot 2.4. I have performed extensive diagnostics and would appreciate any guidance. Environment Operating System: Debian 13 (Trixie), amd64 Repository: Official Dovecot CE repository (repo.dovecot.org/ce-2.4-latest/debian/trixie) Installed package version:2:2.4.4-5+debian13 Installed packages: dovecot-core dovecot-sieve dovecot-lmtpd dovecot-imapd dovecot-pop3d dovecot-managesieved dovecot-pgsql All packages are installed at the exact same version. Problem Description When enabling the Sieve delivery plugin with: mail_plugins { sieve = yes } Dovecot fails to load the plugin with the following error: sievec: Fatal: Couldn't load required plugin /usr/lib/dovecot/modules/lib90_sieve_plugin.so: dlopen() failed: /usr/lib/dovecot/modules/lib90_sieve_plugin.so: undefined symbol: mail_deliver_ctx_get_log_var_expand_table As a result, message delivery fails entirely whenever the Sieve delivery plugin is enabled. Disabling the plugin restores normal mail delivery. Diagnostics Performed1. Package verification All installed packages are version 2:2.4.4-5+debian13 from the official Dovecot CE repository. There are no mixed package versions. 2. Symbol verification I searched for the symbol mail_deliver_ctx_get_log_var_expand_table throughout the Dovecot libraries. Results: nm -D on libdovecot-lda.so.0.0.0 → symbol not found readelf -sW on libdovecot-lda.so.0.0.0 → symbol not found Recursive search across all .so files under: /usr/lib/dovecot/ /usr/lib/x86_64-linux-gnu/dovecot/ → symbol not exported anywhere The symbol string exists inside both: lib90_sieve_plugin.so libdovecot-lda.so.0.0.0 but only as plain text (visible via grep), never as an exported ELF symbol. 3. Dependency verification ldd on lib90_sieve_plugin.so shows dependencies on: libdovecot-sieve.so.0 libdovecot-storage.so.0 libdovecot.so.0 It does not directly link against libdovecot-lda.so. 4. Standalone sievec Compiling a Sieve script works correctly: sievec /tmp/test.sieve Exit status is 0 as long as no configuration enables the Sieve delivery plugin. The failure only occurs once: mail_plugins { sieve = yes } is present in the configuration. Reproduced on a Clean Debian 13 Installation To rule out configuration or upgrade issues, I reproduced the exact same problem on a completely clean debian:13-slim container. I installed only the Debian stable packages: apt-get install dovecot-core dovecot-sieve dovecot-lmtpd (version 1:2.4.1+dfsg1-6+deb13u6) Using the same production configuration with: mail_plugins { sieve = yes } produces the exact same error: sievec: Fatal: Couldn't load required plugin ... undefined symbol: mail_deliver_ctx_get_log_var_expand_table This indicates the problem is not caused by: mixed package versions an upgrade issue local modifications custom configuration The issue reproduces consistently on both: Dovecot CE 2.4.4 Debian 13's packaged Dovecot 2.4.1 Questions Is mail_deliver_ctx_get_log_var_expand_table a new internal function that should be exported in Dovecot 2.4? Was there an ABI change affecting the Pigeonhole plugin? Is this a known issue in the Dovecot 2.4.x packages? If so, is a fix planned (for example, in 2.4.5)? Is there any recommended workaround for using Sieve during message delivery (e.g. moving spam to the Junk folder) while this issue is unresolved? I am trying to use the new Dovecot 2.4 configuration syntax (sieve_script_type, sieve_script_precedence, and sieve_script_path), since sieve_before and the plugin {} block were removed in 2.4. Thank you for your time. I would be happy to provide additional logs, the complete doveconf -n output, or a core dump if that would be helpful. Best regards, Miguel A Morales Server: Debian 13 (amd64) Dovecot: 2.4.4-5+debian13 Hello Dovecot Team, I am reporting what appears to be a packaging/ABI issue affecting the Pigeonhole Sieve delivery plugin in Dovecot 2.4. I have performed extensive diagnostics and would appreciate any guidance. Environment o Operating System: Debian 13 (Trixie), amd64 o Repository: Official Dovecot CE repository (repo.dovecot.org/ce-2.4-latest/debian/trixie) o Installed package version: 2:2.4.4-5+debian13 Installed packages: o dovecot-core o dovecot-sieve o dovecot-lmtpd o dovecot-imapd o dovecot-pop3d o dovecot-managesieved o dovecot-pgsql All packages are installed at the exact same version. Problem Description When enabling the Sieve delivery plugin with: mail_plugins { sieve = yes } Dovecot fails to load the plugin with the following error: sievec: Fatal: Couldn't load required plugin /usr/lib/dovecot/modules/lib90_sieve_plugin.so: dlopen() failed: /usr/lib/dovecot/modules/lib90_sieve_plugin.so: undefined symbol: mail_deliver_ctx_get_log_var_expand_table As a result, message delivery fails entirely whenever the Sieve delivery plugin is enabled. Disabling the plugin restores normal mail delivery. Diagnostics Performed 1. Package verification All installed packages are version 2:2.4.4-5+debian13 from the official Dovecot CE repository. There are no mixed package versions. 2. Symbol verification I searched for the symbol mail_deliver_ctx_get_log_var_expand_table throughout the Dovecot libraries. Results: o nm -D on libdovecot-lda.so.0.0.0 -> symbol not found o readelf -sW on libdovecot-lda.so.0.0.0 -> symbol not found o Recursive search across all .so files under: o /usr/lib/dovecot/ o /usr/lib/x86_64-linux-gnu/dovecot/ -> symbol not exported anywhere o The symbol string exists inside both: o lib90_sieve_plugin.so o libdovecot-lda.so.0.0.0 but only as plain text (visible via grep), never as an exported ELF symbol. 3. Dependency verification ldd on lib90_sieve_plugin.so shows dependencies on: o libdovecot-sieve.so.0 o libdovecot-storage.so.0 o libdovecot.so.0 It does not directly link against libdovecot-lda.so. 4. Standalone sievec Compiling a Sieve script works correctly: sievec /tmp/test.sieve Exit status is 0 as long as no configuration enables the Sieve delivery plugin. The failure only occurs once: mail_plugins { sieve = yes } is present in the configuration. Reproduced on a Clean Debian 13 Installation To rule out configuration or upgrade issues, I reproduced the exact same problem on a completely clean debian:13-slim container. I installed only the Debian stable packages: apt-get install dovecot-core dovecot-sieve dovecot-lmtpd (version 1:2.4.1+dfsg1-6+deb13u6) Using the same production configuration with: mail_plugins { sieve = yes } produces the exact same error: sievec: Fatal: Couldn't load required plugin ... undefined symbol: mail_deliver_ctx_get_log_var_expand_table This indicates the problem is not caused by: o mixed package versions o an upgrade issue o local modifications o custom configuration The issue reproduces consistently on both: o Dovecot CE 2.4.4 o Debian 13's packaged Dovecot 2.4.1 Questions 1. Is mail_deliver_ctx_get_log_var_expand_table a new internal function that should be exported in Dovecot 2.4? Was there an ABI change affecting the Pigeonhole plugin? 2. Is this a known issue in the Dovecot 2.4.x packages? If so, is a fix planned (for example, in 2.4.5)? 3. Is there any recommended workaround for using Sieve during message delivery (e.g. moving spam to the Junk folder) while this issue is unresolved? I am trying to use the new Dovecot 2.4 configuration syntax (sieve_script_type, sieve_script_precedence, and sieve_script_path), since sieve_before and the plugin {} block were removed in 2.4. Thank you for your time. I would be happy to provide additional logs, the complete doveconf -n output, or a core dump if that would be helpful. Best regards, Miguel A Morales Server: Debian 13 (amd64) Dovecot: 2.4.4-5+debian13

3 2

Can't connect to MySQL without SSL
by dovecot＠fagschlunger.at 08 Aug '26

08 Aug '26

Hi! I try to migrate from dovecot 2.3 to dovecot CE 2.4, but within a docker container. But I always get the following error from Dovecot CE: Error: auth: mysql(host.docker.internal): Connect failed to database (foobar): TLS/SSL error: self-signed certificate in certificate chain - waiting for 1 seconds before retry This is how I configured the mysql connection: mysql host.docker.internal { host = host.docker.internal port = 3306 user = ... password = ... dbname = foobar ssl = no } So MySQL runs on the host machine directly, not in a container - therefore I'm using "host.docker.internal" as an host. But I don't need SQL encryption, since both MySQL and Dovecot run on the same machine. So why the heck does it look like, Dovecot is trying to connect using SSL? Best regards, Billie

5 7

Re: [patch] proposed patch for managing timeouts on doveadm iteration with -A
by MK 07 Aug '26

07 Aug '26

>On 26. Jul 2026, at 0.17, John Fawcett via dovecot <dovecot(a)dovecot.org> wrote: >> >> Hi >> >> having further investigated the timeouts mentioned in the previous thread, I am submitting this patch as a potential solution. >> >> Basically with -A doveadm parameter, the 155 second timeout is applied to the whole end to end transaction. With -F parameter the 155 second >timeout is applied to each single user operation. Changing the logic of -A to follow -F looks a bit invasive. So this patch does more or less the >same thing, i.e. giving each user operation 155 seconds timeout, done in a different way. Whenever another username is returned, the timeout is >reset. >> >> PS I got help from Claude on this. > >Some further improvements via Fable - does this still fix your problems? > >commit 9deefd88b37d4e9bfecdf6bd1c9fa8a3f9d6b100 >Author: Timo Sirainen <timo.sirainen(a)open-xchange.com> >Date: Wed Aug 5 21:33:42 2026 +0000 > > lib-auth-client: Extend LIST request timeout whenever a username is returned > > The auth lookup timeout (155 seconds by default) was applied to the whole > LIST request, i.e. the entire user enumeration. With large installations > listing all users can easily take longer than that, causing e.g. > doveadm -A to fail with "Auth server request timed out". > > Reset the request's timeout whenever another username is returned. This > turns the timeout into a "no progress for N seconds" stall detector, > matching how independent short-lived per-user requests (e.g. as used when > iterating -F user files) never accumulate a single deadline across the > whole run. > > The timeout handling assumes that requests expire in creation order, so > extending is only allowed while the request is alone on its connection. > This is always the case for LIST, which is asserted at init already. Timo can you say in which upcoming version this patch will be included? Oliver

1 0

Detection of HAVE_MYSQL_SSL_VERIFY_SERVER_CERT is broken
by Tomas Volf 31 Jul '26

31 Jul '26

Hello, detection of HAVE_MYSQL_SSL_VERIFY_SERVER_CERT is currently broken, and the macro is never defined. The test program fails, but for incorrect reason. --8<---------------cut here---------------start------------->8--- conftest.c:140:20: error: expected declaration specifiers or '...' before ',' token 140 | _au_m4_changequote(,) | ^ conftest.c:140:21: error: expected declaration specifiers or '...' before ')' token 140 | _au_m4_changequote(,) | ^ In file included from /gnu/store/6lmd3apzkmhwzj3yn0inyj484m9qml6y-mariadb-connector-c-3.4.5/include/mariadb/mysql.h:35, from conftest.c:141: /gnu/store/7iwcvd0gjrfq6bj7zvf3r59baw6vib5a-gcc-14.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/14.3.0/include/stdarg.h:103:9: error: unknown type name '__gnuc_va_list' 103 | typedef __gnuc_va_list va_list; | ^~~~~~~~~~~~~~ --8<---------------cut here---------------end--------------->8--- Fix is to delete the only occurrence of _au_m4_changequote([,]) in the m4/want_mysql.m4 file. However, that will reveal a compilation error here: --8<---------------cut here---------------start------------->8--- #ifdef HAVE_MYSQL_SSL_VERIFY_SERVER_CERT int ssl_verify_server_cert = ssl_set->ssl_client_require_valid_cert ? 1 : 0; mysql_options(db->mysql, MYSQL_OPT_SSL_VERIFY_SERVER_CERT, (void *)&ssl_verify_server_cert); #endif #else --8<---------------cut here---------------end--------------->8--- Variable `ssl_set' does not exist and should be `db->ssl_set'. Have a nice day, Tomas Volf -- There are only two hard things in Computer Science: cache invalidation, naming things and off-by-one errors.

1 0

Confusion about mailbox_list_layout=fs
by Roland Hieber 27 Jul '26

27 Jul '26

Hi, I am trying to understand how 'mailbox_list_layout = fs' is working when I have 'mail_driver = Maildir' (on Dovecot 2.4.1-4 from Debian trixie). If I look at 'doveconf -a', I see that 'mailbox_list_layout = fs' is already the default, however, the Maildirs from my freshly created test user are still created on disk in the format .INBOX.subfolder instead of INBOX/subfolder, which is in conflict with what the documentation says at <https://doc.dovecot.org/2.4.3/core/config/mailbox_formats/maildir.html#dire…>. However, if I set 'mailbox_list_layout = fs' *explicitely* in the global part of /etc/dovecot/conf.d/10-mail.conf, then delete all files and folders in the mail home of my test user, reload dovecot, send a test mail, and create a new subfolder for my test user in Thunderbird, the new subfolder gets created on disk as INBOX/subfolder, just like I expect. The behaviour is consistently reproducable in both directions: I also get maildirs with dots again after I delete mailbox_list_layout from 10-mail.conf, reload dovecot, and clear the mail home of my test user. What is going on here…? I don't see any change in 'doveconf -a' between these two use cases (the output is literally identical, and it's both showing 'mailbox_list_layout = fs'), and obviously it's missing from 'doveconf -n' because it's the default. I also found out that it is possible to override this setting per user by using the 'userdb_mailbox_list_layout' option in the extra fields of the userdb, but I'm not setting any extra fields there for my test user. Output of 'doveconf -n' (redacted where necessary) is attached. Regards, - Roland -- Roland Hieber, Pengutronix e.K. | rhi(a)pengutronix.de | Steuerwalder Str. 21 | https://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |

2 2

Doveadm and iterate query
by Bogusław Juza 25 Jul '26

25 Jul '26

Hi! I have a problem with Dovecot on Debian 13: Dovecot 2.4.1, Debian Trixie > doveadm mailbox path -A INBOX >/dev/null doveadm(email@domain): Error: auth-master: userdb list: User listing returned failure doveadm: Error: cmd mailbox path: Failed to iterate through some users userdb sql { iterate_query = SELECT `email` AS `user` FROM `users` } In syslog file: dovecot: auth: Error: auth-worker: Aborted LIST request for *: Shutting down I have over 30k users in sql DB, doveadm with -A option is dying few second after start with this error, printing about 10% of users. I need much more time (FE command "doveadm expunge -A mailbox Spam savedbefore 3w" could run over 30 minutes). What I'm doing wrong? Bogusław Juza

5 12

Reporting a Patch needed: Fix-anvil-busy-loop-on-EPOLLHUP-when-a-multiplex-cha.patch
by David Lanning 24 Jul '26

24 Jul '26

The attached patch spec is what we had to deploy to cover the matter. The attached patch spec is what we had to deploy to cover the matter.

2 1