[patch] proposed patch for managing timeouts on doveadm iteration with -A
Hi
having further investigated the timeouts mentioned in the previous thread, I am submitting this patch as a potential solution.
Basically with -A doveadm parameter, the 155 second timeout is applied to the whole end to end transaction. With -F parameter the 155 second timeout is applied to each single user operation. Changing the logic of -A to follow -F looks a bit invasive. So this patch does more or less the same thing, i.e. giving each user operation 155 seconds timeout, done in a different way. Whenever another username is returned, the timeout is reset.
PS I got help from Claude on this.
best regards
John
I can't see the attachment I added so here it is inline --- a/src/lib-auth-client/auth-master.c +++ b/src/lib-auth-client/auth-master.c @@ -1341,6 +1341,26 @@ return ctx; } +static void +auth_master_user_list_extend_timeout(struct auth_master_user_list_ctx *ctx) +{ + /* A LIST request is a single long-lived, multi-reply request that + stays alive (cycling through AUTH_MASTER_REQUEST_STATE_REPLIED_MORE) + for the entire user enumeration, which for large installations can + take far longer than the normal auth lookup timeout allows for a + single request. Since we just made real progress (another username + was returned), push the request's deadline out from now instead of + leaving it anchored to when the LIST request was first created. + This turns the timeout into a "no progress for N seconds" stall + detector for listing, matching how independent, short-lived + per-user requests (e.g. as used when iterating -F user files) + never accumulate a single deadline across the whole run. */ + if (ctx->req != NULL) { + ctx->req->create_stamp = ioloop_timeval; + auth_master_connection_update_timeout(ctx->conn); + } +} + static const char * auth_master_user_do_list_next(struct auth_master_user_list_ctx *ctx) { @@ -1363,8 +1383,10 @@ } if (ctx->finished || ctx->failed) return NULL; - if (str_len(ctx->username) > 0) + if (str_len(ctx->username) > 0) { + auth_master_user_list_extend_timeout(ctx); return str_c(ctx->username); + } } /* wait for more data */ @@ -1377,6 +1399,7 @@ if (ctx->finished || ctx->failed || ctx->req == NULL) return NULL; i_assert(str_len(ctx->username) > 0); + auth_master_user_list_extend_timeout(ctx); return str_c(ctx->username); } On 25/07/2026 23:17, John Fawcett via dovecot wrote:
Hi
having further investigated the timeouts mentioned in the previous thread, I am submitting this patch as a potential solution.
Basically with -A doveadm parameter, the 155 second timeout is applied to the whole end to end transaction. With -F parameter the 155 second timeout is applied to each single user operation. Changing the logic of -A to follow -F looks a bit invasive. So this patch does more or less the same thing, i.e. giving each user operation 155 seconds timeout, done in a different way. Whenever another username is returned, the timeout is reset.
PS I got help from Claude on this.
best regards
John
_______________________________________________ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
"John" == John Fawcett via dovecot <dovecot@dovecot.org> writes:
having further investigated the timeouts mentioned in the previous thread, I am submitting this patch as a potential solution.
I like this, it makes sense that the timeout is per-lookup, not overall length. Now I also admit that for 20,000 (or more!) look ups, maybe you do want it to time out after a while... or give some feedback on how far it's gotten into the lookup.
Basically with -A doveadm parameter, the 155 second timeout is applied to the whole end to end transaction. With -F parameter the 155 second timeout is applied to each single user operation. Changing the logic of -A to follow -F looks a bit invasive. So this patch does more or less the same thing, i.e. giving each user operation 155 seconds timeout, done in a different way. Whenever another username is returned, the timeout is reset.
Simple and clean patch.
PS I got help from Claude on this.
best regards
John
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
On 26/07/2026 20:14, John Stoffel via dovecot wrote:
"John" == John Fawcett via dovecot <dovecot@dovecot.org> writes: having further investigated the timeouts mentioned in the previous thread, I am submitting this patch as a potential solution. I like this, it makes sense that the timeout is per-lookup, not overall length. Now I also admit that for 20,000 (or more!) look ups, maybe you do want it to time out after a while... or give some feedback on how far it's gotten into the lookup.
With this fix the -A operations won't timeout so long as data is coming back regularly. It should still timeout if the server side hangs. If the original 155 seconds timeout was due to this being applied to all operations including list operations there could be a case to reduce that timeout for all operations.
For the quota get operation, and also for other operations like user, there is an output per user, so the progress is visible.
Operations like doveadm quote recalc -A don't normally produce output, but adding -v will provide a progress counter per 100 users processed.
Basically with -A doveadm parameter, the 155 second timeout is applied to the whole end to end transaction. With -F parameter the 155 second timeout is applied to each single user operation. Changing the logic of -A to follow -F looks a bit invasive. So this patch does more or less the same thing, i.e. giving each user operation 155 seconds timeout, done in a different way. Whenever another username is returned, the timeout is reset. Simple and clean patch.
PS I got help from Claude on this. best regards John
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
On 26. Jul 2026, at 0.17, John Fawcett via dovecot <dovecot@dovecot.org> wrote:
Hi
having further investigated the timeouts mentioned in the previous thread, I am submitting this patch as a potential solution.
Basically with -A doveadm parameter, the 155 second timeout is applied to the whole end to end transaction. With -F parameter the 155 second timeout is applied to each single user operation. Changing the logic of -A to follow -F looks a bit invasive. So this patch does more or less the same thing, i.e. giving each user operation 155 seconds timeout, done in a different way. Whenever another username is returned, the timeout is reset.
PS I got help from Claude on this.
Some further improvements via Fable - does this still fix your problems? commit 9deefd88b37d4e9bfecdf6bd1c9fa8a3f9d6b100 Author: Timo Sirainen <timo.sirainen@open-xchange.com> Date: Wed Aug 5 21:33:42 2026 +0000 lib-auth-client: Extend LIST request timeout whenever a username is returned The auth lookup timeout (155 seconds by default) was applied to the whole LIST request, i.e. the entire user enumeration. With large installations listing all users can easily take longer than that, causing e.g. doveadm -A to fail with "Auth server request timed out". Reset the request's timeout whenever another username is returned. This turns the timeout into a "no progress for N seconds" stall detector, matching how independent short-lived per-user requests (e.g. as used when iterating -F user files) never accumulate a single deadline across the whole run. The timeout handling assumes that requests expire in creation order, so extending is only allowed while the request is alone on its connection. This is always the case for LIST, which is asserted at init already. Based on code by John Fawcett diff --git a/src/lib-auth-client/auth-master-private.h b/src/lib-auth-client/auth-master-private.h index 770195c2dc..10986ee155 100644 --- a/src/lib-auth-client/auth-master-private.h +++ b/src/lib-auth-client/auth-master-private.h @@ -89,6 +89,7 @@ struct auth_master_connection { unsigned int auth_master_request_get_timeout_msecs(struct auth_master_request *req); +void auth_master_request_extend_timeout(struct auth_master_request *req); void auth_master_request_send(struct auth_master_request *req); int auth_master_request_got_reply(struct auth_master_request **_req, diff --git a/src/lib-auth-client/auth-master-request.c b/src/lib-auth-client/auth-master-request.c index cc6feba483..ef153ac18a 100644 --- a/src/lib-auth-client/auth-master-request.c +++ b/src/lib-auth-client/auth-master-request.c @@ -31,6 +31,19 @@ auth_master_request_get_timeout_msecs(struct auth_master_request *req) return (unsigned int)(msecs < 0 ? 0 : msecs); } +void auth_master_request_extend_timeout(struct auth_master_request *req) +{ + struct auth_master_connection *conn = req->conn; + + /* The timeout handling assumes that requests expire in the order + they were created, so the deadline of the oldest request must not + be pushed past the deadlines of newer requests. */ + i_assert(conn->requests_count == 1); + + req->create_stamp = ioloop_timeval; + auth_master_connection_update_timeout(conn); +} + static void auth_master_request_remove(struct auth_master_request *req) { struct auth_master_connection *conn = req->conn; diff --git a/src/lib-auth-client/auth-master.c b/src/lib-auth-client/auth-master.c index 4c19cfbcc3..5414f723c5 100644 --- a/src/lib-auth-client/auth-master.c +++ b/src/lib-auth-client/auth-master.c @@ -1381,6 +1381,17 @@ const char *auth_master_user_list_next(struct auth_master_user_list_ctx *ctx) if (username == NULL) return NULL; + /* A LIST request is a single long-lived, multi-reply request that + stays alive for the entire user enumeration, which for large + installations can take far longer than the normal auth lookup + timeout allows for a single request. Since we just made progress + (another username was returned), push the request's deadline out + from now instead of leaving it anchored to when the LIST request + was first created. This turns the timeout into a "no progress for + N seconds" stall detector for listing. */ + i_assert(ctx->req != NULL); + auth_master_request_extend_timeout(ctx->req); + e_debug(ctx->event, "Returned username: %s", username); return username; }
Thanks for this Timo, I'll take a look and post my results asap. On 05/08/2026 23:35, Timo Sirainen via dovecot wrote:
On 26. Jul 2026, at 0.17, John Fawcett via dovecot <dovecot@dovecot.org> wrote:
Hi
having further investigated the timeouts mentioned in the previous thread, I am submitting this patch as a potential solution.
Basically with -A doveadm parameter, the 155 second timeout is applied to the whole end to end transaction. With -F parameter the 155 second timeout is applied to each single user operation. Changing the logic of -A to follow -F looks a bit invasive. So this patch does more or less the same thing, i.e. giving each user operation 155 seconds timeout, done in a different way. Whenever another username is returned, the timeout is reset.
PS I got help from Claude on this. Some further improvements via Fable - does this still fix your problems?
commit 9deefd88b37d4e9bfecdf6bd1c9fa8a3f9d6b100 Author: Timo Sirainen <timo.sirainen@open-xchange.com> Date: Wed Aug 5 21:33:42 2026 +0000
lib-auth-client: Extend LIST request timeout whenever a username is returned
The auth lookup timeout (155 seconds by default) was applied to the whole LIST request, i.e. the entire user enumeration. With large installations listing all users can easily take longer than that, causing e.g. doveadm -A to fail with "Auth server request timed out".
Reset the request's timeout whenever another username is returned. This turns the timeout into a "no progress for N seconds" stall detector, matching how independent short-lived per-user requests (e.g. as used when iterating -F user files) never accumulate a single deadline across the whole run.
The timeout handling assumes that requests expire in creation order, so extending is only allowed while the request is alone on its connection. This is always the case for LIST, which is asserted at init already.
Based on code by John Fawcett
diff --git a/src/lib-auth-client/auth-master-private.h b/src/lib-auth-client/auth-master-private.h index 770195c2dc..10986ee155 100644 --- a/src/lib-auth-client/auth-master-private.h +++ b/src/lib-auth-client/auth-master-private.h @@ -89,6 +89,7 @@ struct auth_master_connection {
unsigned int auth_master_request_get_timeout_msecs(struct auth_master_request *req); +void auth_master_request_extend_timeout(struct auth_master_request *req);
void auth_master_request_send(struct auth_master_request *req); int auth_master_request_got_reply(struct auth_master_request **_req, diff --git a/src/lib-auth-client/auth-master-request.c b/src/lib-auth-client/auth-master-request.c index cc6feba483..ef153ac18a 100644 --- a/src/lib-auth-client/auth-master-request.c +++ b/src/lib-auth-client/auth-master-request.c @@ -31,6 +31,19 @@ auth_master_request_get_timeout_msecs(struct auth_master_request *req) return (unsigned int)(msecs < 0 ? 0 : msecs); }
+void auth_master_request_extend_timeout(struct auth_master_request *req) +{ + struct auth_master_connection *conn = req->conn; + + /* The timeout handling assumes that requests expire in the order + they were created, so the deadline of the oldest request must not + be pushed past the deadlines of newer requests. */ + i_assert(conn->requests_count == 1); + + req->create_stamp = ioloop_timeval; + auth_master_connection_update_timeout(conn); +} + static void auth_master_request_remove(struct auth_master_request *req) { struct auth_master_connection *conn = req->conn; diff --git a/src/lib-auth-client/auth-master.c b/src/lib-auth-client/auth-master.c index 4c19cfbcc3..5414f723c5 100644 --- a/src/lib-auth-client/auth-master.c +++ b/src/lib-auth-client/auth-master.c @@ -1381,6 +1381,17 @@ const char *auth_master_user_list_next(struct auth_master_user_list_ctx *ctx) if (username == NULL) return NULL;
+ /* A LIST request is a single long-lived, multi-reply request that + stays alive for the entire user enumeration, which for large + installations can take far longer than the normal auth lookup + timeout allows for a single request. Since we just made progress + (another username was returned), push the request's deadline out + from now instead of leaving it anchored to when the LIST request + was first created. This turns the timeout into a "no progress for + N seconds" stall detector for listing. */ + i_assert(ctx->req != NULL); + auth_master_request_extend_timeout(ctx->req); + e_debug(ctx->event, "Returned username: %s", username); return username; }
_______________________________________________ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
participants (3)
-
John Fawcett
-
John Stoffel
-
Timo Sirainen