doveadm backup / purge problems since migration from 2.3 to 2.4
Hi everyone,
I have a two-server setup where all user data from the actively used server is periodically transferred to the second server using "doveadm backup". For this, I currently use a script that caches a list of all users (containing only the usernames) via "doveadm user '*' " and then executes the command "doveadm backup -u [username] tcp:1.2.3.4" (target host) for each user. I use a similarly structured script for a daily "doveadm purge", where a purge is performed per user instead of a backup.
Under version 2.3, I always ran both commands using the "-A" flag (doveadm purge -A and doveadm backup -A tcp:1.2.3.4). Since migrating from Dovecot 2.3 to 2.4, this no longer works. When running the purge, the process always aborts after a certain number of users with the following error message:
doveadm(john@doe.de): Error: auth-master: userdb list: User listing returned failure
It's not the same User allways but after nearly the same time or amount of users the script has processed. Since I couldn't identify the exact cause of the error and suspect it might be related to changes in how the SQL iterate query is integrated in the 2.4 configuration (I use MySQL as the backend) I switched to the script described above.
Then since is used the new script I have a non-reproducible issue. After the initial data transfer (when the target is still empty) the amount of data is equal to the source. Then after one of the next transfers the data is twice as high. It is random as to which subsequent transfer triggers this. When comparing the quotas on the source and target server, the output is identical on both. Even running quota recalc makes no difference. If I check the user directories, I can see that the storage folder contains nearly double the number of files and used disk space per user.
My second problem is that when I run the purge script on the second server (target), it sometimes deletes all emails for all users, also non-reproducible. Afterwards, all files under "storage" are gone, and the quota shows 0 emails. I did not have this issue with the exactly same purge script on the first server which is the source for the backup.
The version change was during the update from Debian 12 to 13, involving an upgrade of Dovecot from version 2.3.19 to 2.4.1. (both packages from debian repo) We use MySQL as the backend and mdbox as the storage format.
I hope some can help me to solve this issue.
Greetings, Oliver
On 09/07/2026 11:58 EEST MK via dovecot <dovecot@dovecot.org> wrote:
Hi everyone,
I have a two-server setup where all user data from the actively used server is periodically transferred to the second server using "doveadm backup". For this, I currently use a script that caches a list of all users (containing only the usernames) via "doveadm user '*' " and then executes the command "doveadm backup -u [username] tcp:1.2.3.4" (target host) for each user. I use a similarly structured script for a daily "doveadm purge", where a purge is performed per user instead of a backup.
Under version 2.3, I always ran both commands using the "-A" flag (doveadm purge -A and doveadm backup -A tcp:1.2.3.4). Since migrating from Dovecot 2.3 to 2.4, this no longer works. When running the purge, the process always aborts after a certain number of users with the following error message:
doveadm(john@doe.de): Error: auth-master: userdb list: User listing returned failure
Try adding
log_debug=category=auth
and see if it can shed light one this.
It's not the same User allways but after nearly the same time or amount of users the script has processed. Since I couldn't identify the exact cause of the error and suspect it might be related to changes in how the SQL iterate query is integrated in the 2.4 configuration (I use MySQL as the backend) I switched to the script described above.
Then since is used the new script I have a non-reproducible issue. After the initial data transfer (when the target is still empty) the amount of data is equal to the source. Then after one of the next transfers the data is twice as high. It is random as to which subsequent transfer triggers this. When comparing the quotas on the source and target server, the output is identical on both. Even running
quota recalcmakes no difference. If I check the user directories, I can see that thestoragefolder contains nearly double the number of files and used disk space per user.My second problem is that when I run the purge script on the second server (target), it sometimes deletes all emails for all users, also non-reproducible. Afterwards, all files under "storage" are gone, and the quota shows 0 emails. I did not have this issue with the exactly same purge script on the first server which is the source for the backup.
What does your purge script do?
The version change was during the update from Debian 12 to 13, involving an upgrade of Dovecot from version 2.3.19 to 2.4.1. (both packages from debian repo) We use MySQL as the backend and mdbox as the storage format.
Aki
participants (2)
-
Aki Tuomi
-
MK