Hi there
On a Dovecot 2.4.2 (Debian Bookworm) server with over 1300 users, I am using the following config to retrieve quota usage information with doveadm:
quota "User quota" { }
mail_plugins = quota acl mail_log notify
userdb sql { iterate_query = SELECT username AS user, CONCAT(quota, 'M') AS quota_storage_size FROM mailaccounts WHERE active = 1
query =
SELECT '/var/vmail/%{user | domain}/%{user | username}' AS home, 'vmail' AS uid, 'vmail' AS gid, CONCAT(quota, 'M') AS quota_storage_size
FROM mailaccounts
WHERE username = '%{user}' AND (active = 1 OR transport_suspended = 0)
}
Initially, I had to run doveadm quota recalc -A to recalc quotas and populate the count quota driver which is enabled by default for "User quota".
But still, it took roughly 20ms per single user or 15-20s for all users:
$ time doveadm -f tab quota get -u demo@example.com real 0m0.020s $ time doveadm -f tab quota get -A real 0m16.800s
Now, to greatly speed this up, I have introduced parallel processing:
doveadm_worker_count = 4
Like this, doveadm -f tab quota get -A runs approx 10x faster, done in 1.5 - 2.5s
But there is one major drawback which I assume to be a bug:
sane output without doveadm_worker_count = 4 => always 6 fields
demo@example.com User quota STORAGE 29 204800 0 demo@example.com User quota MESSAGE 1 - 0
broken output with doveadm_worker_count = 4 => sometimes 7 fields
demo@example.com User quota STORAGE 29 2048 00 0 demo@example.com User quota MESSAGE 1 - 0
Somehow, the tab formatter does not play nicely together with parallel processing. On some output, the value gets split apart by an extra tab, e.g. '2024<TAB>00' in above example. Can you explain this corrupted data? Some kind of "output interleaving" seems to happen when running parallel workers.
If I switch to json output (doveadm -f json quota get -A), this does not seem to happen and I always get valid JSON output, even with doveadm_worker_count = 4.
Cheers, Philip