hi,
we use dovecot 2.2.13, and we've lately started using maildir quota (previously fs quota was used, right now it's "quota = maildir:User quota" changed from "quota = fs:User quota:user:inode_per_mail").
When the change was applied we didn't manually forced generating maildirsize files. So, when the first deliver happens (or the file grows larger then 5KB) it starts quota recalc. However for users with big maildirs, receiving email every couple of seconds, there are tens of delivery processes each doing its own recalc (as they open file for APPEND and there's no locking). What's even worse, after all the stats and getdents (it takes a lot of IO for 60GB account with 2M emails), deliver makes stat on 'new' dir and detects changes after which it unlinks freshly created maildirsize. So possibly this will never be finished as every process drops its own work.
The same thing happened when we checked the native dovecot LMTP server (every connection has its own forked process - each running independently of the others).
Is it a known unwanted "feature" of maildir quota (and we should use dict quota instead) or a bug that will be fixed?
cheers, Dawid Stawiarski
On 19 Sep 2014, at 12:05, Dawid Stawiarski - nazwa.pl dawid.stawiarski@netart.pl wrote:
we use dovecot 2.2.13, and we've lately started using maildir quota (previously fs quota was used, right now it's "quota = maildir:User quota" changed from "quota = fs:User quota:user:inode_per_mail").
When the change was applied we didn't manually forced generating maildirsize files. So, when the first deliver happens (or the file grows larger then 5KB) it starts quota recalc. However for users with big maildirs, receiving email every couple of seconds, there are tens of delivery processes each doing its own recalc (as they open file for APPEND and there's no locking). What's even worse, after all the stats and getdents (it takes a lot of IO for 60GB account with 2M emails), deliver makes stat on 'new' dir and detects changes after which it unlinks freshly created maildirsize. So possibly this will never be finished as every process drops its own work.
The same thing happened when we checked the native dovecot LMTP server (every connection has its own forked process - each running independently of the others).
Is it a known unwanted "feature" of maildir quota (and we should use dict quota instead) or a bug that will be fixed?
It's a feature that won't be changed. The stat()s are especially bad if you see them - they wouldn't happen if your Maildir filenames contain the ,S=size part. Switch to dict file, which won't be recalculated unless you manually do it.
W dniu 23.09.2014 o 17:25, Timo Sirainen pisze:
On 19 Sep 2014, at 12:05, Dawid Stawiarski - nazwa.pl dawid.stawiarski@netart.pl wrote:
we use dovecot 2.2.13, and we've lately started using maildir quota (previously fs quota was used, right now it's "quota = maildir:User quota" changed from "quota = fs:User quota:user:inode_per_mail").
When the change was applied we didn't manually forced generating maildirsize files. So, when the first deliver happens (or the file grows larger then 5KB) it starts quota recalc. However for users with big maildirs, receiving email every couple of seconds, there are tens of delivery processes each doing its own recalc (as they open file for APPEND and there's no locking). What's even worse, after all the stats and getdents (it takes a lot of IO for 60GB account with 2M emails), deliver makes stat on 'new' dir and detects changes after which it unlinks freshly created maildirsize. So possibly this will never be finished as every process drops its own work.
The same thing happened when we checked the native dovecot LMTP server (every connection has its own forked process - each running independently of the others).
Is it a known unwanted "feature" of maildir quota (and we should use dict quota instead) or a bug that will be fixed?
It's a feature that won't be changed. The stat()s are especially bad if you see them - they wouldn't happen if your Maildir filenames contain the ,S=size part. Switch to dict file, which won't be recalculated unless you manually do it.
Ok, thank you kindly for clarification.
ps. Our email filenames contain the size part. The stats/getents are for each folder in Maildir (there are folders with 100-200K emails, so even listing it takes a long time - especially if there are tens of processes doing it in parallel)
participants (2)
-
Dawid Stawiarski - nazwa.pl
-
Timo Sirainen