Opportunistic quota recalc
I'm switching quota backend to redis (from maildir++), so all my current usages are reset.
I get that I can do "doveadm quota recalc -A", but it takes forever (and auth processes time out/crash).
It would make much more sense to recalc the usage on login if the current usage is unset. Is there no way to do a recalc-quota-on-first-login?
Thanks.
-- Tom Sommer
On 06 Apr 2016, at 13:43, Tom Sommer mail@tomsommer.dk wrote:
I'm switching quota backend to redis (from maildir++), so all my current usages are reset.
I get that I can do "doveadm quota recalc -A", but it takes forever (and auth processes time out/crash).
It would make much more sense to recalc the usage on login if the current usage is unset. Is there no way to do a recalc-quota-on-first-login?
I'd actually recommend switching to "count" quota: http://wiki2.dovecot.org/Quota/Count
Then if you also want to see the current quotas in Redis, you can use http://wiki2.dovecot.org/Plugins/QuotaClone
Also, quota-dict is actually able to recalculate quota on missing keys, but specifically in Redis case its INCRBY command succeeds even if it didn't originally exist, so Dovecot doesn't detect it. And it would be pretty wasteful to do a GET every time before INCRBY.
On 2016-04-06 20:38, Timo Sirainen wrote:
On 06 Apr 2016, at 13:43, Tom Sommer mail@tomsommer.dk wrote:
I'm switching quota backend to redis (from maildir++), so all my current usages are reset.
I get that I can do "doveadm quota recalc -A", but it takes forever (and auth processes time out/crash).
It would make much more sense to recalc the usage on login if the current usage is unset. Is there no way to do a recalc-quota-on-first-login?
I'd actually recommend switching to "count" quota: http://wiki2.dovecot.org/Quota/Count
I tried count, but it crashes because of dotlocks :) There is another post about it :)
Then if you also want to see the current quotas in Redis, you can use http://wiki2.dovecot.org/Plugins/QuotaClone
Also, quota-dict is actually able to recalculate quota on missing keys, but specifically in Redis case its INCRBY command succeeds even if it didn't originally exist, so Dovecot doesn't detect it. And it would be pretty wasteful to do a GET every time before INCRBY.
Well, GET is cheap in redis, so - But I get what you are saying.
-- Tom
On 2016-04-06 21:09, Tom Sommer wrote:
On 2016-04-06 20:38, Timo Sirainen wrote:
On 06 Apr 2016, at 13:43, Tom Sommer mail@tomsommer.dk wrote:
I'm switching quota backend to redis (from maildir++), so all my current usages are reset.
I get that I can do "doveadm quota recalc -A", but it takes forever (and auth processes time out/crash).
It would make much more sense to recalc the usage on login if the current usage is unset. Is there no way to do a recalc-quota-on-first-login?
I'd actually recommend switching to "count" quota: http://wiki2.dovecot.org/Quota/Count
I tried count, but it crashes because of dotlocks :) There is another post about it :)
Any change you will have time to fix this? Because I would really love to switch to "count".
On 07 Apr 2016, at 14:49, Tom Sommer mail@tomsommer.dk wrote:
On 2016-04-06 21:09, Tom Sommer wrote:
On 06 Apr 2016, at 13:43, Tom Sommer mail@tomsommer.dk wrote:
I'm switching quota backend to redis (from maildir++), so all my current usages are reset. I get that I can do "doveadm quota recalc -A", but it takes forever (and auth processes time out/crash). It would make much more sense to recalc the usage on login if the current usage is unset. Is there no way to do a recalc-quota-on-first-login? I'd actually recommend switching to "count" quota: http://wiki2.dovecot.org/Quota/Count I tried count, but it crashes because of dotlocks :) There is another
On 2016-04-06 20:38, Timo Sirainen wrote: post about it :)
Any change you will have time to fix this? Because I would really love to switch to "count".
Looks unlikely right now, but lets see. You could also in theory just mount with -o nolock and use fcntl/flock locking so each server does the locking internally only. Having director working correctly is of course even more important in that case.
On 2016-04-07 15:02, Timo Sirainen wrote:
On 07 Apr 2016, at 14:49, Tom Sommer mail@tomsommer.dk wrote:
On 2016-04-06 21:09, Tom Sommer wrote:
On 06 Apr 2016, at 13:43, Tom Sommer mail@tomsommer.dk wrote:
I'm switching quota backend to redis (from maildir++), so all my current usages are reset. I get that I can do "doveadm quota recalc -A", but it takes forever (and auth processes time out/crash). It would make much more sense to recalc the usage on login if the current usage is unset. Is there no way to do a recalc-quota-on-first-login? I'd actually recommend switching to "count" quota: http://wiki2.dovecot.org/Quota/Count I tried count, but it crashes because of dotlocks :) There is another
On 2016-04-06 20:38, Timo Sirainen wrote: post about it :)
Any change you will have time to fix this? Because I would really love to switch to "count".
Looks unlikely right now, but lets see. You could also in theory just mount with -o nolock and use fcntl/flock locking so each server does the locking internally only. Having director working correctly is of course even more important in that case.
I tried to disable dotlocks and enable "count" with vsizes.
If you enable "quota_vsizes = yes" then this setting is stored in the index files? You cannot turn it off again without deleting dovecot.index?
So if I go back to dotlocks I'm basically breaking everything because dotlocks does not work with vsize?
On 2016-07-27 16:32, Tom Sommer wrote:
On 2016-04-07 15:02, Timo Sirainen wrote:
On 07 Apr 2016, at 14:49, Tom Sommer mail@tomsommer.dk wrote:
On 2016-04-06 21:09, Tom Sommer wrote:
On 06 Apr 2016, at 13:43, Tom Sommer mail@tomsommer.dk wrote:
I'm switching quota backend to redis (from maildir++), so all my current usages are reset. I get that I can do "doveadm quota recalc -A", but it takes forever (and auth processes time out/crash). It would make much more sense to recalc the usage on login if the current usage is unset. Is there no way to do a recalc-quota-on-first-login? I'd actually recommend switching to "count" quota: http://wiki2.dovecot.org/Quota/Count I tried count, but it crashes because of dotlocks :) There is another
On 2016-04-06 20:38, Timo Sirainen wrote: post about it :)
Any change you will have time to fix this? Because I would really love to switch to "count".
Looks unlikely right now, but lets see. You could also in theory just mount with -o nolock and use fcntl/flock locking so each server does the locking internally only. Having director working correctly is of course even more important in that case.
I tried to disable dotlocks and enable "count" with vsizes.
Oh, and since vsizes create .vsize.lock files in the Maildir folders, I now get a ton of errors:
Jul 27 14:44:19 imap(xxxx): Error: stat(/xxxxx/Maildir/.vsize.lock3a32c03afb509d28/tmp) failed: Not a directory
On 27 Jul 2016, at 17:34, Tom Sommer mail@tomsommer.dk wrote:
On 2016-07-27 16:32, Tom Sommer wrote:
On 2016-04-07 15:02, Timo Sirainen wrote:
On 2016-04-06 21:09, Tom Sommer wrote:
On 06 Apr 2016, at 13:43, Tom Sommer mail@tomsommer.dk wrote: > I'm switching quota backend to redis (from maildir++), so all my current usages are reset. > I get that I can do "doveadm quota recalc -A", but it takes forever (and auth processes time out/crash). > It would make much more sense to recalc the usage on login if the current usage is unset. Is there no way to do a recalc-quota-on-first-login? I'd actually recommend switching to "count" quota: http://wiki2.dovecot.org/Quota/Count I tried count, but it crashes because of dotlocks :) There is another
On 2016-04-06 20:38, Timo Sirainen wrote: post about it :) Any change you will have time to fix this? Because I would really love to switch to "count". Looks unlikely right now, but lets see. You could also in theory just mount with -o nolock and use fcntl/flock locking so each server does
On 07 Apr 2016, at 14:49, Tom Sommer mail@tomsommer.dk wrote: the locking internally only. Having director working correctly is of course even more important in that case. I tried to disable dotlocks and enable "count" with vsizes.
Oh, and since vsizes create .vsize.lock files in the Maildir folders, I now get a ton of errors:
Jul 27 14:44:19 imap(xxxx): Error: stat(/xxxxx/Maildir/.vsize.lock3a32c03afb509d28/tmp) failed: Not a directory
This can be avoided with maildir_stat_dirs=yes, although it makes LIST performance a bit worse. I guess another alternative would be to store index files to a separate location.
participants (2)
-
Timo Sirainen
-
Tom Sommer