Does domain quota work and how?
Hello,
I'm in the process of moving users from an old dovecot setup with system users to a modern virtual user setup.
I can't find good documentation on domain quota. It is mentioned in the docs as quota2 = dict:domain:%d:proxy::quota_domain but not much more. Searching the mailing list or Google doesn't reveal much more info. Searching the German mailing list at jpberlin finds mostly information where they say that it doesn't work. So, does it or does it not?
And if it works, does it work like I'm used to? e.g. with system users and aquota the filesystem quota kicks in whenever either the user or the domain quota is reached. That makes sense. It prohibits that single users can "fill" the whole domain (group) quota.
I found one particular example where they say the dict quota works different, e.g. it's an either/or. They give this example for the user_query (the relevant part):
CONCAT('*:bytes=',
IF(mailbox.quota = 0, domain.maxquota*1024000, mailbox.quota))
as quota_rule
So, this works as a default domain quota that is overridden on a user by user basis. But this means that a user quota that is set cannot honor the domain quota. So, in theory all users together can reach a maximum quota that is way over the domain quota. Not desirable.
Concluding from the example in 90-quota.conf I want to set up this user_query (relevant part):
concat('*:bytes=', m.quota) AS quota_rule, concat('*:bytes=', (s.quota*1024*1024)) AS quota2_rule
(with m and s being the two tables for user and domain quota and s holding the domain quota in MB)
Does this work together with this quota plugin setup and would provide both a user and domain quota?
quota = dict:user::proxy::quota quota2 = dict:domain:%d:proxy::quota_domain quota_rule = *:storage=100M # default quota2_rule = *:storage=1000M # default
I would like to know if this is supposed to work (and how) before I start testing in vain.
One thing that I can confirm from several statements on the mailing lists is that the recalc option of doveadm doesn't work correctly. A doveadm quota recalc -A sets the used domain quota to the user used quota that it handles last for the domain (and a recalc -u sets the whole domain used quota to the one of that user). Which sets the domain quota completely wrong. I tried iterating thru the various users but got the same result as with -A. So, this is clearly broken, at least from my point of view.
I'm using dovecot 2.3 from this repo: deb https://repo.dovecot.org/ce-2.3-latest/ubuntu/xenial xenial main
I installed it because of the native blowfish encryption support. So far, it seems to be running great. Except for this particular aspect.
Thanks!
Kai
On 16 August 2018 at 20:51 Kai Schaetzl maillists@conactive.com wrote:
Hello,
I'm in the process of moving users from an old dovecot setup with system users to a modern virtual user setup.
I can't find good documentation on domain quota. It is mentioned in the docs as quota2 = dict:domain:%d:proxy::quota_domain but not much more. Searching the mailing list or Google doesn't reveal much more info. Searching the German mailing list at jpberlin finds mostly information where they say that it doesn't work. So, does it or does it not?
And if it works, does it work like I'm used to? e.g. with system users and aquota the filesystem quota kicks in whenever either the user or the domain quota is reached. That makes sense. It prohibits that single users can "fill" the whole domain (group) quota.
I found one particular example where they say the dict quota works different, e.g. it's an either/or. They give this example for the user_query (the relevant part):
CONCAT('*:bytes=',
IF(mailbox.quota = 0, domain.maxquota*1024000, mailbox.quota))
as quota_ruleSo, this works as a default domain quota that is overridden on a user by user basis. But this means that a user quota that is set cannot honor the domain quota. So, in theory all users together can reach a maximum quota that is way over the domain quota. Not desirable.
Concluding from the example in 90-quota.conf I want to set up this user_query (relevant part):
concat('*:bytes=', m.quota) AS quota_rule, concat('*:bytes=', (s.quota*1024*1024)) AS quota2_rule
(with m and s being the two tables for user and domain quota and s holding the domain quota in MB)
Does this work together with this quota plugin setup and would provide both a user and domain quota?
quota = dict:user::proxy::quota quota2 = dict:domain:%d:proxy::quota_domain quota_rule = *:storage=100M # default quota2_rule = *:storage=1000M # default
I would like to know if this is supposed to work (and how) before I start testing in vain.
One thing that I can confirm from several statements on the mailing lists is that the recalc option of doveadm doesn't work correctly. A doveadm quota recalc -A sets the used domain quota to the user used quota that it handles last for the domain (and a recalc -u sets the whole domain used quota to the one of that user). Which sets the domain quota completely wrong. I tried iterating thru the various users but got the same result as with -A. So, this is clearly broken, at least from my point of view.
I'm using dovecot 2.3 from this repo: deb https://repo.dovecot.org/ce-2.3-latest/ubuntu/xenial xenial main
I installed it because of the native blowfish encryption support. So far, it seems to be running great. Except for this particular aspect.
Thanks!
Kai
Can you provide doveconf -n please?
Aki
Aki Tuomi wrote on Thu, 16 Aug 2018 21:03:44 +0300 (EEST):
Can you provide doveconf -n please?
Thanks for the quick reply. I think I already provided the most relevant portions of config. I ran a doveconf -n and it doesn't reveal important information like sql setup. I'll do my best to sum up the relevant parts below, anyway.
But first I would like to know how the dictionary-based domain quota is intended to work as that is mentioned nowhere. Is it like in the either/or example where they say that dovecot can only act on one of the quotas at a given time or is it intended to be used like the filesystem aquota works?
So, there are two things: First: I do not quite understand how it is supposed to work. Once I knew I could test my setup whether it acts on the limits correctly. The second is the apparent bug with recalc. I omitted to say that the updating of both quotas by dovecot when mail comes in *does* seem to work correctly. But if I run a recalc it all gets messed up. This has been reported a few times over time, so I guess it's not specific to my setup.
So, here's the relevant bits again (and a bit more):
dict { quota = mysql:/etc/dovecot/dovecot-dict-sql-user.conf quota_domain = mysql:/etc/dovecot/dovecot-dict-sql-domain.conf }
mail_home = /var/vmail/%d/%n mail_location = maildir:/var/vmail/%d/%n/mail/ mail_plugins = " quota"
plugin { quota = dict:user::proxy::quota quota2 = dict:domain:%d:proxy::quota_domain quota_rule = *:storage=100M quota_rule2 = Trash:storage=+10%% quota_rule3 = Spam:ignore quota2_rule = *:storage=1000M quota2_rule2 = Trash:storage=+10%% quota2_rule3 = Spam:ignore quota_vsizes = yes (+ quota_status, _warning etc. not relevant ...) }
service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } }
service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 10024 } }
userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql }
/etc/dovecot/dovecot-dict-sql-user.conf connect = ... map { pattern = priv/quota/storage table = quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota username_field = username value_field = messages }
/etc/dovecot/dovecot-dict-sql-domain.conf connect = map { pattern = priv/quota/storage table = quota_domain username_field = domain value_field = bytes } map { pattern = priv/quota/messages table = quota_domain username_field = domain value_field = messages }
/etc/dovecot/dovecot-sql.conf user_query = SELECT CONCAT('/var/vmail/',m.maildir) AS home, CONCAT ('maildir:/var/vmail/',m.maildir,'mail/') AS mail, 200 AS uid, 200 AS gid, concat('*:bytes=', m.quota) AS quota_rule, concat('*:bytes=', (s.ServerPopQuota*1024*1024)) AS quota2_rule FROM mailbox AS m LEFT JOIN server AS s ON (m.ServerID = s.ServerID) WHERE (m.username = '%u' OR m.local_part = "%n") AND m.active = '1'
doveadm quota get of a test domain: Username Quota name Type Value Limit % example_01@example.com user STORAGE 5567 100000 5 example_01@example.com user MESSAGE 70 - 0 example_01@example.com domain STORAGE 33426 1024000 3 example_01@example.com domain MESSAGE 1292 - 0 example_spam@example.com user STORAGE 33403 100000 33 example_spam@example.com user MESSAGE 1280 - 0 example_spam@example.com domain STORAGE 33426 1024000 3 example_spam@example.com domain MESSAGE 1292 - 0
Note: these are values after a recalc and after the users (both) received some more test mails. So, the figures in general are not correct (if I sum them up), but you see that user quota is getting updated separate for each user and that domain quota is equal for both but not equal with any of the users. Now, if I would do a recalc -A it would look like this:
example_01@example.com user STORAGE 5567 100000 5 example_01@example.com user MESSAGE 70 - 0 example_01@example.com domain STORAGE 33403 1024000 3 example_01@example.com domain MESSAGE 1280 - 0 example_spam@example.com user STORAGE 33403 100000 33 example_spam@example.com user MESSAGE 1280 - 0 example_spam@example.com domain STORAGE 33403 1024000 3 example_spam@example.com domain MESSAGE 1280 - 0
So, the last user processed determiens the domain values. Actually, the user values would be recalculated correctly after the recalc and might be different from above and the domain quota then follows that by using the last processed user as the single source for it.
Thanks!
Kai
So, is this config correct for domain quota dictionary-based and how should it work?
Kai
Kai Schaetzl wrote on Mon, 20 Aug 2018 13:32:59 +0200:
So, is this config correct for domain quota dictionary-based and how should it work?
Hello,
resolving a problem and knowing that there is a problem are two different things. I would still like to know if this config is correct and how the domain quota should work, e.g. either user quota or domain quota or are user and domain quota handled on the same level and if either one is hit the quota is reached? Only then I would be able to check if the quota handling is working like it should.
As an additional information. I noticed today that I didn't get any figures in the quota and quota_domain tables. Actually, I had noticed this earlier, but didn't find the time to troubleshoot this. But doveadm was showing me figures. As it turned out I had still set the connect string only for these two tables to the testing database while all other connect strings pointed to the correct database. I simply changed the two files and dovecot started immediately to use the different database, without a restart. The tables where empty at this stage. Dovecot recalculated the quotas and filled the tables. Good.
After some checking I can confirm that the user quotas seem to be correct, e.g. I counted the message numbers for each user of a domain and compared it with the table. That's easier than to compare sizes.
But the domain quotas are not correct. Dovecot used the quota of the *first* user to fill the domain quota. It didn't add the other users to it. Note, this is without doveadm, just happens if Dovecot encounters empty records I suppose. So, the domain quota is getting recalculated wrongly, but not (as I said earlier about doveadm) by using the last user, but the first.
Ok, then I tried a doveadm quota recalc -A again and, lo and behold, it changed the domain quota values to show the last user's quota. So, the recalc in dovecot itself uses the first user, the recalc in doveadm uses the last user. But both are wrong.
Kai
participants (2)
-
Aki Tuomi
-
Kai Schaetzl