[Dovecot] Quota not working with dict proxy
Hello all, I'm sure this has been covered somewhere before, but my googlefu is not up to the challenge.
Basically, I'm trying to configure quota plugin to use a dictionary service (specifically proxy with mysql) so that I can store the quota usage in a database and use that information in a lookup for postfix to reject mail if over quota. I'm doing this because postfix+avamis+dovecot setup with amavis re-injecting into postfix results in mail being effectively accepted before dovecot lmtp knows if mailbox is full.
I'm currently using quota maildir:
quota = maildir:User quota
and this works;
doveadm -Df tab quota get -u 'user@domain'
Quota name Type Value Limit % User quota STORAGE 55388 204800 27 User quota MESSAGE 4883 - 0
When I use quota dict:
quota = dict:User quota:%u:proxy::quota
I get this:
"doveadm -f tab quota get -u user@domain"
Quota name Type Value Limit % User quota STORAGE 0 204800 0 User quota MESSAGE 0 - 0
So far so good.
But manually setting database entry to values retrieved above with maildir quota results in this:
Quota name Type Value Limit % User quota STORAGE 54 204800 0 User quota MESSAGE 4883 - 0
I'm guessing this is returning values in KB, so that makes sense, I guess.
Attempting to recalc quota on one account using command:
"doveadm -f tab quota recalc -u user@domain"
returns with nothing, and when I repeat 'quota get' command, it says:
doveadm(user@domain): Fatal: User doesn't exist
Err............what?!
Indeed, looking in the database shows the account in question was actually DELETED!
Obviously, I've got something messed here, but I don't know what.
I need some guidance here.
Dovecot version is 2.1.12
Here is my config:
# 2.1.12: /etc/dovecot/dovecot.conf # OS: Linux 3.7.5-hardened-r1 x86_64 Gentoo Base System release 2.1 ext4 auth_master_user_separator = * auth_mechanisms = plain login auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@& auth_verbose_passwords = plain default_process_limit = 200 dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no login_greeting = Awaiting command... mail_location = maildir:/home/vmail/%d/%n/Maildir mail_plugins = " quota" mail_privileged_group = 100 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash 7 expire2 = Trash/* 7 expire3 = Spam 7 quota = dict:User quota:%u:proxy::quota quota_rule = *:storage=200M quota_warning = storage=99%% quota-warning 99 %n %d quota_warning2 = storage=95%% quota-warning 95 %n %d quota_warning3 = storage=80%% quota-warning 80 %n %d quota_warning4 = -storage=95%% quota-warning 'less than 95' %n %d sieve = ~/.dovecot.sieve sieve_default = /home/vmail/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_global_dir = /home/vmail/dovecot/sieve } protocols = imap pop3 sieve lmtp service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = dovecot mode = 0666 user = dovecot } user = $default_internal_user } service dict { unix_listener dict { mode = 0600 user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service quota-warning { executable = script /etc/dovecot/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } ssl_cert = </etc/ssl/dovecot/server.pem ssl_key = </etc/ssl/dovecot/server.key userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = sieve quota postmaster_address = postmaster@domain } protocol lda { mail_plugins = " quota sieve quota" } protocol imap { mail_max_userip_connections = 20 mail_plugins = " quota quota imap_quota" } protocol pop3 { mail_plugins = " quota quota" }
From dovecot-dict-sql.conf.ext:
connect = host=localhost dbname=maildb user=dbuser password=dbpass
# CREATE TABLE quota ( # username varchar(100) not null, # bytes bigint not null default 0, # messages integer not null default 0, # primary key (username) # );
map { pattern = priv/quota/storage table = email username_field = address value_field = quota_bytes } map { pattern = priv/quota/messages table = email username_field = address value_field = quota_messages }
# CREATE TABLE expires ( # username varchar(100) not null, # mailbox varchar(255) not null, # expire_stamp integer not null, # primary key (username, mailbox) # );
map { pattern = shared/expire/$user/$mailbox table = email value_field = expire_stamp
fields { address = $user maildir = $mailbox } }
Other than some guesswork, I've not got the FOGGIEST idea what the settings in the file actually mean; it doesn't seem to be documented anywhere.
Also, the quota plugin settings, at least with respect to using a proxy service, don't seem to be real explanatory either, e.g. the reason I'm using %u for the user is because I stumbled across a post suggesting that leaving it blank would result in just getting the username instead of the username@domain. This is not at all obvious from the documentation, IMO. The docs would benefit greatly from some example use cases (no, I'm not volunteering; I'm TERRIBLE with documentation ;) ).
Anyway, if someone could proffer some help or at least guidance here, I'd be very grateful.
On Sun, May 12, 2013 12:24 pm, Chris Richards wrote:
Hello all, I'm sure this has been covered somewhere before, but my googlefu is not up to the challenge.
More info; this is the debug output from the doveadm command:
doveadm -Df tab quota recalc -u 'user@domain'
doveadm(root): Debug: Loading modules from directory: /usr/lib64/dovecot doveadm(root): Debug: Module loaded: usr/lib64/dovecot/lib10_quota_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_ doveadm_fts_plugin.so: undefined symbol: fts_backend_rescan (this is usually intentional, so just ignore this message) doveadm(user@domain): Debug: auth input: user@domain home=/home/vmail/domains/domain/user/ mail=maildir:/home/vmail/domains/domain/user/Maildir/ uid=1004 gid=100 quota_rule=*:storage=200M doveadm(user@domain): Debug: Added userdb setting: mail=maildir:/home/vmail/domains/domain/user/Maildir/ doveadm(user@domain): Debug: Added userdb setting: plugin/quota_rule=*:storage=200M doveadm(user@domain): Debug: Effective uid=1004, gid=100, home=/home/vmail/domains/domain/user/ doveadm(user@domain): Debug: Quota root: name=User quota backend=dict args=user@domain:proxy::quota doveadm(user@domain): Debug: Quota rule: root=User quota mailbox=* bytes=209715200 messages=0 doveadm(user@domain): Debug: Quota warning: bytes=207618048 (99%) messages=0 reverse=no command=quota-warning 99 user domain doveadm(user@domain): Debug: Quota warning: bytes=199229440 (95%) messages=0 reverse=no command=quota-warning 95 user domain doveadm(user@domain): Debug: Quota warning: bytes=167772160 (80%) messages=0 reverse=no command=quota-warning 80 user domain doveadm(user@domain): Debug: Quota warning: bytes=199229440 (95%) messages=0 reverse=yes command=quota-warning 'less than 95' user domain doveadm(user@domain): Debug: dict quota: user=user@domain, uri=proxy::quota, noenforcing=0 doveadm(user@domain): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/vmail/domains/domain/user/Maildir/ doveadm(user@domain): Debug: maildir++: root=/home/vmail/domains/domain/user/Maildir, index=, control=, inbox=/home/vmail/domains/domain/user/Maildir, alt=
I don't see anything here that would explain why it is deleting my user account, but I can tell you with certainty that this command is doing SOMETHING that is resulting in the deletion of my account.
Chris Richards wrote:
Attempting to recalc quota on one account using command: "doveadm -f tab quota recalc -u user@domain" returns with nothing, and when I repeat 'quota get' command, it says: doveadm(user@domain): Fatal: User doesn't exist
Indeed, looking in the database shows the account in question was actually DELETED!
Obviously, I've got something messed here, but I don't know what.
dovecot-dict-sql.conf.ext:
connect = host=localhost dbname=maildb user=dbuser password=dbpass
map { pattern = priv/quota/storage table = email username_field = address value_field = quota_bytes } map { pattern = priv/quota/messages table = email username_field = address value_field = quota_messages }
Are you using the same SQL table "email" for user lookup and quota/storage accounting?
Try to use two different tables for user and quota database, because the quota accounting might have deleted an entry from the "user" table while it only tried to delete a row from the "quota" table.
Regards Daniel
On Mon, May 13, 2013 5:55 pm, Daniel Parthey wrote:
Are you using the same SQL table "email" for user lookup and quota/storage accounting?
Try to use two different tables for user and quota database, because the quota accounting might have deleted an entry from the "user" table while it only tried to delete a row from the "quota" table.
Regards Daniel
I am using the same table for both user lookkup and quota accounting. I'll try creating another table for the lookup and quota accounting, but I find it disconcerting that it would be deleting entries from the database at all when all I asked it to do was recalc the quota. One wouldn't think that 'update an entry with the correct information' would equate to 'delete this row and recreate it'.
Chris
Are you using the same SQL table "email" for user lookup and quota/storage accounting?
Try to use two different tables for user and quota database, because the quota accounting might have deleted an entry from the "user" table while it only tried to delete a row from the "quota" table.
Regards Daniel
Daniel, Per your suggestion, I created a new table, quote_usage, and changed the config files to look at it instead. Things appear to be working now. Thank you very much for your guidance.
Is there any documentation that goes into more detail regarding the 'map' settings, what they mean, etc., of which you are aware? I was rather hoping to avoid digging through the code just to satisfy my curiosity.
Thanks again.
Chris
Chris Richards wrote:
Is there any documentation that goes into more detail regarding the 'map' settings, what they mean, etc., of which you are aware? hoping to avoid digging through the code just to satisfy my curiosity.
I couldn't find much in the docs: http://wiki.dovecot.org/Quota/Dict
So here's some documentation, please correct me if I'm wrong:
map { pattern = priv/quota/storage # dictionary for storage bytes table = quota # table where to write storage count username_field = username # username of whom storage should be counted value_field = bytes # number of bytes in user mailbox }
map { pattern = priv/quota/messages # dictionary for message count table = quota # table where to write email count username_field = username # username whose emails should be counted value_field = messages # number of messages in user mailbox }
Regards Daniel
On Wed, May 15, 2013 9:15 pm, Daniel Parthey wrote:
map { pattern = priv/quota/storage # dictionary for storage bytes table = quota # table where to write storage count username_field = username # username of whom storage should be counted value_field = bytes # number of bytes in user mailbox }
map { pattern = priv/quota/messages # dictionary for message count table = quota # table where to write email count username_field = username # username whose emails should be counted value_field = messages # number of messages in user mailbox }
Regards Daniel
I think more correctly, value_field is the name of the field in the db. The 'storage' dictionary will always contain bytes,and the 'messages' dictionary will always store the number of messages into the db field named by the 'value_field' parameter.
I would guess that if you changed the pattern to 'shared/quota/messages' then you could set the shared quota as well.
participants (2)
-
Chris Richards
-
Daniel Parthey