[Dovecot] 2.2.4 - Some questions about and needing help with quota-status
Help! I'm stuck. :-(
The config of my experimental setup appears at the end of this message; I'm providing hereafter some more info that may not be immediately obvious.
This is dovecot 2.2.4 with changesets 9091d0f2d971 and 2be295a0b64f.
All involved databases are sqlite ones.
passdb and userdb are devised so as to change usernames. For example, I could have a user with addresses "doe@oldexample.com", "johnd@oldexample.com" and "john.doe@example.com" needing to be able to log in as "jdoe" or "u123456"; all db lookups for that user then end with name "john.doe".
This is a single mail user setup (user/group "dovemailer", uid/gid 999).
Service lmtp has been configured to run as that user; that required a slight adjustment at the auth-userdb socket level.
As a general rule, the quota dict appears to be correctly updated upon message arrivals and removals (thru lmtp, pop, imap), and to be correctly queried/interpreted by the various parts of the server.
For example, thru the userdb query, user john.doe has been given an even lower quota limit than the already low default defined for testings:
$ sudo doveadm quota get -u john.doe
Quota name Type Value Limit %
Quota utilisateur STORAGE 20 5 400
Quota utilisateur MESSAGE 14 - 0and is clearly recognized as being over-quota by lmtp:
$ telnet /_ROOT/var/run/dovecot/lmtp
Trying /_ROOT/var/run/dovecot/lmtp...
Connected to (null).
Escape character is '^]'.
220 almba.local Dovecot ready.
mail from:<mary.jane@example.com>
250 2.1.0 OK
rcpt to:<john.doe@example.com>
250 2.1.5 OK
data
354 OK
Subject: test
.
552 5.2.2 <john.doe@example.com> Quota exceeded (mailbox for user is full)Note that both services config and dict are launched if they aren't running.
Since it has been previously seen that running quota-status as root comes with its own problems, and since it is a single user setup anyway, I'm trying to run it as dovemailer as well.
So, let's try to see what quota-status thinks about john.doe:
$ sudo -u _postfix telnet /_ROOT/var/spool/postfix/private/quota-policyd
Trying /_ROOT/var/spool/postfix/private/quota-policyd...
Connected to (null).
Escape character is '^]'.
Connection closed by foreign host.Clearly, not much... Looking in the log:
dovecot[10554]: quota-status: Fatal: Error reading configuration: net_connect_unix(/_ROOT/var/run/dovecot/config) failed: Permission deniedThis thus raises a first question:
Q1. What makes lmtp and quota-status different? How does lmtp manage to fetch all needed info, while quota-status seems to require an access to the config socket?
Let's then slightly adjust dovecot.conf, in the hope to make quota-status happy:
service config {
unix_listener config {
group = dovemailer
mode = 0660
}
}Q2. Should the above really be needed, wouldn't there be a better way?
I ask, because it seems to me that I'm starting to seriously lose the benefits of privilege separation...
Anyway, let's ask quota-status again:
$ sudo -u _postfix telnet /_ROOT/var/spool/postfix/private/quota-policyd
Password:
Trying /_ROOT/var/spool/postfix/private/quota-policyd...
Connected to (null).
Escape character is '^]'.
recipient=john.doe@example.com
size=10000
action=OK
^]
telnet> quit
Connection closed.In the log:
dovecot[11050]: auth: Debug: userdb out: USER 1 john.doe@example.com quota_rule=*:storage=5k
dovecot[11050]: quota-status: Debug: auth input: john.doe@example.com quota_rule=*:storage=5k
dovecot[11050]: quota-status: Debug: Added userdb setting: plugin/quota_rule=*:storage=5k
dovecot[11050]: quota-status(john.doe@example.com): Debug: Effective uid=999, gid=999, home=/_Mailstores/john.doe
dovecot[11050]: quota-status(john.doe@example.com): Debug: Quota root: name=Quota utilisateur backend=dict args=john.doe:proxy::sql_quota
dovecot[11050]: quota-status(john.doe@example.com): Debug: Quota rule: root=Quota utilisateur mailbox=* bytes=5120 messages=0
dovecot[11050]: quota-status(john.doe@example.com): Debug: Quota grace: root=Quota utilisateur bytes=512 (10%)
dovecot[11050]: quota-status(john.doe@example.com): Debug: dict quota: user=john.doe, uri=proxy::sql_quota, noenforcing=0
dovecot[11050]: quota-status(john.doe@example.com): Debug: fs: root=/_Mailstores/john.doe/mboxes, index=, indexpvt=, control=, inbox=/_Mailstores/john.doe/mboxes/inbox, alt=It is to be noted that the config server is now launched as expected, but that the dict server still isn't.
Trying a dirsize backend instead of the dict backend doesn't help.
The problem seems to be circumscribed to the interaction with the quota backend since, for example, querying quota-status for an unknown recipient yields an expected "action=REJECT Unknown user".
Hence my last question, after having tried a lot of things for several days now, with no more success:
Q3. What am I doing wrong?
Many thanks in advance, Axel
# 2.2.4: /_ROOT/etc/dovecot/dovecot.conf
# OS: Darwin 12.4.0 x86_64
auth_debug = yes
dict {
sql_quota = sqlite:/_ROOT/etc/dovecot/dict-quota.conf
}
disable_plaintext_auth = no
listen = *
mail_debug = yes
mail_gid = dovemailer
mail_home = /_Mailstores/%n
mail_location = mbox:~/mboxes:INBOX=~/mboxes/inbox
mail_plugins = quota
mail_uid = dovemailer
managesieve_sieve_capability = vacation
mbox_write_locks = fcntl
passdb {
args = /_ROOT/etc/dovecot/db.conf
driver = sql
}
plugin {
quota = dict:Quota utilisateur:%n:proxy::sql_quota
quota_rule = *:storage=50M
sieve = ~/sieve/active_script.sieve
sieve_dir = ~/sieve/scripts
sieve_extensions = vacation
sieve_max_actions = 3
sieve_max_redirects = 2
sieve_max_script_size = 10K
sieve_user_log = ~/sieve/log
}
service auth {
unix_listener auth-userdb {
group = dovemailer
mode = 0660
user = $default_internal_user
}
}
service dict {
unix_listener dict {
group = dovemailer
mode = 0660
}
}
service lmtp {
group = dovemailer
user = dovemailer
}
service quota-status {
client_limit = 1
executable = quota-status -p postfix
group = dovemailer
unix_listener /_ROOT/var/spool/postfix/private/quota-policyd {
user = _postfix
}
user = dovemailer
}
ssl = no
userdb {
driver = prefetch
}
userdb {
args = /_ROOT/etc/dovecot/db.conf
driver = sql
}
protocol lmtp {
mail_plugins = quota sieve
postmaster_address = postmaster@example.com
sendmail_path = /_ROOT/postfix/sbin/sendmail
}
protocol pop3 {
pop3_uidl_format = %08Xv%08Xu
}
Hello,
I ended my previous message with :
[...] Q3. What am I doing wrong? [...]
Given the details discussed in another thread (http://www.dovecot.org/list/dovecot/2013-July/091309.html), I tried by changing the user_query's SELECT from:
user_query = \
SELECT DISTINCT \
nickname AS user, \
mail_home AS home, \
mail_location AS mail, \
quota_rule AS quota_rule \
FROM \
[...]to:
user_query = \
SELECT DISTINCT \
nickname AS user, \
coalesce(mail_home, '/_Mailstores/' || nickname) AS home, \
mail_location AS mail, \
'dict:Quota utilisateur:' || nickname || ':proxy::sql_quota' AS quota, \
quota_rule AS quota_rule \
FROM \
[...]while keeping dovecot.conf unchanged (see my previous message).
The idea is to not rely anymore on the expansion of %u (or %n or %d) in dovecot.conf, while still keeping the ability to have per-user settings.
Currently, a doveadm quota get -u john.doe yields:
Quota name Type Value Limit %
Quota utilisateur STORAGE 31 34 91
Quota utilisateur MESSAGE 23 - 0So, let's ask quota-status what it believes about a message with a size of 100000.
The reply is still "action=OK", the dict server still isn't launched, and the log shows:
auth: Debug: userdb out: USER 1 john.doe@example.com home=/_Mailstores/john.doe quota=dict:Quota utilisateur:john.doe:proxy::sql_quota quota_rule=*:storage=35000b
quota-status: Debug: auth input: john.doe@example.com home=/_Mailstores/john.doe quota=dict:Quota utilisateur:john.doe:proxy::sql_quota quota_rule=*:storage=35000b
quota-status: Debug: Added userdb setting: plugin/quota=dict:Quota utilisateur:john.doe:proxy::sql_quota
quota-status: Debug: Added userdb setting: plugin/quota_rule=*:storage=35000b
quota-status(john.doe@example.com): Debug: Effective uid=999, gid=999, home=/_Mailstores/john.doe
quota-status(john.doe@example.com): Debug: Quota root: name=Quota utilisateur backend=dict args=john.doe:proxy::sql_quota
quota-status(john.doe@example.com): Debug: Quota rule: root=Quota utilisateur mailbox=* bytes=35000 messages=0
quota-status(john.doe@example.com): Debug: Quota grace: root=Quota utilisateur bytes=3500 (10%)
quota-status(john.doe@example.com): Debug: dict quota: user=john.doe, uri=proxy::sql_quota, noenforcing=0
quota-status(john.doe@example.com): Debug: fs: root=/_Mailstores/john.doe/mboxes, index=, indexpvt=, control=, inbox=/_Mailstores/john.doe/mboxes/inbox, alt=To rule out any other side-effects potentially introduced by the user_query, I even tried with the "nickname AS user" removed from the SELECT.
Even with that, the reply is "action=OK", the dict server still isn't launched, and the lines written to the log are undistinguishable from above ones...
It is to be noted that no lines in the log are related to possible problems encountered for launching it. It is a bit as if quota_check() in src/plugins/quota/quota-status.c always immediately returned with 1 at the first test.
Anyway, I'm still stuck. And still very interested in replies to Q1, Q2 and Q3. ;-)
TIA, Axel
Le 18 juil. 2013 à 11:25, Axel Luttgens a écrit :
[...] It is to be noted that no lines in the log are related to possible problems encountered for launching [the dict server]. It is a bit as if quota_check() in src/plugins/quota/quota-status.c always immediately returned with 1 at the first test. [...]
Tracing with gdb, it appears this is indeed the case.
Here's the beginning of quota_check():
static int
quota_check(struct mail_user *user, uoff_t mail_size, const char **error_r)
{
struct quota_user *quser = QUOTA_USER_CONTEXT(user);
[...]
if (quser == NULL) {
/* no quota for user */
return 1;
}
[...]and one has for quser:
(gdb) p quser
$1 = (struct quota_user *) 0x0Yet, struct user passed as argument doesn't show obvious problems (but I have to confess the details are faaar beyond me); I reproduce it at the end of this message.
So, either my users aren't recognized as being subjected to quotas, or something goes wrong with macro QUOTA_USER_CONTEXT (which in turn translates into macro MODULE_CONTEXT which I just don't understand), or both.
As a reminder, with the same configs, "doveadm quota" and lmtp do not show such a behavior: they both take quotas into account for my users.
Any ideas?
TIA, Axel
(gdb) p *user
$2 = {
pool = 0x7fed9b829020,
v = {
deinit = 0x10b190dd0 <quota_user_deinit>
},
vlast = 0x7fed9b82a188,
refcount = 1,
username = 0x7fed9b829110 "john.doe@example.com",
_home = 0x7fed9b829e08 "/_Mailstores/john.doe",
uid = 999,
gid = 999,
service = 0x7fed9b829e30 "quota-status",
local_ip = 0x0,
remote_ip = 0x0,
auth_token = 0x0,
var_expand_table = 0x7fed9b829e40,
error = 0x0,
set_info = 0x7fed9b814d60,
unexpanded_set = 0x7fed9b829138,
set = 0x7fed9b829770,
namespaces = 0x7fed9b4046b0,
storages = 0x7fed9b404780,
hooks = {
arr = {
buffer = 0x7fed9b82a130,
element_size = 8
},
v = 0x7fed9b82a130,
v_modifiable = 0x7fed9b82a130
},
mountpoints = 0x0,
default_normalizer = 0x10b0c1d00 <uni_utf8_to_decomposed_titlecase>,
_attr_dict = 0x0,
module_contexts = {
arr = {
buffer = 0x7fed9b829da8,
element_size = 8
},
v = 0x7fed9b829da8,
v_modifiable = 0x7fed9b829da8
},
nonexistent = 0,
home_looked_up = 1,
anonymous = 0,
autocreated = 0,
initialized = 1,
mail_debug = 1,
inbox_open_error_logged = 0,
fuzzy_search = 0,
dsyncing = 0,
attr_dict_failed = 0
}On 19.7.2013, at 16.02, Axel Luttgens AxelLuttgens@swing.be wrote:
Le 18 juil. 2013 à 11:25, Axel Luttgens a écrit :
[...] It is to be noted that no lines in the log are related to possible problems encountered for launching [the dict server]. It is a bit as if quota_check() in src/plugins/quota/quota-status.c always immediately returned with 1 at the first test. [...]
Tracing with gdb, it appears this is indeed the case.
Here's the beginning of quota_check():
static int quota_check(struct mail_user *user, uoff_t mail_size, const char **error_r) { struct quota_user *quser = QUOTA_USER_CONTEXT(user); [...]
if (quser == NULL) { /* no quota for user */ return 1; } [...]and one has for quser:
(gdb) p quser $1 = (struct quota_user *) 0x0
And if you're still stuck with this, set a breakpoint to quota_mail_user_created and step through it to figure out why MODULE_CONTEXT_SET() isn't being called.
Le 1 août 2013 à 14:29, Timo Sirainen a écrit :
And if you're still stuck with this, set a breakpoint to quota_mail_user_created and step through it to figure out why MODULE_CONTEXT_SET() isn't being called.
Yes, still stuck. :-(
Did you mean function quota_mail_user_created from quota-storage.c?
If yes, could it be that it is never called in my case? Desperately trying to have the program break there, without success...
Could I try to break somewhere earlier in the call chain?
TIA, Axel
On 1.8.2013, at 19.02, Axel Luttgens AxelLuttgens@swing.be wrote:
Le 1 août 2013 à 14:29, Timo Sirainen a écrit :
And if you're still stuck with this, set a breakpoint to quota_mail_user_created and step through it to figure out why MODULE_CONTEXT_SET() isn't being called.
Yes, still stuck. :-(
Did you mean function quota_mail_user_created from quota-storage.c?
Yes.
If yes, could it be that it is never called in my case?
If not, then there's definitely some problem :)
Desperately trying to have the program break there, without success...
Could I try to break somewhere earlier in the call chain?
It should definitely stop in hook_mail_user_created, which should call quota_mail_user_created as one of the hooks. If not, the user then doesn't actually have quota plugin enabled..
Le 1 août 2013 à 18:05, Timo Sirainen a écrit :
On 1.8.2013, at 19.02, Axel Luttgens wrote:
[...] If yes, could it be that it is never called in my case?
If not, then there's definitely some problem :)
[...] Could I try to break somewhere earlier in the call chain?
It should definitely stop in hook_mail_user_created, which should call quota_mail_user_created as one of the hooks. If not, the user then doesn't actually have quota plugin enabled..
And I'm definitely not a gdb guru. :-(
Still trying to have it provide me with some enlightening info, but if I may in the meantime paraphrase one of my initial questions on this thread:
What makes doveadm-quota/lmtp and quota-status different?As a reminder:
doveadm-quota and lmtp correctly understand my quota-related settings, and over-qauota users are handled as such.
On the other hand, quota-status always returns "action=OK" for any existing user, whether over-quota or not. According to the logs, the userdb queries correctly return all needed quota-related info for the user; on the other hand, the dict service never gets launched.
It could thus be inferred that quota-status is following a slightly different path for fetching/handling quota information.
Knowing the difference could help to focus my miserable gdb investigations and/or to understand what may be at the fringe in my config.
TIA, Axel
On Fri, 2013-08-02 at 10:30 +0200, Axel Luttgens wrote:
Le 1 août 2013 à 18:05, Timo Sirainen a écrit :
On 1.8.2013, at 19.02, Axel Luttgens wrote:
[...] If yes, could it be that it is never called in my case?
If not, then there's definitely some problem :)
[...] Could I try to break somewhere earlier in the call chain?
It should definitely stop in hook_mail_user_created, which should call quota_mail_user_created as one of the hooks. If not, the user then doesn't actually have quota plugin enabled..
And I'm definitely not a gdb guru. :-(
I'd just do:
start quota-status service by e.g. connecting to it via telnet
gdb -p
pidof quota-statusb hook_mail_user_created contrecipient=user
does it stop?.. if yes, keep hitting "s" to see if it goes to quota code.
Still trying to have it provide me with some enlightening info, but if I may in the meantime paraphrase one of my initial questions on this thread:
What makes doveadm-quota/lmtp and quota-status different?
Not much..
doveadm-quota and lmtp correctly understand my quota-related settings, and over-qauota users are handled as such.
On the other hand, quota-status always returns "action=OK" for any existing user, whether over-quota or not.
I've no idea. Send your current doveconf -n and I'll see if I can reproduce the problem with it?
Le 2 août 2013 à 14:13, Timo Sirainen a écrit :
I'd just do:
start quota-status service by e.g. connecting to it via telnet
gdb -p
pidof quota-statusb hook_mail_user_created contrecipient=user
does it stop?.. if yes, keep hitting "s" to see if it goes to quota code.
To be sure, tried again, but still getting quite anarchistic behaviors, requiring some "luck" for retrieving useful info... Could be a clang vs gdb thing; I also tried to compile the quota plugin without optimization in the hope to bring some consistency back, without much success.
Still trying to have it provide me with some enlightening info, but if I may in the meantime paraphrase one of my initial questions on this thread:
What makes doveadm-quota/lmtp and quota-status different?
Not much..
This is what I was tempted to believe, until... ;-)
doveadm-quota and lmtp correctly understand my quota-related settings, and over-qauota users are handled as such.
On the other hand, quota-status always returns "action=OK" for any existing user, whether over-quota or not.
I've no idea. Send your current doveconf -n and I'll see if I can reproduce the problem with it?
Thank you for your kind proposal; it would be such a relief, should you find something I'm overlooking. I provided that info at the very beginning of the thread, but I may have changed one detail or another in the meantime; I'll thus send you my current config privately.
Best Regards, Axel
On 2.8.2013, at 16.18, Axel Luttgens AxelLuttgens@swing.be wrote:
Le 2 août 2013 à 14:13, Timo Sirainen a écrit :
I'd just do:
start quota-status service by e.g. connecting to it via telnet
gdb -p
pidof quota-statusb hook_mail_user_created contrecipient=user
does it stop?.. if yes, keep hitting "s" to see if it goes to quota code.
To be sure, tried again, but still getting quite anarchistic behaviors, requiring some "luck" for retrieving useful info... Could be a clang vs gdb thing; I also tried to compile the quota plugin without optimization in the hope to bring some consistency back, without much success.
Optimization always makes things rather annoying. Especially with clang -O2 makes it just about impossible for gdb to do anything useful. You'd probably need to disable optimization for lib-storage also, not just quota plugin.
participants (2)
-
Axel Luttgens
-
Timo Sirainen