[Dovecot] Some questions about deliver
While investigating Dovecot's deliver with Postfix, I encountered some
behaviors making me wonder whether I really understand the purpose of
that binary. So, if you allow...
This is from Postfix' main.cf:
mailbox_command = /usr/local/dovecot/libexec/dovecot/deliver -e -nThis is my quick setup for Dovecot:
# 1.2.rc4: /usr/local/etc/dovecot.conf
# OS: Darwin 9.7.0 i386
protocols: pop3
ssl: no
disable_plaintext_auth: no
login_dir: /usr/local/var/run/dovecot/login
login_executable: /usr/local/dovecot-1.2.rc4/libexec/dovecot/pop3- login first_valid_uid: 2001 mail_location: mbox:~/_mailboxes:INBOX=~/_inbox/mailspool mbox_read_locks: flock mbox_write_locks: flock dotlock mail_executable: /usr/local/dovecot-1.2.rc4/libexec/dovecot/pop3 mail_plugin_dir: /usr/local/dovecot-1.2.rc4/lib/dovecot/pop3 pop3_lock_session: yes pop3_uidl_format: %08Xv%08Xu auth default: passdb: driver: pam args: * userdb: driver: passwd
Here is an excerpt from mail.log:
postfix/smtpd[4117]: connect from localhost[127.0.0.1]
postfix/smtpd[4117]: 163E1CA2675: client=localhost[127.0.0.1]
postfix/cleanup[4120]: 163E1CA2675: message-id=<27002A3F-AF44-410D-A500-368D04F5FA9A@almbp.local >
postfix/qmgr[3232]: 163E1CA2675: from=<testuser@almbp.local>,
size=2650, nrcpt=1 (queue active)
dovecot[4129]: deliver(www): mail_location: mbox: mkdir(/Library/
WebServer/_mailboxes) failed: Permission denied (euid=70(_www)
egid=70(_www) missing +w perm: /Library/WebServer)
dovecot[4129]: deliver(www): Fatal: Namespace initialization failed
postfix/local[4122]: 163E1CA2675: to=<www@almbp.local>,
relay=local, delay=0.09, delays=0.02/0/0/0.07, dsn=4.3.0,
status=deferred (temporary failure)
postfix/smtpd[4117]: disconnect from localhost[127.0.0.1]
- Shouldn't deliver honor the first_valid_uid setting?
After all, if some user isn't allowed to login, is that user supposed
to receive mail?
- What exactly is the -e option supposed to do?
User _www clearly isn't configured for receiving mail, but Postfix
will nevertheless try to deliver the message until
maximal_queue_lifetime has expired.
From the log excerpt, it appears that Postfix has been led to
consider a temporary failure; unless I'm wrong, that's because Postfix
didn't receive neither a "D.S.N text" string nor an EX_* fatal return
code.
From deliver.c, it seems that many calls to i_fatal() are liable to
be executed before the -e option is taken into account (through the
local variable stderr_rejection).
As a result, I'm wondering under which circumstances exactly deliver
is liable to return EX_NOPERM, so that Postfix may consider a
permanent failure and bounce the message.
Note that the question may somehow be re-phrased as: when invoked
without the -e option, under which circumstances will deliver send a
rejection message?
- Doesn't failure_exit_callback() in deliver.c tend to merge many
(possibly dissimilar) errors into the single EX_TEMPFAIL one?
And BTW, is that function guaranteed to be always called with an EX_*
value? Seems to be the case, but... ;-)
TIA for your patience, Axel
On Sun, 2009-05-24 at 18:35 +0200, Axel Luttgens wrote:
- Shouldn't deliver honor the first_valid_uid setting?
I'm not sure. Somehow enforcing it there seems like a bad idea to me.
- What exactly is the -e option supposed to do? .. Note that the question may somehow be re-phrased as: when invoked
without the -e option, under which circumstances will deliver send a
rejection message?
Only when user is over quota. If you don't want messages to be delivered to some users, Postfix shouldn't call deliver for the user.
- Doesn't failure_exit_callback() in deliver.c tend to merge many
(possibly dissimilar) errors into the single EX_TEMPFAIL one?
Yes. EX_TEMPFAIL is the safest choice almost always. If something breaks, you typically want to fix it and get the mail delivered again, instead of being rejected the first time and never seeing the mail.
And BTW, is that function guaranteed to be always called with an EX_*
value? Seems to be the case, but... ;-)
What do you mean? It's called with FATAL_* values and it replaces them with EX_TEMPFAIL.
Le 26 mai 09 à 02:08, Timo Sirainen a écrit :
On Sun, 2009-05-24 at 18:35 +0200, Axel Luttgens wrote:
- Shouldn't deliver honor the first_valid_uid setting?
I'm not sure. Somehow enforcing it there seems like a bad idea to me.
Yes, could well be that I overlooked some possible side-effects.
On the other hand, this would help to enforce the deliver+Dovecot pair
consistency (if John Doe can't fetch his mails, should he receive
mails?). Or help to enforce, even if inefficiently, some local
policies not easily implemented otherwise.
Someone else? ;-)
- What exactly is the -e option supposed to do? .. Note that the question may somehow be re-phrased as: when invoked without the -e option, under which circumstances will deliver send a rejection message?
Only when user is over quota.
OK, this would thus be intended design.
If you don't want messages to be delivered to some users, Postfix shouldn't call deliver for the user.
I agree that waiting to have a message enqueued for rejecting it
afterwards is somewhat inefficient.
But... ;-)
Let's first consider deliver as a replacement of Postfix' mailbox
delivery, because of the nice benefits (indexing, headers
sanitization, plugins...) it comes with.
With Postfix' own mailbox delivery, i.e.
mailbox_command =one gets with my previous example:
postfix/smtpd[8639]: connect from localhost[127.0.0.1]
postfix/smtpd[8639]: 1AFE4CA5D97: client=localhost[127.0.0.1]
postfix/cleanup[8642]: 1AFE4CA5D97: message-id=<029EE72B-B412-437F-A211-33C3597C8C54@almbp.local >
postfix/qmgr[8637]: 1AFE4CA5D97: from=<testuser@almbp.local>,
size=560, nrcpt=1 (queue active)
postfix/local[8643]: 1AFE4CA5D97: to=<www@almbp.local>,
relay=local, delay=0.08, delays=0.01/0.01/0/0.06, dsn=5.2.0,
status=bounced (cannot update mailbox /Library/WebServer/_inbox/
mailspool for user www. unable to create lock file /Library/WebServer/
_inbox/mailspool.lock: No such file or directory)
postfix/cleanup[8642]: 2C455CA5D99: message-id=<20090526075000.2C455CA5D99@almbp.local
postfix/qmgr[8637]: 2C455CA5D99: from=<>, size=2896, nrcpt=1 (queue active)
postfix/bounce[8644]: 1AFE4CA5D97: sender non-delivery
notification: 2C455CA5D99
postfix/local[8643]: 2C455CA5D99: to=<testuser@almbp.local>,
relay=local, delay=0.01, delays=0/0/0/0, dsn=2.0.0, status=sent
(delivered to mailbox)
postfix/qmgr[8637]: 1AFE4CA5D97: removed
postfix/qmgr[8637]: 2C455CA5D99: removed
postfix/smtpd[8639]: disconnect from localhost[127.0.0.1]
So, Postfix' default behavior is to bounce the message; strictly
speaking, deliver can't thus be viewed as a transparent substitute.
Now, let's consider the default behavior of Postfix when facing an
over-quota recipient:
postfix/smtpd[8977]: connect from localhost[127.0.0.1]
postfix/smtpd[8977]: 91CEECA5FDF: client=localhost[127.0.0.1]
postfix/cleanup[8980]: 91CEECA5FDF: message-id=<B46CADD6-E42D-493C-AB04-46B0BFF3F78A@swing.be >
postfix/qmgr[8938]: 91CEECA5FDF: from=<AxelLuttgens@swing.be>,
size=12634, nrcpt=1 (queue active)
postfix/local[8981]: 91CEECA5FDF: to=<testuser@almbp.local>,
relay=local, delay=0.04, delays=0.02/0.01/0/0.01, dsn=5.2.2,
status=bounced (cannot update mailbox /Volumes/ALMbpSpare/People/a/
testuser/_inbox/mailspool for user testuser. error writing message:
File too large)
postfix/cleanup[8980]: 980EECA5FE1: message-id=<20090526085344.980EECA5FE1@almbp.fusl.ac.be
postfix/bounce[8982]: 91CEECA5FDF: sender non-delivery notification: 980EECA5FE1
postfix/qmgr[8938]: 91CEECA5FDF: removed
postfix/qmgr[8938]: 980EECA5FE1: from=<>, size=2702, nrcpt=1
(queue active)
postfix/smtp[8983]: 980EECA5FE1: to=<AxelLuttgens@swing.be>,
relay=in.mx.skynet.be[195.238.5.129]:25, delay=2.8,
delays=0.01/0.01/2.8/0.05, dsn=2.0.0, status=sent (250 ok: Message
446634039 accepted)
postfix/qmgr[8938]: 980EECA5FE1: removed
postfix/smtpd[8977]: disconnect from localhost[127.0.0.1]
In this case, by default, Posfix adopts the same behavior as deliver.
But I could have needed for some administrative reason to configure
Postfix with, for example,
soft_bounce = yesand then again face a problem when considering to make use of deliver
as mailbox transport.
More generally, since one of deliver's goal is to replace an MTA's
local delivery agent, it would be nice to have some ways to fine tune
deliver's behavior.
That would allow to transparently integrate deliver into an existing
MTA setup.
Or even to augment the capabilities of that setup, for example by
refining local policies.
At least, it would be nice to have a very precise description of how
deliver is supposed to behave when facing various conditions.
This would then be a documentation matter.
Of course, the source code always is the ultimate documentation :-)
- Doesn't failure_exit_callback() in deliver.c tend to merge many (possibly dissimilar) errors into the single EX_TEMPFAIL one?
Yes. EX_TEMPFAIL is the safest choice almost always. If something breaks, you typically want to fix it and get the mail delivered again, instead of being rejected the first time and never seeing the mail.
Yes, to be right under any circumstances is rather difficult;
ultimately, perhaps is this a matter of local policy.
Anyway, let's consider this (stupid) one:
mailbox_command = /usr/local/dovecot/libexec/dovecot/deliver -e -n -xHere, deliver immediately returns with EX_USAGE and, by default,
Postix will reject the message; but this is something I could have
noticed and fixed in the meantime.
On the other hand, it could also be argued that an over-quota
recipient is fixable as well: just pick the phone and ask the
recipient to clean his mailbox. And one could thus conclude that
deliver's default behavior isn't the right one...
Now, that last point may be controlled thru QUOTA_FULL_TEMPFAIL, so
that deliver nevertheless appears to be at least partially configurable.
Hence the temptation to ask for more configurability...
(in fact, I was musing on the possibility to modulate the switch
statement with some config settings)
And BTW, is that function guaranteed to be always called with an EX_* value? Seems to be the case, but... ;-)
What do you mean? It's called with FATAL_* values and it replaces them with EX_TEMPFAIL.
Sorry, some kind of mental short circuit occurred here; I meant:
"guaranteed to always return with status set to an EX_* value?".
I asked because of the default case in failure_exit_callback(), which
just returns and leaves status as it was on entry. Conceptually, one
could thus enter and leave the function with status set to a value
differing from one of the FATAL_* and the EX_*, that value being
ultimately returned to the caller. May this happen in practice? If
yes, when and which values?
Again, thanks a lot Timo for bearing with a guy who's just a plain
newbie with Dovecot,
Axel
On Tue, 2009-05-26 at 14:35 +0200, Axel Luttgens wrote:
postfix/local[8643]: 1AFE4CA5D97: to=<www@almbp.local>,relay=local, delay=0.08, delays=0.01/0.01/0/0.06, dsn=5.2.0,
status=bounced (cannot update mailbox /Library/WebServer/_inbox/ mailspool for user www. unable to create lock file /Library/WebServer/ _inbox/mailspool.lock: No such file or directory)
That looks like a server configuration mistake.
So, Postfix' default behavior is to bounce the message; strictly
speaking, deliver can't thus be viewed as a transparent substitute.
If I was running a mail server, I'd prefer Postfix not to bounce the message in that situation.
At least, it would be nice to have a very precise description of how
deliver is supposed to behave when facing various conditions. This would then be a documentation matter.
Maybe.. Although it can be summarized pretty easily:
- Invalid command line parameter gives EX_USAGE
- Invalid configuration gives EX_CONFIG
- User-over-quota optionally bounces or exits with EX_NOPERM
- Anything else is EX_TEMPFAIL.
Anyway, let's consider this (stupid) one:
mailbox_command = /usr/local/dovecot/libexec/dovecot/deliver -e -n -x
Here, deliver immediately returns with EX_USAGE and, by default,
Postix will reject the message; but this is something I could have
noticed and fixed in the meantime.
It could be argued that Postfix shouldn't be bouncing the message. :) EX_USAGE is meant exactly for that error, so I don't think it's a good idea for deliver to fail with EX_TEMPFAIL just to work around a Postfix issue.
On the other hand, it could also be argued that an over-quota
recipient is fixable as well: just pick the phone and ask the
recipient to clean his mailbox. And one could thus conclude that
deliver's default behavior isn't the right one...
The difference is that over-quota can be caused by a user. Configuration mistakes can't be caused by a user. Dovecot in general uses this logic in writing errors. Users can't (well, shouldn't) ever cause Dovecot to log warnings/errors.
And BTW, is that function guaranteed to be always called with an EX_* value? Seems to be the case, but... ;-)
What do you mean? It's called with FATAL_* values and it replaces them with EX_TEMPFAIL.
Sorry, some kind of mental short circuit occurred here; I meant:
"guaranteed to always return with status set to an EX_* value?".
Pretty much, yes.
I asked because of the default case in failure_exit_callback(), which
just returns and leaves status as it was on entry. Conceptually, one
could thus enter and leave the function with status set to a value
differing from one of the FATAL_* and the EX_*, that value being
ultimately returned to the caller. May this happen in practice? If
yes, when and which values?
The code looks like that to allow some future code change or plugin to exit with other values. Currently it never happens. But note that all the i_fatal_status(EX_USAGE, ..) etc. calls also go through that failure_exit_callback(). So it can't convert everything to EX_TEMPFAIL anyway, it would have to catch the used EX_* values at least.
Le 28 mai 09 à 23:51, Timo Sirainen a écrit :
On Tue, 2009-05-26 at 14:35 +0200, Axel Luttgens wrote:
[...]
That looks like a server configuration mistake.
No, it's just a Postfix system quickly and dirtily brought to life for
the sole purpose of testing deliver from within a MTA. ;-)
On that occasion, I just took Postfix' default behaviors for
illustrative purposes. But it could have been Sendmail, qmail, Exim...
as well, or even a very esoteric homegrown system: any of those
systems, especially the already operational and fine-tuned ones, have
precise expectations on how their native LDA behaves.
Replacing that LDA by another one is thus something far from benign
and to be done with care, if possible with the help of as much info as
possible about how the new LDA will behave.
Hence my investigations, then my dumb questions about deliver: how
does it behave, is that behavior configurable and if yes to what
extent, have I missed something in the docs or in the source code, and
so on.
[...]
At least, it would be nice to have a very precise description of how deliver is supposed to behave when facing various conditions. This would then be a documentation matter.
Maybe.. Although it can be summarized pretty easily:
- Invalid command line parameter gives EX_USAGE
- Invalid configuration gives EX_CONFIG
- User-over-quota optionally bounces or exits with EX_NOPERM
with a mention about the -e option and the quota_full_tempfail setting
- Anything else is EX_TEMPFAIL.
Fine!
I really believe that it would be worth to engrave that behavior
somewhere in the docs.
This could prove extremely useful to people considering to replace
their existing LDA in their existing setup, by making explicit some
points to take care of.
As a side-effect, this might also act as a reminder of the behaviors
that should not be altered in case of code rewriting.
Last but not least, no more irritating questions. ;-)
Thanks a lot, Axel
Le 30 mai 09 à 00:04, I wrote :
[...]
I really believe that it would be worth to engrave that behavior
somewhere in the docs. This could prove extremely useful to people considering to replace
their existing LDA in their existing setup, by making explicit some
points to take care of. As a side-effect, this might also act as a reminder of the behaviors
that should not be altered in case of code rewriting. Last but not least, no more irritating questions. ;-)
Just noticed that the info has been added to the wiki some days ago,
under the "Return values" heading. :-)
Many, many thanks Timo, Axel
participants (2)
-
Axel Luttgens
-
Timo Sirainen