[Dovecot] Dovecot discards mail over quota
I'm new to Dovecot, using 1.1.8 and I just wanted to verify that what I am observing is the intended behavior. I'm using Maildir++ quota. When a message is delivered to a mailbox that cannot accept the message because it would place them over the quota limit, deliver states to the MTA that the messsage was delivered successfully (example using postfix):
Jan 17 12:05:31 msa postfix/pipe[13983]: 088C5B9D47: to=<test@example.com>, relay=dovecot, delay=0.72, delays=0.34/0.02/0/0.36, dsn=2.0.0, status=sent (delivered via dovecot service)
But in actuality the mail is discarded (dropped). Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not.
deliver(test@example.com): 2009-01-17 12:05:31 Info: msgid=<1293474859.20090117120812@example.com>: save failed to INBOX: Quota exceeded (mailbox for user is full) deliver(test@example.com): 2009-01-17 12:05:31 Info: sieve runtime error: Keep: Generic Error deliver(test@example.com): 2009-01-17 12:05:31 Error: sieve_execute_bytecode(/var/sieve/global.sievec) failed deliver(test@example.com): 2009-01-17 12:05:31 Info: msgid=<1293474859.20090117120812@example.com>: rejected: Quota exceeded (mailbox for user is full) deliver(garyv@example.com): 2009-01-17 12:05:31 Info: msgid=<dovecot-1232219131-401523-0@msa>: saved mail to INBOX
-- Gary V
On Jan 17, 2009, at 2:36 PM, Gary V wrote:
Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not.
a) deliver -e
b) quota_full_tempfail=yes
c) a+b
Timo Sirainen <tss@iki.fi> wrote:
On Jan 17, 2009, at 2:36 PM, Gary V wrote:
Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not.
a) deliver -e
b) quota_full_tempfail=yes
c) a+b
Are there any plans to allow MTA check recipient over quota status?
Sendmail support simple "socket map" protocol that can be used for such queries.
It should allow to reject (not all) messages a) in replies to "RCPT TO:" for messages with right message size passed in "MAIL FROM:" b) in replies to "the final dot" for single recipient messages.
P.S. Cyrus IMAP supports "socket map" :-)
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu : anfi@xl.wp.pl I bought some used paint. It was in the shape of a house. -- Steven Wright
Andrzej Adam Filip a écrit :
Timo Sirainen <tss@iki.fi> wrote:
On Jan 17, 2009, at 2:36 PM, Gary V wrote:
Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not. a) deliver -e
b) quota_full_tempfail=yes
c) a+b
Are there any plans to allow MTA check recipient over quota status?
Sendmail support simple "socket map" protocol that can be used for such queries.
It should allow to reject (not all) messages a) in replies to "RCPT TO:" for messages with right message size passed in "MAIL FROM:" b) in replies to "the final dot" for single recipient messages.
P.S. Cyrus IMAP supports "socket map" :-)
This is a difficult problem:
- with aliases and virtual aliases, you don't know which quota to check unless you expand aliases during the smtp transaction.
- since many messages may be received at small intervals, and delivery is not atomic (mail may be passed to a spam filter, ... etc). dealing with this is not easy in the general case.
That said, a solution that "mostly works" may be good enough.
On Sat, 2009-01-17 at 21:02 +0100, Andrzej Adam Filip wrote:
Are there any plans to allow MTA check recipient over quota status?
Not really. Perhaps once LMTP server is implemented, since it pretty much requires the same functionality.
Sendmail support simple "socket map" protocol that can be used for such queries.
Wasn't the socket map also (mainly?) used for verifying if a user exists? That I have been planning on implementing some day (for last 5 years or so).
Timo Sirainen <tss@iki.fi> wrote:
On Sat, 2009-01-17 at 21:02 +0100, Andrzej Adam Filip wrote:
Are there any plans to allow MTA check recipient over quota status?
Not really. Perhaps once LMTP server is implemented, since it pretty much requires the same functionality.
Sendmail support simple "socket map" protocol that can be used for such queries.
Wasn't the socket map also (mainly?) used for verifying if a user exists? That I have been planning on implementing some day (for last 5 years or so).
AFAIR Socket map was originally intended to allow "stable sendmail code" integration of "multitude" of databases (mysql,oracle,postgress,...).
Currently it is used to ask Cyrus IMAP if user (recipient) exists.
It may be used to: 1a) ask mailbox server with virtual domain (e.g. dovecot) which domain it wants/handles at sendmail daemon startup 1b) reject messages to overquota mailboxes in reply to "RCPT TO:" (4??/5???) [Cyrus IMAP does not try to detect "this message" will cross quota] 1c) implement aliases It should allow "per virtual domain" email administrators implemented in dovecot. 1d) allow users to specify "at 'RCPT TO:'" sieve scripts
All the above (1*) requires only modifications in sendmail.cf at "sendmail side".
After modification of sendmail source code socket map may be used to pass to sendmail information about virtual users (uid,gid,$HOME).
It should allow: 2a) supporting .forward files *for virtual users* (e.g. ~/.forward.user) 2b) making sendmail execute deliver program with uid and gid of virtual user
Consider also implementing support for (simple) saslauth protocol together with socket map => it will allow sendmail (without dovecot sasl support) to use/check passwords stored by dovecot in SMTP AUTH.
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu This generation doesn't have emotional baggage. We have emotional moving vans. -- Bruce Feirstein
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, 26 Jan 2009, Andrzej Adam Filip wrote:
Wasn't the socket map also (mainly?) used for verifying if a user exists? That I have been planning on implementing some day (for last 5 years or so).
AFAIR Socket map was originally intended to allow "stable sendmail code" integration of "multitude" of databases (mysql,oracle,postgress,...).
Which is running pretty well. I'm running my virtuser and Aliases with a socket map server querying (and caching) LDAP and Postgres databases and mangle our myriads of domain aliases.
It may be used to: 1a) ask mailbox server with virtual domain (e.g. dovecot) which domain it wants/handles at sendmail daemon startup 1b) reject messages to overquota mailboxes in reply to "RCPT TO:" (4??/5???) [Cyrus IMAP does not try to detect "this message" will cross quota]
At this point I made no success. The problem I encountered is that with aliases and forwards I don't know the point when to return the failure, meaning "there could a another user resolution step to forward the mail along". E.g. ".forward" files come last, out of reach of the socket map server.
1c) implement aliases It should allow "per virtual domain" email administrators implemented in dovecot. 1d) allow users to specify "at 'RCPT TO:'" sieve scripts
?? Huh, Sieve scripts at RCPT TO phase?
After modification of sendmail source code socket map may be used to pass to sendmail information about virtual users (uid,gid,$HOME).
It should allow: 2a) supporting .forward files *for virtual users* (e.g. ~/.forward.user)
You mean by to extend the map-scheme to getpwnam() ?
2b) making sendmail execute deliver program with uid and gid of virtual user
Consider also implementing support for (simple) saslauth protocol together with socket map => it will allow sendmail (without dovecot sasl support) to use/check passwords stored by dovecot in SMTP AUTH.
Dunno, but there is already a simple saslauth protocol sendmail can use:
IN <len>username<len>password<len>service<len>realm OUT <len>code
code: OK [reason] NO [reason]
Googling for it I found a comment about its history: http://www.opensource.apple.com/darwinsource/10.3/passwordserver_sasl-14/cyr...
But as you already pointed out, one needs a getpw*(), too.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBSX7HP3WSIuGy1ktrAQLrmQf+MFSGjIIU3S5qP4sursOWDkRmIyj91DW6 qVADgtk0xaYyvWdqKEY4tezWjtBZjsOEiKZOh/RZa47oinlWgUTwDr3peIpGJOTW 5kl/LrtU635pkxXkafqNOeSszNs/GNslIOZbSG9Z6vY05I2hIc4w613UzWQe78JO Xu0PpFcGsvxw3DOHBlXZqQZ4bYAVXfLl0Rqkv9VDI6vE34i9+/T1FzGCFPLtrmCQ sRwr5Tvtn154jQanxGCaOf3CVay4UZZUvFRTvzAxw6P+etRC0fq6HesJsvBWDqOS oinS3jkjgZ1jW/lXIaHQnPx/NJWyW7YuCJ3PfVk2x9ENAn5Gxn4tNA== =L7lG -----END PGP SIGNATURE-----
Steffen Kaiser <skdovecot@smail.inf.fh-brs.de> wrote:
On Mon, 26 Jan 2009, Andrzej Adam Filip wrote:
Wasn't the socket map also (mainly?) used for verifying if a user exists? That I have been planning on implementing some day (for last 5 years or so).
AFAIR Socket map was originally intended to allow "stable sendmail code" integration of "multitude" of databases (mysql,oracle,postgress,...).
Which is running pretty well. I'm running my virtuser and Aliases with a socket map server querying (and caching) LDAP and Postgres databases and mangle our myriads of domain aliases.
It may be used to: 1a) ask mailbox server with virtual domain (e.g. dovecot) which domain it wants/handles at sendmail daemon startup 1b) reject messages to overquota mailboxes in reply to "RCPT TO:" (4??/5???) [Cyrus IMAP does not try to detect "this message" will cross quota]
At this point I made no success. The problem I encountered is that with aliases and forwards I don't know the point when to return the failure, meaning "there could a another user resolution step to forward the mail along". E.g. ".forward" files come last, out of reach of the socket map server.
You can use "post aliases/forwards" rule set (rule set localaddr=5). Make sure sendmail select mailer with F=A (aliasing) and F=5 (use rule set 5). Be warned sendmail *IGNORES* temp codes (4??) generated in this rule set.
1c) implement aliases It should allow "per virtual domain" email administrators implemented in dovecot. 1d) allow users to specify "at 'RCPT TO:'" sieve scripts
?? Huh, Sieve scripts at RCPT TO phase?
There will be no headers to check, and no body to "redirect" but IMHO it makes a perfect sense to allow "end user" *reject* messages at this point using "personalized rules".
After modification of sendmail source code socket map may be used to pass to sendmail information about virtual users (uid,gid,$HOME).
It should allow: 2a) supporting .forward files *for virtual users* (e.g. ~/.forward.user)
You mean by to extend the map-scheme to getpwnam() ?
Sendmail supports "mailbox databases" interface with currently provided getpwnam and LDAP "incarnations".
The best way wild be to code new mbdb taking user data in getpwnam format from sendmail.cf rule set [ the rule set may call socket map].
2b) making sendmail execute deliver program with uid and gid of virtual user
Consider also implementing support for (simple) saslauth protocol together with socket map => it will allow sendmail (without dovecot sasl support) to use/check passwords stored by dovecot in SMTP AUTH.
Dunno, but there is already a simple saslauth protocol sendmail can use:
IN <len>username<len>password<len>service<len>realm OUT <len>code
code: OK [reason] NO [reason]
Googling for it I found a comment about its history: http://www.opensource.apple.com/darwinsource/10.3/passwordserver_sasl-14/cyr...
But as you already pointed out, one needs a getpw*(), too.
I was thinking about allowing cyrus sasl used by sendmail to query dovecot about SMTP AUTH passwords validity using the protocol you have mentioned.
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu We fight only when there is no other choice. We prefer the ways of peaceful contact. -- Kirk, "Spectre of the Gun", stardate 4385.3
On 1/17/09, Timo Sirainen wrote:
On Jan 17, 2009, at 2:36 PM, Gary V wrote:
Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not.
a) deliver -e
b) quota_full_tempfail=yes
c) a+b
Thank you, this helped to point me to the pertinent pages in the Wiki.
-- Gary V
Timo Sirainen pisze:
On Jan 17, 2009, at 2:36 PM, Gary V wrote:
Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not.
a) deliver -e
b) quota_full_tempfail=yes
c) a+b
Yeah, this is very useful. I wish dovecot 1.0.13(which I am using :) ) has no such option. MS Exchange for example when gets "automatically rejected" message replaces it with "Your message to <> was deleted without being read" or something like that. This is very confusing for users.
Tomasz Suchodolski
On 1/17/09, Timo Sirainen wrote:
On Jan 17, 2009, at 2:36 PM, Gary V wrote:
Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not.
a) deliver -e
b) quota_full_tempfail=yes
c) a+b
Just as a matter of interest. On my Postfix system:
a) Using deliver -e, Postfix bounces the message immediately 5.7.0 -> Subject: Undelivered Mail Returned to Sender. Partial body: "<test@example.com>: permission denied. Command output: Quota exceeded (mailbox for user is full)". Postfix does not retain the message.
b) quota_full_tempfail=yes: defers the message with 4.3.0. If the user makes room for the message, then it will eventually be delivered. If they don't, then _eventually_ a bounce will be sent. In this case the bounce is less informative. Partial body: "<test@example.com>: temporary failure". In the case where the message is not delivered, using default settings in Postfix, the sender will be notified 5 days after they sent the message.
c) For over quota with a+b, it behaves the same way as b, but the bounce notice will be more informative: Partial body: "<test@example.com>: temporary failure. Command output: Quota exceeded (mailbox for user is full)".
I would say this is expected.
Each of the four possibilites has advantages and disadvantages, and personally I think a) might be closest to "doing the right thing", but it would be cool to have the option of deferring the mail (using option a+b) and additionally have deliver immediately send a message to the sender notifying them that their mail has been delayed due to the recipient being over quota. Something like:
Mail Delay Notification Your message to <user@example.com> has been delayed because the recipient's maibox is full. Your message will be retained on our server for an unspecified amount of time and a number delivery attemps may be made on your behalf. If delivery eventually fails, you will be sent a notification via email that delivery failed - or that your message was undeliverable. Note: notifications of this type are often identified as Junk Mail or Spam by mail systems! This aspect is beynd our control, so it may be in your best interest to contact the recipient by some other means and let them know you received this message.
Of course, one big problem would be to only send the message once, and not each time a delivery was attempted. No doubt there would be other issues. I'm just throwing this out there as a thought. I have no idea if would even be possible to implement something like this, or whether there is any interest or not.
-- Gary V
On Sun, 18 Jan 2009, Gary V wrote:
On 1/17/09, Timo Sirainen wrote:
On Jan 17, 2009, at 2:36 PM, Gary V wrote:
Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not.
a) deliver -e
b) quota_full_tempfail=yes
c) a+b
Just as a matter of interest. On my Postfix system:
a) Using deliver -e, Postfix bounces the message immediately 5.7.0 -> Subject: Undelivered Mail Returned to Sender. Partial body: "<test@example.com>: permission denied. Command output: Quota exceeded (mailbox for user is full)". Postfix does not retain the message.
b) quota_full_tempfail=yes: defers the message with 4.3.0. If the user makes room for the message, then it will eventually be delivered. If they don't, then _eventually_ a bounce will be sent. In this case the bounce is less informative. Partial body: "<test@example.com>: temporary failure". In the case where the message is not delivered, using default settings in Postfix, the sender will be notified 5 days after they sent the message.
c) For over quota with a+b, it behaves the same way as b, but the bounce notice will be more informative: Partial body: "<test@example.com>: temporary failure. Command output: Quota exceeded (mailbox for user is full)".
I would say this is expected.
Each of the four possibilites has advantages and disadvantages, and personally I think a) might be closest to "doing the right thing", but it would be cool to have the option of deferring the mail (using option a+b) and additionally have deliver immediately send a message to the sender notifying them that their mail has been delayed due to the recipient being over quota. Something like:
I prefer a) because it does not involve backscatter in the case of spoofed sender addresses.
-- Sahil Tandon <sahil@tandon.net>
Sahil Tandon a écrit :
On Sun, 18 Jan 2009, Gary V wrote:
On 1/17/09, Timo Sirainen wrote:
On Jan 17, 2009, at 2:36 PM, Gary V wrote:
Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not.
a) deliver -e
b) quota_full_tempfail=yes
c) a+b
Just as a matter of interest. On my Postfix system:
a) Using deliver -e, Postfix bounces the message immediately 5.7.0 -> Subject: Undelivered Mail Returned to Sender. Partial body: "<test@example.com>: permission denied. Command output: Quota exceeded (mailbox for user is full)". Postfix does not retain the message.
b) quota_full_tempfail=yes: defers the message with 4.3.0. If the user makes room for the message, then it will eventually be delivered. If they don't, then _eventually_ a bounce will be sent. In this case the bounce is less informative. Partial body: "<test@example.com>: temporary failure". In the case where the message is not delivered, using default settings in Postfix, the sender will be notified 5 days after they sent the message.
c) For over quota with a+b, it behaves the same way as b, but the bounce notice will be more informative: Partial body: "<test@example.com>: temporary failure. Command output: Quota exceeded (mailbox for user is full)".
I would say this is expected.
Each of the four possibilites has advantages and disadvantages, and personally I think a) might be closest to "doing the right thing", but it would be cool to have the option of deferring the mail (using option a+b) and additionally have deliver immediately send a message to the sender notifying them that their mail has been delayed due to the recipient being over quota. Something like:
I prefer a) because it does not involve backscatter in the case of spoofed sender addresses.
that doesn't help. postfix has already accepted and queued the message. so there be backscatter...
This brings one issue: a bounce should not be generated if the message was tagged as spam by a content filter (X-Spam-Flag, X-Bogosity, X-DSPAM-Result, ...).
On 1/18/09, Sahil Tandon wrote:
On Sun, 18 Jan 2009, Gary V wrote:
On 1/17/09, Timo Sirainen wrote:
On Jan 17, 2009, at 2:36 PM, Gary V wrote:
Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not.
a) deliver -e
b) quota_full_tempfail=yes
c) a+b
Just as a matter of interest. On my Postfix system:
a) Using deliver -e, Postfix bounces the message immediately 5.7.0 -> Subject: Undelivered Mail Returned to Sender. Partial body: "<test@example.com>: permission denied. Command output: Quota exceeded (mailbox for user is full)". Postfix does not retain the message.
b) quota_full_tempfail=yes: defers the message with 4.3.0. If the user makes room for the message, then it will eventually be delivered. If they don't, then _eventually_ a bounce will be sent. In this case the bounce is less informative. Partial body: "<test@example.com>: temporary failure". In the case where the message is not delivered, using default settings in Postfix, the sender will be notified 5 days after they sent the message.
c) For over quota with a+b, it behaves the same way as b, but the bounce notice will be more informative: Partial body: "<test@example.com>: temporary failure. Command output: Quota exceeded (mailbox for user is full)".
I would say this is expected.
Each of the four possibilites has advantages and disadvantages, and personally I think a) might be closest to "doing the right thing", but it would be cool to have the option of deferring the mail (using option a+b) and additionally have deliver immediately send a message to the sender notifying them that their mail has been delayed due to the recipient being over quota. Something like:
I prefer a) because it does not involve backscatter in the case of spoofed sender addresses.
-- Sahil Tandon <sahil@tandon.net>
Hmm, in my test, mail is not rejected during smtp conversation, so as far as I can see, there is no prevention of backscatter. Posftix creates a bounce after the fact and sends it to whomever the sender is (or at least attempts to).
As a side note: It looks like Couier/Maildrop uses the functional equivilent of a+b. At least when using Postfix, the message is deferred with a descriptive error: "temporary failure. Command output: maildrop: maildir over quota." I have not looked to see if this is configurable or not. I would assume it is, but have not checked into it.
-- Gary V
a) Using deliver -e, Postfix bounces the message immediately 5.7.0 -> Subject: Undelivered Mail Returned to Sender. Partial body: "<test@example.com>: permission denied. Command output: Quota exceeded (mailbox for user is full)". Postfix does not retain the message.
Each of the four possibilites has advantages and disadvantages, and personally I think a) might be closest to "doing the right thing"...
Assuming one chooses a) as a preference, the current default is not far from a). The only functional difference I can see (at least as far as 'over quota' is concerned) is who sends the bounce (and subsequently - what message the bounce contains). If that's the case, it's a matter of which notification the mail admin prefers.
-- Gary V
On 1/18/2009 5:47 PM, Gary V wrote:
The only functional difference I can see (at least as far as 'over quota' is concerned) is who sends the bounce (and subsequently - what message the bounce contains). If that's the case, it's a matter of which notification the mail admin prefers.
Again... the only unit responsible for sending actual bounce messages is the SENDERS MTA. Your (receiving) MTA should only either ACCEPT (if so, NEVER generate a 'bounce' later), DEFER or REJECT.
--
Best regards,
Charles
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, 19 Jan 2009, Charles Marcus wrote:
On 1/18/2009 5:47 PM, Gary V wrote:
The only functional difference I can see (at least as far as 'over quota' is concerned) is who sends the bounce (and subsequently - what message the bounce contains). If that's the case, it's a matter of which notification the mail admin prefers.
Again... the only unit responsible for sending actual bounce messages is the SENDERS MTA. Your (receiving) MTA should only either ACCEPT (if so, NEVER generate a 'bounce' later), DEFER or REJECT.
That's wrong. To "accept" means to take over the responsibility to deliver the mail and/or notify the sender about its forthcoming. A failed delivery is just a DSN as read or delivered DSNs are.
RFC2821 sec 2.1
"In either case, a formal handoff of responsibility for the message occurs: the protocol requires that a server accept responsibility for either delivering a message or properly reporting the failure to do so."
either to deliver or to report failure. Once SMTP dialogue is over, "to report failure" means sent a DSN aka bounce message.
RFC2821 sec 2.4 in context of garbled message content
"Delivery SMTP systems MAY reject ("bounce") such messages rather than deliver them." The MTA may decide to not deliver, but bounce in that case.
RFC2821 sec 3.7 about relaying explicitly states bounces, too,
RC2821 sec 4.2.5 Reply Codes After DATA and the Subsequent <CRLF>.<CRLF>
" When an SMTP server returns a positive completion status (2yz code) after the DATA command is completed with <CRLF>.<CRLF>, it accepts responsibility for:
- delivering the message (if the recipient mailbox exists), or
- if attempts to deliver the message fail due to transient
conditions, retrying delivery some reasonable number of times at
intervals as specified in section 4.5.4.
- if attempts to deliver the message fail due to permanent
conditions, or if repeated attempts to deliver the message fail
due to transient conditions, returning appropriate notification to
the sender of the original message (using the address in the SMTP
MAIL command)."
permanent failure => appropriate notification of sender
Because no MTA I'm aware of delivers during SMTP DATA phase, permanently failed delivery attempts have to generate a bounce message per RFC.
If the MTA can detect the temp or perm problem, if it will try to deliver the mail into the pysical mailbox later, fine - it can send a 4xy or 5xy response for DATA, but the lag between the detection and the actual delivery, esp. if the mail is sent to more than one recipient or an aliase / list, may result in a failed delivery attempt, although the test in DATA phase succeeded.
Actually it would be a GoodThing, if failed delivery attempts could be routed to another account, e.g. local Postmaster, if a specific condition is fullfilled, e.g. a is-SPAM tag is present.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBSXWxDHWSIuGy1ktrAQIAbggAt431hphUNLlhZn9M/kundiaqzFChjuTS LxtsOa5csFFLwbLK+wy+G6tZXMZp/mcd2N8EzAeDz3VnZ8FrpZuMw4X2CxRz86ou g1grQroWvBHAFJrMMQmjS9Nc8szWTFxo0cpjJ2nqCKs/bQ/ExDLOQd2XQxu4W0nd CAYWKpB5CcfTJSEQ9FKY0W1Nx8OE1FbT6JX7fTnDWhPthcZXR2L5i3O/cAJl9TRu rs2d7+/K4k3O8luDF+d47+uNXc5w/y2tPXcJs9AV+P4MkJkcMOPpsAeX7K54XVcm JAKXYABbQC/QFr6LNY96BkW6wsW7IRSCTUHJrCrKBqBZI9+jwLVcig== =XzBE -----END PGP SIGNATURE-----
On 1/20/2009 6:10 AM, Steffen Kaiser wrote:
Again... the only unit responsible for sending actual bounce messages is the SENDERS MTA. Your (receiving) MTA should only either ACCEPT (if so, NEVER generate a 'bounce' later), DEFER or REJECT.
That's wrong. To "accept" means to take over the responsibility to deliver the mail and/or notify the sender about its forthcoming. A failed delivery is just a DSN as read or delivered DSNs are.
Correct - this is what I said. Reject is a failed delivery, while a defer will be retried until max retires is reached.
either to deliver or to report failure. Once SMTP dialogue is over, "to report failure" means sent a DSN aka bounce message.
A DSN is NOT a bounce message - it is an smtp transaction. The actual BOUNCE message is GENERATED by the ORIGINATING server.
'Bounces' from servers other than the originating server (the server that the original sender used to send the message) are called BACKSCATTER.
RFC2821 sec 2.4 in context of garbled message content
"Delivery SMTP systems MAY reject ("bounce") such messages rather than deliver them." The MTA may decide to not deliver, but bounce in that case.
The word 'bounce' is NOT the primary word used - reject is. The word 'bounce' (in parenthesis), in my opinion, shouldn't be included here, as it is responsible for the confusion surrounding the difference between smtp transaction responses (ie reject or defer) and an actual EMAIL BONCE message, that many people confuse with it.
permanent failure => appropriate notification of sender
Correct... but the only responsibility of the receiving server is to reject the message appropriately. It is the responsibility of the originating (sending) server to inform the SENDER with the NDR/Bounce email.
Because no MTA I'm aware of delivers during SMTP DATA phase, permanently failed delivery attempts have to generate a bounce message per RFC.
But that is my point. The DESTINATION server does NOT generate THE BOUNCE - it only rejects the message with the appropriate smtp reject code. The ORIGINATING (SENDING) server generates the actual BOUNCE message (the email the sender gets informing them of the failed delivery).
--
Best regards,
Charles
Hi Steffen,
Steffen Kaiser schrieb:
On Mon, 19 Jan 2009, Charles Marcus wrote:
On 1/18/2009 5:47 PM, Gary V wrote:
The only functional difference I can see (at least as far as 'over quota' is concerned) is who sends the bounce (and subsequently - what message the bounce contains). If that's the case, it's a matter of which notification the mail admin prefers.
Again... the only unit responsible for sending actual bounce messages is the SENDERS MTA. Your (receiving) MTA should only either ACCEPT (if so, NEVER generate a 'bounce' later), DEFER or REJECT.
That's wrong. To "accept" means to take over the responsibility to deliver the mail and/or notify the sender about its forthcoming. A failed delivery is just a DSN as read or delivered DSNs are.
RFC2821 sec 2.1
"In either case, a formal handoff of responsibility for the message occurs: the protocol requires that a server accept responsibility for either delivering a message or properly reporting the failure to do so."
either to deliver or to report failure. Once SMTP dialogue is over, "to report failure" means sent a DSN aka bounce message.
RFC2821 sec 2.4 in context of garbled message content
"Delivery SMTP systems MAY reject ("bounce") such messages rather than deliver them." The MTA may decide to not deliver, but bounce in that case.
RFC2821 sec 3.7 about relaying explicitly states bounces, too,
RC2821 sec 4.2.5 Reply Codes After DATA and the Subsequent <CRLF>.<CRLF>
" When an SMTP server returns a positive completion status (2yz code) after the DATA command is completed with <CRLF>.<CRLF>, it accepts responsibility for:
delivering the message (if the recipient mailbox exists), or
if attempts to deliver the message fail due to transient conditions, retrying delivery some reasonable number of times at intervals as specified in section 4.5.4.
if attempts to deliver the message fail due to permanent conditions, or if repeated attempts to deliver the message fail due to transient conditions, returning appropriate notification to the sender of the original message (using the address in the SMTP MAIL command). "
permanent failure => appropriate notification of sender
Because no MTA I'm aware of delivers during SMTP DATA phase, permanently failed delivery attempts have to generate a bounce message per RFC.
If the MTA can detect the temp or perm problem, if it will try to deliver the mail into the pysical mailbox later, fine - it can send a 4xy or 5xy response for DATA, but the lag between the detection and the actual delivery, esp. if the mail is sent to more than one recipient or an aliase / list, may result in a failed delivery attempt, although the test in DATA phase succeeded.
Actually it would be a GoodThing, if failed delivery attempts could be routed to another account, e.g. local Postmaster, if a specific condition is fullfilled, e.g. a is-SPAM tag is present.
anyway by this rfc discussion, this feature would be a very nice to have !
Bye,
-- Steffen Kaiser
-- Best Regards
MfG Robert Schetterer
Germany/Munich/Bavaria
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tue, 20 Jan 2009, Markus Schönhaber wrote:
Just as a side note (since Charles made his point very clear - to which I completely agree BTW): RFC 2821 is obsoleted by RFC 5321.
RFC5321 sec 4.2.4
the same wording about:
o if attempts to deliver the message fail due to permanent
conditions, or if repeated attempts to deliver the message fail
due to transient conditions, returning appropriate notification to
the sender of the original message (using the address in the SMTP
MAIL command).RFC3461 sec 5.2.6 Failure of a conforming MTA to deliver a message (referenced by both 2821 and 5321)
" (c) If no NOTIFY parameter was supplied for the recipient, a "failed" DSN MUST be issued. "
RFC3462 sec 1 (referenced by both 2821 and 5321)
" 1) [Required] The first body part contains human readable message. The purpose of this message is to provide an easily understood description of the condition(s) that caused the report to be generated, for a human reader who may not have a user agent capable of interpreting the second section of the Multipart/Report. "
And this sentence backs my argumentation, that a DSN should be in English (and probably in another second language), because an error message in human language XYZ only is not "easily understood". :)
Which comes down to Timo's original question, in order to remove the notification of Dovecot's deliver in favor of returning the proper exit code to let the MTA handles the stuff, seems to be good.
In order to handle Charles's idea, Dovecot deliver is to return "No error" and send no DSN in case of failure.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBSXXkz3WSIuGy1ktrAQJWrQgAvvTqxgmWzV8c3S8wL2IoCIQZH2eBj5nK 0H6hkSU0b9GEiJIw89CvVBHqprbC9+ALQLnMo+ys/1ER49SVbWtgrJDvp5spGLis xiaSuWcIjKQFuSCWYpF9K/QmgrtSGYiGrHm1OURM+0J5YdscKPdkNzIakViIYtHQ /pjTR/aD87sgH+CjBKj1suFX2A6ynUTOm1rRdIoz82KRqB+cIInpWrzNCgTIvgph 5tKjICEPbV+ap6MsUiI0McCCbyGGIzKDSS/KKcxdN3/kwF4BUg8R/11aFm4yi0av yTUYn5SICKBXK5bRtFu0LtxpzdNfpCORf6Ad0X9MR5qBwGD38kEdLQ== =rATz -----END PGP SIGNATURE-----
Steffen Kaiser:
RFC5321 sec 4.2.4
the same wording about:
o if attempts to deliver the message fail due to permanent conditions, or if repeated attempts to deliver the message fail due to transient conditions, returning appropriate notification to the sender of the original message (using the address in the SMTP MAIL command).
As I see it, no-one in this thread doubts that bounces are spec compliant.
Charles' point - as I understand it and to which I agreed - is the question *who* should generate a bounce. IMO the "receiving" MTA *must not* generate a bounce. The reason for this is not that any part of any RFC I know of does say so, but because of the simple fact that the receiving MTA very likely is *not able* to send a useful bounce since chances are that the sender address (i. e. the address the bounce should be sent to) is faked anyway. Accepting a mail and generating a bounce later will eventually make you a source of backscatter and get you blacklisted. The exception to this rule obviously is the case when you relay mail for your users (which is a different story anyway since you are the "sending" and not the "receiving" MTA).
Which comes down to Timo's original question, in order to remove the notification of Dovecot's deliver in favor of returning the proper exit code to let the MTA handles the stuff, seems to be good.
...if the MTA can find out about deliver's exit status before the SMTP transaction is completed...
Regards mks
It's simply a fact these days that people find backscatter and misdirected bounces annoying. In many cases they end up at a forged address that had nothing to do with the original transaction. A well behaved receiver will make decisions during the SMTP transactions with appropriate response codes. It should never accept and generate a message later.
It's not a matter of mincing what an RFC says or does not say, it's about not being a dick on the internet and respecting your virtual neighbors. It's like if you were to go through your postal mail box, separate the junk mail and throw it into your neighbor's yard.
~Seth
On 1/20/2009, Seth Mattinen (sethm@rollernet.us) wrote:
It's simply a fact these days that people find backscatter and misdirected bounces annoying. In many cases they end up at a forged address that had nothing to do with the original transaction. A well behaved receiver will make decisions during the SMTP transactions with appropriate response codes. It should never accept and generate a message later.
Well, its also about the fact that if one engages in such behavior, one will quickly find one's mailserver/IP address on numerous blacklists that are a pain to get off of, IF you can get off of them.
--
Best regards,
Charles
Charles Marcus wrote:
On 1/20/2009, Seth Mattinen (sethm@rollernet.us) wrote:
It's simply a fact these days that people find backscatter and misdirected bounces annoying. In many cases they end up at a forged address that had nothing to do with the original transaction. A well behaved receiver will make decisions during the SMTP transactions with appropriate response codes. It should never accept and generate a message later.
Well, its also about the fact that if one engages in such behavior, one will quickly find one's mailserver/IP address on numerous blacklists that are a pain to get off of, IF you can get off of them.
Merely a reinforcement by the people who find backscatter unacceptable directed at people who can't take a hint. ;)
~Seth
On 1/20/2009, Steffen Kaiser (skdovecot@smail.inf.fh-brs.de) wrote:
In order to handle Charles's idea, Dovecot deliver is to return "No error" and send no DSN in case of failure.
? Not sure where you got that idea...
If Timos idea was in fact to 'pass-thru' these status codes, I'm all in favor of it...
My point was the distinction between a BOUNCE message, and an smtp REJECT.
--
Best regards,
Charles
On Sun, 18 Jan 2009, Gary V wrote:
On 1/18/09, Sahil Tandon wrote:
On Sun, 18 Jan 2009, Gary V wrote:
On 1/17/09, Timo Sirainen wrote:
On Jan 17, 2009, at 2:36 PM, Gary V wrote:
Then a bounce is created stating the mail was rejected:
Your message to <test@example.com> was automatically rejected: Quota exceeded (mailbox for user is full).
Question: is it possible (without changing code) to alter this to where deliver would instead tempfail or something. Somehow it seems wrong to me to tell the MTA that everything is good, and then silently discard messages - regardless of the fact dovecot creates a bounce. This is not necessarily ideal either, but I _am_ wondering if this is configurable or not.
a) deliver -e
b) quota_full_tempfail=yes
c) a+b
Just as a matter of interest. On my Postfix system:
a) Using deliver -e, Postfix bounces the message immediately 5.7.0 -> Subject: Undelivered Mail Returned to Sender. Partial body: "<test@example.com>: permission denied. Command output: Quota exceeded (mailbox for user is full)". Postfix does not retain the message.
b) quota_full_tempfail=yes: defers the message with 4.3.0. If the user makes room for the message, then it will eventually be delivered. If they don't, then _eventually_ a bounce will be sent. In this case the bounce is less informative. Partial body: "<test@example.com>: temporary failure". In the case where the message is not delivered, using default settings in Postfix, the sender will be notified 5 days after they sent the message.
c) For over quota with a+b, it behaves the same way as b, but the bounce notice will be more informative: Partial body: "<test@example.com>: temporary failure. Command output: Quota exceeded (mailbox for user is full)".
I would say this is expected.
Each of the four possibilites has advantages and disadvantages, and personally I think a) might be closest to "doing the right thing", but it would be cool to have the option of deferring the mail (using option a+b) and additionally have deliver immediately send a message to the sender notifying them that their mail has been delayed due to the recipient being over quota. Something like:
I prefer a) because it does not involve backscatter in the case of spoofed sender addresses.
-- Sahil Tandon <sahil@tandon.net>
Hmm, in my test, mail is not rejected during smtp conversation, so as far as I can see, there is no prevention of backscatter. Posftix creates a bounce after the fact and sends it to whomever the sender is (or at least attempts to).
Ah, I construed your report to mean Postfix _did_ reject at SMTP; my bad interpretation! Right now we have it working with tempfail and users generally free up their mailbox so an actual bounce (backscatter) is yet to ocurr.
-- Sahil Tandon <sahil@tandon.net>
On Sun, Jan 18, 2009 at 12:26:46PM -0700, Gary V wrote:
Just as a matter of interest. On my Postfix system:
a) Using deliver -e, Postfix bounces the message immediately 5.7.0 -> Subject: Undelivered Mail Returned to Sender. Partial body: "<test@example.com>: permission denied. Command output: Quota exceeded (mailbox for user is full)". Postfix does not retain the message.
b) quota_full_tempfail=yes: defers the message with 4.3.0. If the user makes room for the message, then it will eventually be delivered. If they don't, then _eventually_ a bounce will be sent. In this case the bounce is less informative. Partial body: "<test@example.com>: temporary failure". In the case where the message is not delivered, using default settings in Postfix, the sender will be notified 5 days after they sent the message.
c) For over quota with a+b, it behaves the same way as b, but the bounce notice will be more informative: Partial body: "<test@example.com>: temporary failure. Command output: Quota exceeded (mailbox for user is full)".
I would say this is expected.
You can also quota_exceeded_message to something like "4.2.2 Mailbox full" (or "5.2.2 ..."). This works whenever -e is specified, i.e. for a) and c), and you use Postfix 2.3 or later (see pipe(8)).
Each of the four possibilites has advantages and disadvantages, and personally I think a) might be closest to "doing the right thing", but it would be cool to have the option of deferring the mail (using option a+b) and additionally have deliver immediately send a message to the sender notifying them that their mail has been delayed due to the recipient being over quota. Something like:
This is also easy with Postfix, if you use delay_warning_time (see postconf(5)).
On 1/19/09, Ulrich Zehl wrote:
On Sun, Jan 18, 2009 at 12:26:46PM -0700, Gary V wrote:
Just as a matter of interest. On my Postfix system:
a) Using deliver -e, Postfix bounces the message immediately 5.7.0 -> Subject: Undelivered Mail Returned to Sender. Partial body: "<test@example.com>: permission denied. Command output: Quota exceeded (mailbox for user is full)". Postfix does not retain the message.
b) quota_full_tempfail=yes: defers the message with 4.3.0. If the user makes room for the message, then it will eventually be delivered. If they don't, then _eventually_ a bounce will be sent. In this case the bounce is less informative. Partial body: "<test@example.com>: temporary failure". In the case where the message is not delivered, using default settings in Postfix, the sender will be notified 5 days after they sent the message.
c) For over quota with a+b, it behaves the same way as b, but the bounce notice will be more informative: Partial body: "<test@example.com>: temporary failure. Command output: Quota exceeded (mailbox for user is full)".
I would say this is expected.
You can also quota_exceeded_message to something like "4.2.2 Mailbox full" (or "5.2.2 ..."). This works whenever -e is specified, i.e. for a) and c), and you use Postfix 2.3 or later (see pipe(8)).
Each of the four possibilites has advantages and disadvantages, and personally I think a) might be closest to "doing the right thing", but it would be cool to have the option of deferring the mail (using option a+b) and additionally have deliver immediately send a message to the sender notifying them that their mail has been delayed due to the recipient being over quota. Something like:
This is also easy with Postfix, if you use delay_warning_time (see postconf(5)).
Right, thank you so much. I thought there was something like delay_warning_time that could make c) a workable choice. In fact, now I remember setting that up on my production server (when I had one a year ago) so senders would get notified their mail was still queued:
This is the mail system at host msa.example.com.
#################################################################### # THIS IS A WARNING ONLY. YOU DO NOT NEED TO RESEND YOUR MESSAGE. # ####################################################################
Your message could not be delivered for more than 1 hour(s). It will be retried until it is 3 day(s) old.
For further assistance, please send mail to postmaster.
If you do so, please include this problem report. You can delete your own text from the attached returned message.
The mail system<test@example.com>: temporary failure. Command output: Quota exceeded (mailbox for user is full)
-- Gary V
Right, thank you so much. I thought there was something like delay_warning_time that could make c) a workable choice. In fact, now I remember setting that up on my production server (when I had one a year ago) so senders would get notified their mail was still queued:
But then this creates an additional bounce message and Charles makes a valid point:
Again... the only unit responsible for sending actual bounce messages is the SENDERS MTA. Your (receiving) MTA should only either ACCEPT (if so, NEVER generate a 'bounce' later), DEFER or REJECT.
However, I remember setting delay_warning_time for the benefit of local domain senders, so they would get notification of delayed delivery. If this is added, bounces to non-local domain sender would be a side effect of deferring over-quota messages.
At any rate, thanks. I now have a good picture of what all the options are.
-- Gary V
participants (12)
-
Andrzej Adam Filip
-
Charles Marcus
-
Gary V
-
Markus Schönhaber
-
mouss
-
Robert Schetterer
-
Sahil Tandon
-
Seth Mattinen
-
Steffen Kaiser
-
Timo Sirainen
-
Tomasz Suchodolski
-
Ulrich Zehl