[Dovecot] Postfix aliases with quota-status service
The latest HG commits seem to have fixed some underlying problems with the 'quota-status' service. Now doing some quick tests I wonder if this can be used with aliases on the Postfix side. Appears the 'check_policy_service' used in the example below will query existing users via Dovecot's Auth Backend:
http://sys4.de/en/blog/2013/04/05/dovecot-quota-mit-postfix-abfragen/ http://www.postfix.org/SMTPD_POLICY_README.html
This works fine for Dovecot user 'philo' where the mail is rejected upfront before it enters the queue:
==> /var/log/mail.log <== Jul 30 13:41:18 spectre postfix/smtpd[31072]: NOQUEUE: reject: RCPT from host.example.com[1.2.3.4]: 554 5.2.2 <philo@trashheap.net>: Recipient address rejected: Quota exceeded (mailbox for use r is full); from=<user@example.com> to=<philo@trashheap.net> proto=ESMTP helo=<host.example.com>
Now user 'gunge' is an alias for 'philo' in Postfix, thus the quota check will not recognize the user, will queue the message and reject it at the MDA stage:
==> /var/log/mail.log <== Jul 30 14:41:02 spectre postfix/lmtp[31460]: 3c4HSD5ZkBzBP: to=<philo@trashheap.net>, orig_to=<gunge@trashheap.net>, relay=spectre.leuxner.net[private/dovecot-lmtp], delay=1.2, delays=1.1/0.01/0/0.13, dsn=4.2.2, status=deferred (host spectre.leuxner.net[private/dovecot-lmtp] said: 452 4.2.2 <philo@trashheap.net> Quota exceeded (mailbox for user is full) (in reply to end of DATA command))
This is probably intended behaviour, just want to make sure that I'm not missing a point here. For now the only fix that comes to my mind to create "quota aware" aliases - is creating 'dummy' users in Dovecot which point to the same mailbox rather than performing aliasing on the Postfix end. Open to suggestions...
Regards Thomas
On Tue, Jul 30, 2013 at 03:20:47PM +0200, Thomas Leuxner wrote:
This is probably intended behaviour, just want to make sure that I'm not missing a point here. For now the only fix that comes to my mind to create "quota aware" aliases - is creating 'dummy' users in Dovecot which point to the same mailbox rather than performing aliasing on the Postfix end. Open to suggestions...
It depends on your user and/or alias database (i.e. how and where they're stored).
I use LDAP, and configured Dovecot's userdb lookup to handle both "main" and "alias" addresses, like so:
user_filter = (|(mail=%u)(mailAlternateAddress=%u))
(As far as Dovecot is concerned, there's actually no difference between "main" and "alias".)
If you store your mailbox and alias information in the same data source (LDAP, SQL, ...), you should be able to do the same.
- Ulrich Zehl <ulrich-dovecot@topfen.net> 2013.08.01 16:39:
If you store your mailbox and alias information in the same data source (LDAP, SQL, ...), you should be able to do the same.
Thanks. I did address this using a restriction class which works fine for my scenario and allows selective quota checking.
/etc/postfix/main.cf: smtpd_restriction_classes = quota_users quota_users = check_policy_service unix:private/quota-status
smtpd_recipient_restrictions = ... reject_unverified_recipient, check_recipient_access hash:/etc/postfix/quota_users
/etc/postfix/quota_users: someone@example.com quota_users ...
Now the real problem along the road is the submitting server. If that server does not indicate the message size during handshake the pre-queue rejection simply can not work.
MAIL FROM:<someone@example.com> SIZE=2924764
Google for instance is not doing this...
Thomas
On 6.8.2013, at 18.49, Thomas Leuxner <tlx@leuxner.net> wrote:
Now the real problem along the road is the submitting server. If that server does not indicate the message size during handshake the pre-queue rejection simply can not work.
quota_grace was meant to solve that. You'll allow the user to become a bit over quota.
- Timo Sirainen <tss@iki.fi> 2013.08.06 18:15:
Now the real problem along the road is the submitting server. If that server does not indicate the message size during handshake the pre-queue rejection simply can not work.
quota_grace was meant to solve that. You'll allow the user to become a bit over quota.
What I meant is before the mail enters the Postfix queues. If the SIZE extension is not used during MAIL FROM by the remote server, then there's no way to reject an over-quota mail upfront, losing the benefit of the policy service.
On 6.8.2013, at 19.25, Thomas Leuxner <tlx@leuxner.net> wrote:
- Timo Sirainen <tss@iki.fi> 2013.08.06 18:15:
Now the real problem along the road is the submitting server. If that server does not indicate the message size during handshake the pre-queue rejection simply can not work.
quota_grace was meant to solve that. You'll allow the user to become a bit over quota.
What I meant is before the mail enters the Postfix queues. If the SIZE extension is not used during MAIL FROM by the remote server, then there's no way to reject an over-quota mail upfront, losing the benefit of the policy service.
The idea behind quota_grace is that the last mail would be allowed to take the user somewhat over quota (e.g. up to 109% quota usage). On the next mail delivery user is already over quota, so the size of the mail is irrelevant because a mail of any size will be rejected. The initial quota-status implementation didn't even support SIZE extension since I didn't remember it existed.
- Timo Sirainen <tss@iki.fi> 2013.08.06 19:42:
The idea behind quota_grace is that the last mail would be allowed to take the user somewhat over quota (e.g. up to 109% quota usage). On the next mail delivery user is already over quota, so the size of the mail is irrelevant because a mail of any size will be rejected. The initial quota-status implementation didn't even support SIZE extension since I didn't remember it existed.
I'm referring to the Postfix side _only_ or the initial SMTP Handshake if you like. My point is that there is no safe way to reject mails at this level *if* the remote server doesn't play nice. I think this was the whole point of writing a policy service for Postfix. I'm not *talking* about quotas that will be handled by the delivery agents...
On 6.8.2013, at 20.57, Thomas Leuxner <tlx@leuxner.net> wrote:
- Timo Sirainen <tss@iki.fi> 2013.08.06 19:42:
The idea behind quota_grace is that the last mail would be allowed to take the user somewhat over quota (e.g. up to 109% quota usage). On the next mail delivery user is already over quota, so the size of the mail is irrelevant because a mail of any size will be rejected. The initial quota-status implementation didn't even support SIZE extension since I didn't remember it existed.
I'm referring to the Postfix side _only_ or the initial SMTP Handshake if you like. My point is that there is no safe way to reject mails at this level *if* the remote server doesn't play nice. I think this was the whole point of writing a policy service for Postfix. I'm not *talking* about quotas that will be handled by the delivery agents...
Either you're still misunderstanding me, or vice versa. The quota rejections can be done complete in SMTP side even without SIZE:
- quota at 99% :
MAIL FROM:<sender@example.com> 250 2.1.0 Ok RCPT TO:<tss@dovecot.org> 250 2.1.0 Ok DATA ... . 250 2.0.0 Ok: queued as 12345
- quota is now at 103% :
MAIL FROM:<sender2@example.com> 250 2.1.0 Ok RCPT TO:<tss@dovecot.org> 554 5.2.2 User is over quota
On Tue, Aug 06, 2013 at 09:27:20PM +0300, Timo Sirainen wrote:
On 6.8.2013, at 20.57, Thomas Leuxner <tlx@leuxner.net> wrote:
- Timo Sirainen <tss@iki.fi> 2013.08.06 19:42:
The idea behind quota_grace is that the last mail would be allowed to take the user somewhat over quota (e.g. up to 109% quota usage). On the next mail delivery user is already over quota, so the size of the mail is irrelevant because a mail of any size will be rejected. The initial quota-status implementation didn't even support SIZE extension since I didn't remember it existed.
I'm referring to the Postfix side _only_ or the initial SMTP Handshake if you like. My point is that there is no safe way to reject mails at this level *if* the remote server doesn't play nice. I think this was the whole point of writing a policy service for Postfix. I'm not *talking* about quotas that will be handled by the delivery agents...
Either you're still misunderstanding me, or vice versa. The quota rejections can be done complete in SMTP side even without SIZE:
Another way, in Postfix, is to wait for end-of-DATA. Regardless of SIZE being given, at that point, the actual size is known.
Of course as Thomas would probably point out, such a rejection is unsafe, because ANY overquota recipient would cause rejection for EVERY recipient; SMTP cannot have per-recipient results except at "RCPT TO:".
Personally, I'd much rather allow the last overquota mail, even in cases where the user goes far over the quota. Apparently Thomas intends to have a solid, inflexible quota.
In that case I'd suggest going for a lower quota and adding quota_grace. Let quota_grace plus quota be the most you can tolerate in your users' mailboxes.
- quota at 99% :
MAIL FROM:<sender@example.com> 250 2.1.0 Ok RCPT TO:<tss@dovecot.org> 250 2.1.0 Ok DATA ... . 250 2.0.0 Ok: queued as 12345
- quota is now at 103% :
MAIL FROM:<sender2@example.com> 250 2.1.0 Ok RCPT TO:<tss@dovecot.org> 554 5.2.2 User is over quota
-- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:
On 2013-08-06 2:49 PM, /dev/rob0 <rob0@gmx.co.uk> wrote:
Another way, in Postfix, is to wait for end-of-DATA. Regardless of SIZE being given, at that point, the actual size is known.
Of course as Thomas would probably point out, such a rejection is unsafe, because ANY overquota recipient would cause rejection for EVERY recipient; SMTP cannot have per-recipient results except at "RCPT TO:".
But LMTP can... right?
Personally, I'd much rather allow the last overquota mail, even in cases where the user goes far over the quota.
I agree - but it could never go over more than the max mail size allowed so that shouldn't be a problem - unless you don't set a max mail size, in which case you're nuts... :)
- /dev/rob0 <rob0@gmx.co.uk> 2013.08.06 20:49:
Personally, I'd much rather allow the last overquota mail, even in cases where the user goes far over the quota. Apparently Thomas intends to have a solid, inflexible quota.
The point I'm trying to make is mail being queued by Postfix because it has no means to validate the mail would take the user over quota. In the scenarios I tested with SIZE being part of MAIL FROM the mail gets rejected at SMTPD stage, while without SIZE supplied it will get queued and eventually be rejected by the MDA.
AFAIK the whole endeavour was undertaken to avoid queue injection of mails knowing they would bounce. This seems to work when the SMTPD receives enough detail.
As to Timo's example:
This also seems to work given the quota is *over* the limit incl. grace:
$ doveadm quota get -u philo@trashheap.net Quota name Type Value Limit % user STORAGE 10914 10240 106 user MESSAGE 5 - 0
Aug 6 20:56:31 spectre postfix/smtpd[27201]: connect from mail-oa0-f44.google.com[209.85.219.44] Aug 6 20:56:32 spectre postfix/smtpd[27201]: Anonymous TLS connection established from mail-oa0-f44.google.com[209.85.219.44]: TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits) Aug 6 20:56:32 spectre postfix/smtpd[27201]: NOQUEUE: reject: RCPT from mail-oa0-f44.google.com[209.85.219.44]: 554 5.2.2 <philo@trashheap.net>: Recipient address rejected: Quota exceeded (mailbox for user is full); from=<user@googlemail.com> to=<philo@trashheap.net> proto=ESMTP helo=<mail-oa0-f44.google.com>
Now everything in between seems to create SMTPD rejections in some cases _or_ queue the mail and let it hit the quota in other cases. That's my whole point...
On Tue, Aug 06, 2013 at 09:20:13PM +0200, Thomas Leuxner wrote:
Now everything in between seems to create SMTPD rejections in some cases _or_ queue the mail and let it hit the quota in other cases. That's my whole point...
I'm sorry, I don't get your point.
Are you saying that quota-status does not eliminate all over-quota bounces?
That's to be expected. quota-status does not reserve quota when questioned, it only tells Postfix whether enough space is currently free or not. This is not free of race conditions, and therefore cannot eliminate all late bounces, even when quota-status has all the information, including size.
For example, if a mailbox has 3000 bytes of free space, and two 2000 byte messages arrive close enough together so that the first one has not been delivered by the time the second one sends "RCPT TO ... SIZE=...", quota-status will allow both messages, even though the second one will be rejected when Postfix actually tries local delivery (LTMP/LDA).
Are you saying that rejects depend on SIZE= being sent during the RCPT TO stage (i.e., messages that announce their size correctly are rejected during the SMTP transaction, while those without size inidcation are passed)?
If so, follow Rob's suggestion, and run quota-status (again) as part of smtpd_end_of_data_restrictions. By then, Postfix will know the size of a message and pass it to quota-status. (This only works for single-recipient messages; multi-recipient messages will always pass at this stage, because Postfix' policy protocol will not send "recipient=..." in this case, and thus quota-status replies with "DUNNO". In my environment, most messages are single-recipient, so it works well enough for me.)
Are you saying something else that I missed? Please tell me, because I'd like to understand your point.
- Ulrich Zehl <ulrich-dovecot@topfen.net> 2013.08.07 08:29:
Are you saying that rejects depend on SIZE= being sent during the RCPT TO stage (i.e., messages that announce their size correctly are rejected during the SMTP transaction, while those without size inidcation are passed)?
Yes.
On 7.8.2013, at 9.29, Ulrich Zehl <ulrich-dovecot@topfen.net> wrote:
On Tue, Aug 06, 2013 at 09:20:13PM +0200, Thomas Leuxner wrote:
Now everything in between seems to create SMTPD rejections in some cases _or_ queue the mail and let it hit the quota in other cases. That's my whole point...
I'm sorry, I don't get your point.
Are you saying that quota-status does not eliminate all over-quota bounces?
That's to be expected. quota-status does not reserve quota when questioned, it only tells Postfix whether enough space is currently free or not. This is not free of race conditions, and therefore cannot eliminate all late bounces, even when quota-status has all the information, including size.
I think it would work also to do the check in SMTP RCPT TO stage and do quota bouncing only there. In LMTP/LDA stage use infinite quota to avoid bouncing. Yeah, user might get a little bit over quota (even over quota_grace) but not by much.
Am 06.08.2013 20:27, schrieb Timo Sirainen:
On 6.8.2013, at 20.57, Thomas Leuxner <tlx@leuxner.net> wrote:
- Timo Sirainen <tss@iki.fi> 2013.08.06 19:42:
The idea behind quota_grace is that the last mail would be allowed to take the user somewhat over quota (e.g. up to 109% quota usage). On the next mail delivery user is already over quota, so the size of the mail is irrelevant because a mail of any size will be rejected. The initial quota-status implementation didn't even support SIZE extension since I didn't remember it existed.
I'm referring to the Postfix side _only_ or the initial SMTP Handshake if you like. My point is that there is no safe way to reject mails at this level *if* the remote server doesn't play nice. I think this was the whole point of writing a policy service for Postfix. I'm not *talking* about quotas that will be handled by the delivery agents...
Either you're still misunderstanding me, or vice versa. The quota rejections can be done complete in SMTP side even without SIZE:
- quota at 99% :
MAIL FROM:<sender@example.com> 250 2.1.0 Ok RCPT TO:<tss@dovecot.org> 250 2.1.0 Ok DATA ... . 250 2.0.0 Ok: queued as 12345
- quota is now at 103% :
MAIL FROM:<sender2@example.com> 250 2.1.0 Ok RCPT TO:<tss@dovecot.org> 554 5.2.2 User is over quota
Thomas is right in general, thats a general problem with mail quota, that was the reason why there wasnt some good solution out for long times,
but the dove policy server does i.e reject mail in smtp session if its allready "assured" that the mailbox is definite ... percent over quota ( configurable by grace parameter ), at that point it does not mater which size the incomming mail has, it will be i.e rejected anyway also it honors ( some kind overides ) other quota setting in i.e lmtp or lda, cause if it wouldnt, a mailbox would never become overquota by rejecting mail before by lmtp/lda settings
Best Regards MfG Robert Schetterer
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich
Thomas Leuxner skrev den 2013-08-06 18:25:
- Timo Sirainen <tss@iki.fi> 2013.08.06 18:15:
Now the real problem along the road is the submitting server. If that server does not indicate the message size during handshake the pre-queue rejection simply can not work.
quota_grace was meant to solve that. You'll allow the user to become a bit over quota.
What I meant is before the mail enters the Postfix queues. If the SIZE extension is not used during MAIL FROM by the remote server, then there's no way to reject an over-quota mail upfront, losing the benefit of the policy service.
dovecot dict sql qouta, then in postfix smtpd_end_of_data_restricttions check sql qoutas in policy deamons or just simple sql qoury will not solve it ?, correct if sender need to send data first to get sizes it begins to be impraktical since if there is just 1 byte free last sender can still send more then 1 byte, but next sender cant
participants (7)
-
/dev/rob0
-
Benny Pedersen
-
Charles Marcus
-
Robert Schetterer
-
Thomas Leuxner
-
Timo Sirainen
-
Ulrich Zehl