On 4/6/2012 1:00 PM, Charles Marcus wrote:

On 2012-04-06 2:53 PM, Daniel L. Miller dmiller@amfes.com wrote:

...

I'm currently using Postfix 2.7, Dovecot 2.1, and the Dovecot LDA. I have a pure virtual user environment stored in LDAP. My messages include X-Original-To and Delivered-To headers.

Well that is great news... at least I'll be able to use the LDA, if not LMTP...

Thanks! :)

...

I had difficulty getting the LMTP transport to work previously - I may revisit that.

If you do, by all means reply back on whether or not the headers are still there...

Thanks again,

From the documentation... http://www.postfix.org/virtual.8.html

The*virtual*(8) http://www.postfix.org/virtual.8.html delivery agent prepends a "*From* /sender/ /time/*_*/stamp/" envelope header to each message, prepends a *Delivered-To:* message header with the envelope recipient address, prepends an*X-Original-To:* header with the recip- ient address as given to Postfix, prepends a*Return-Path:* message header with the envelope sender address, prepends a> character to lines beginning with "*From* ", and appends an empty line.

Using the Postfix pipe agent, which is what is used with the Dovecot LDA, http://www.postfix.org/pipe.8.html

*flags=BDFORXhqu.*> (optional) Optional message processing flags. By default, a message is copied unchanged.

           *B*       Append a blank line at the end of each  mes-
                  sage.  This  is  required  by some mail user
                  agents that recognize  "*From*   "  lines  only
                  when preceded by a blank line.

           *D*       Prepend  a "*Delivered-To:*  /recipient/" message
                  header with the envelope recipient  address.
                  Note: for this to work, the/transport/*_desti-*
                  *nation_recipient_limit*  must be 1  (see  SIN-
                  GLE-RECIPIENT DELIVERY above for details).

                  The*D*   flag  also  enforces  loop detection
                  (Postfix  2.5  and  later):  if  a   message
                  already contains a*Delivered-To:*  header with
                  the same recipient address, then the message
                  is  returned  as  undeliverable. The address
                  comparison is case insensitive.

                  This feature is available as of Postfix 2.0.

           *F*       Prepend  a "*From*  /sender time/*_*/stamp/" envelope
                  header to  the  message  content.   This  is
                  expected by, for example,*UUCP*  software.

           *O*       Prepend  an  "*X-Original-To:*  /recipient/" mes-
                  sage header with the  recipient  address  as
                  given  to  Postfix.  Note: for this to work,
                  the*/transport/_destination_recipient_limit  <http://www.postfix.org/postconf.5.html#transport_destination_recipient_limit>*
                  must  be  1  (see  SINGLE-RECIPIENT DELIVERY
                  above for details).

Unfortunately, the docs for the ltmp agent http://www.postfix.org/lmtp.8.html don't say anything about adding these headers. I tried asking on the Postfix list - didn't get much of an answer.

Daniel

On Sat, 7 Apr 2012 14:30:38 -0400, Jerry wrote:

On Sat, 07 Apr 2012 11:06:48 -0700 Daniel L. Miller articulated:

...

Unfortunately, the docs for the ltmp agent http://www.postfix.org/lmtp.8.html [1] don't say anything about adding these headers. I tried asking on the Postfix list - didn't get much of an answer.

I may be wrong; however, from what I have been able to understand in regards to the Postfix documentation, if it does not explicitly claim to have a feature, then that feature is not available. In other words, if it doesn't state it can do it, it can't.

As I just stated on that list - even though a given feature may be documented, the possible uses of that feature may not be immediately apparent. And while the Postfix lda & virtual transports have the "flag" parameters, and the lmtp transport does not - the lmtp transport DOES have a whole slew of other parameters not available in the lda. So I was simply inquiring if there was a way to achieve my goal - given that my understanding of smtp handling in general, and Postfix in particular, are extremely limited.

For some reason, I seem to irritate people with my polite questions - while others who are (in my opinion) downright rude and aggressive get assistance and acceptance. Maybe I need to start being more of a jerk on purpose...

-- Daniel

Links:

[1] http://www.postfix.org/lmtp.8.html

On 2012-04-09 3:33 AM, Timo Sirainen tss@iki.fi wrote:

On 5.4.2012, at 15.59, Charles Marcus wrote:

...

Does anyone know if the use of LMTP (or even the dovecot LDA) still loses the x-original-to header that the postfix vda adds and that I rely heavily on (since I use a lot of aliases), and if it does, is there any solution to get the original recipient added back in before final delivery?

LMTP adds a new Delivered-To:rcpt-to@address header when there is a single RCPT TO. You can force a single RCPT TO from Postfix side by setting lmtp_destination_recipient_limit=1. LMTP doesn't add/remove/change X-Original-To: header.

Ok, thanks Timo... but...

Are you saying that this 'Delivered-To:' header can somehow be leveraged to provide the same info as the x-original-to header?

If not, since it was the postfix virtual delivery agent that added the x-original-to, and since using lmtp means I would not be using the postfix vda, is the appropriate place to add this header in dovecot's lmtp implementation (and if so, how hard would it be)? Or would this need to be done somehow on the postfix side (if so, I'll go ask on the postfix list)? Sorry for my ignorance - but as I said, I rely on this header (I use a ton of aliases, and without it I can't see the original (alias) recipient), so I need to determine if I'm going to be able to use lmtp or not (obviously, I would much prefer to do so)...

Thanks again Timo...

--

Best regards,

Charles

On 2012-04-09 8:53 AM, Timo Sirainen tss@iki.fi wrote:

I guess X-Original-To is the same address as what Postfix sees as the original RCPT TO address before alias expansion and such? In that case, see my today's mail in Postfix list.

Yep... and hoping that you and Wietse can work out some way to support it...

Thanks for participating in the discussion over there... :)

--

Best regards,

Charles

On 2014-01-08 14:32, Charles Marcus wrote:

On 2012-04-09 8:53 AM, Timo Sirainen tss@iki.fi wrote:

...

On 9.4.2012, at 15.50, Charles Marcus wrote:

...

...

LMTP adds a new Delivered-To:rcpt-to@address header when there is a single RCPT TO. You can force a single RCPT TO from Postfix side by setting lmtp_destination_recipient_limit=1. LMTP doesn't add/remove/change X-Original-To: header.

...

...

Ok, thanks Timo... but...

Are you saying that this 'Delivered-To:' header can somehow be leveraged to provide the same info as the x-original-to header?

...

I guess X-Original-To is the same address as what Postfix sees as the original RCPT TO address before alias expansion and such? In that case, see my today's mail in Postfix list..

Hi Timo,

I just tried to find your email from that day, but don't see it in the archives...

Was this ever resolved (getting x-original-to support in LMTP, like it is for the LDA)?

If not, since it seemed like it wasn't going to be much work, any chance you can revisit it soon?

Thanks,

Hello,

I have tried to keep tabs on the various discussions going on related to the X-Original-To header when using Dovecot LMTP. Until now I have not needed a solution, but I am now finally about to migrate my old server.

Old setup: Postfix + SpamAssassin (after-queue content filter via pipe)

• virtual transport, and Courier-IMAP. New setup: Postfix + amavisd-new (after-queue content filter via smtp, with ClamAV and SpamAssassin) + Dovecot LMTP, and Dovecot for IMAP.

Charles, have you found a way that works for you?

I was experimenting some with my test server and came up with a way that utilizes some additional internal smtp content filter forwarding, which produces some overhead. It should be light compared with the load from ClamAV and SpamAssassin, however.

I'm not yet sure how amavisd-new funneling would handle multiple local recipients with different settings without passing the mail through multiple time, at least once per local user, let alone without first performing address mapping in postfix (for alias expansion). I have configured per-user SpamAssassin bayes filtering, and may introduce a whitelist based on address book entries (Roundcube.)

This solution I'm currently testing will pass each message through amavisd-new one time each per local and remote recipient, and will only add the X-Original-To header to the specific local recipient each envelope is intended for. No external users will receive the header, and no local user will see which other local users (e.g. via BCC) have potentially received the same message.

Flow: all mail in (both external and tls-authenticated internal) -> smtp (1) -> smtp-split (2) -> smtp-to-me (3a) | smtp-to-external (3b) -> smtp-amavis (4) -> dovecot-lmtp (5)

1. I rely on default_destination_recipient_limit=1 in main.cf to split each incoming mail into one stream per recipient.
2. smtp-split will receive one stream per recipient. Default content_filter=smtp-to-me, followed by option "smtpd_recipient_restrictions=permit_auth_destination,check_recipient_access,pcre:/usr/local/etc/postfix/filter-to-external.pcre,permit_mynetworks,reject" means I stop processing restrictions if my server is the destination. If my server is not the destination, the FILTER in check_recipient_access will override the preceding smtp-to-me filter.

Both 1) and 2) smtpd instances include option receive_override_options=no_address_mappings, to wait with mapping to internal recipient until we can add X-Original-To header for my server's users only.

3a) For mail to my server, smtp-to-me will add X-Original-To using a pcre script, in a similar fashion to step 2's filter. This step also expands the address mapping (by not specifying any receive_override_options). -o smtpd_recipient_restrictions=check_recipient_access,pcre:/usr/local/etc/postfix/recipient_access_x-orig.pcre,permit_mynetworks,reject

3b) For mail leaving my server, smtp-to-external will not add any processing besides implied expanding of the address mapping.

4. Mail is funneled through amavisd-new, once per final recipient. Mails leaving the server (sent from smtp-to-external) will be scanned by ClamV only. Mails with my server as the destination (sent from smtp-to-me) will go through ClamV, and SpamAssassin (together with per-user bayes filtering).

5. Nothing special is done here. The final destination address is sent to LMTP for delivery.

Contents of /usr/local/etc/postfix/recipient_access_x-orig.pcre: /(.+)/ prepend X-Original-To: <$1>

Contents of /usr/local/etc/postfix/filter-to-external.pcre: /^/ FILTER smtp-to-external:[127.0.0.1]:<port>

Room for improvement: Postfix seem to know the orig_to even after processing in amavisd-new, however I have yet to find a way to use this option. I can move the amavisd-new filter to before the X-Original-To header addition, however for amavisd-new to utilize per-user bayes, I currently need to do the address mapping in postfix before sending the content to amavisd-new. This may be possible to circumvent either with alias lookup in amavisd-new, or if I can find another way to use the postfix-available "orig_to" to populate X-Original-To header after scanning in amavisd-new.

I've tried to split the mails into one per recipient after address mapping and amavisd-new, instead of before as my solution above, without default_destination_recipient_limit=1 in main.cf. Instead I tried the options as part of smtp (and even smtpd) services in master.cf, unfortunately without success. I keep ending up with multiple X-Original-To with all local recipients (including BCC) in all internal copies delivered.

Have anyone successfully tackled this conundrum with other solutions?

Regards, Tobias

I'm resurrecting this again because I'm getting pretty close to possibly being ready to install a brand new dovecot server (finally), but I still need for dovecots LMTP to add the x-original-to header.

So... was this completed quietly, or is support for it still not there?

Thanks,

Charles

On Tue Apr 28 2015 15:27:07 GMT-0400 (Eastern Standard Time), Charles Marcus CMarcus@Media-Brokers.com wrote:

On 4/28/2015 1:40 PM, Tobias Franzén lists.zxinn@otaking.se wrote:

...

On 2014-01-08 14:32, Charles Marcus wrote:

...

On 2012-04-09 8:53 AM, Timo Sirainen tss@iki.fi wrote:

...

On 9.4.2012, at 15.50, Charles Marcus wrote:

...

...

LMTP adds a new Delivered-To:rcpt-to@address header when there is a single RCPT TO. You can force a single RCPT TO from Postfix side by setting lmtp_destination_recipient_limit=1. LMTP doesn't add/remove/change X-Original-To: header. Ok, thanks Timo... but...

Are you saying that this 'Delivered-To:' header can somehow be leveraged to provide the same info as the x-original-to header? I guess X-Original-To is the same address as what Postfix sees as the original RCPT TO address before alias expansion and such? In that case, see my today's mail in Postfix list.. Hi Timo,

I just tried to find your email from that day, but don't see it in the archives...

Was this ever resolved (getting x-original-to support in LMTP, like it is for the LDA)?

If not, since it seemed like it wasn't going to be much work, any chance you can revisit it soon? Hello,

I have tried to keep tabs on the various discussions going on related to the X-Original-To header when using Dovecot LMTP. Until now I have not needed a solution, but I am now finally about to migrate my old server.

Old setup: Postfix + SpamAssassin (after-queue content filter via pipe)

• virtual transport, and Courier-IMAP. New setup: Postfix + amavisd-new (after-queue content filter via smtp, with ClamAV and SpamAssassin) + Dovecot LMTP, and Dovecot for IMAP.

Charles, have you found a way that works for you?

No, and I simply haven't switched to LMTP yet, for this and one other reason (political, not technical)...

As for the rest below... wow... all I can say is, it sure would be nice if Timo/Wietse could just add the few lines of code that Timo said would be needed to properly support it natively.

...

I was experimenting some with my test server and came up with a way that utilizes some additional internal smtp content filter forwarding, which produces some overhead. It should be light compared with the load from ClamAV and SpamAssassin, however.

I'm not yet sure how amavisd-new funneling would handle multiple local recipients with different settings without passing the mail through multiple time, at least once per local user, let alone without first performing address mapping in postfix (for alias expansion). I have configured per-user SpamAssassin bayes filtering, and may introduce a whitelist based on address book entries (Roundcube.)

This solution I'm currently testing will pass each message through amavisd-new one time each per local and remote recipient, and will only add the X-Original-To header to the specific local recipient each envelope is intended for. No external users will receive the header, and no local user will see which other local users (e.g. via BCC) have potentially received the same message.

Flow: all mail in (both external and tls-authenticated internal) -> smtp (1) -> smtp-split (2) -> smtp-to-me (3a) | smtp-to-external (3b) -> smtp-amavis (4) -> dovecot-lmtp (5)

1. I rely on default_destination_recipient_limit=1 in main.cf to split each incoming mail into one stream per recipient.
2. smtp-split will receive one stream per recipient. Default content_filter=smtp-to-me, followed by option "smtpd_recipient_restrictions=permit_auth_destination,check_recipient_access,pcre:/usr/local/etc/postfix/filter-to-external.pcre,permit_mynetworks,reject" means I stop processing restrictions if my server is the destination. If my server is not the destination, the FILTER in check_recipient_access will override the preceding smtp-to-me filter.

Both 1) and 2) smtpd instances include option receive_override_options=no_address_mappings, to wait with mapping to internal recipient until we can add X-Original-To header for my server's users only.

3a) For mail to my server, smtp-to-me will add X-Original-To using a pcre script, in a similar fashion to step 2's filter. This step also expands the address mapping (by not specifying any receive_override_options). -o smtpd_recipient_restrictions=check_recipient_access,pcre:/usr/local/etc/postfix/recipient_access_x-orig.pcre,permit_mynetworks,reject

3b) For mail leaving my server, smtp-to-external will not add any processing besides implied expanding of the address mapping.

4. Mail is funneled through amavisd-new, once per final recipient. Mails leaving the server (sent from smtp-to-external) will be scanned by ClamV only. Mails with my server as the destination (sent from smtp-to-me) will go through ClamV, and SpamAssassin (together with per-user bayes filtering).

5. Nothing special is done here. The final destination address is sent to LMTP for delivery.

Contents of /usr/local/etc/postfix/recipient_access_x-orig.pcre: /(.+)/ prepend X-Original-To: <$1>

Contents of /usr/local/etc/postfix/filter-to-external.pcre: /^/ FILTER smtp-to-external:[127.0.0.1]:<port>

Room for improvement: Postfix seem to know the orig_to even after processing in amavisd-new, however I have yet to find a way to use this option. I can move the amavisd-new filter to before the X-Original-To header addition, however for amavisd-new to utilize per-user bayes, I currently need to do the address mapping in postfix before sending the content to amavisd-new. This may be possible to circumvent either with alias lookup in amavisd-new, or if I can find another way to use the postfix-available "orig_to" to populate X-Original-To header after scanning in amavisd-new.

I've tried to split the mails into one per recipient after address mapping and amavisd-new, instead of before as my solution above, without default_destination_recipient_limit=1 in main.cf. Instead I tried the options as part of smtp (and even smtpd) services in master.cf, unfortunately without success. I keep ending up with multiple X-Original-To with all local recipients (including BCC) in all internal copies delivered.

Have anyone successfully tackled this conundrum with other solutions?

Regards, Tobias