[Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header

[Charles Marcus]

On 4/28/2015 1:40 PM, Tobias Franzén <lists.zxinn at otaking.se> wrote:
> On 2014-01-08 14:32, Charles Marcus wrote:
>> On 2012-04-09 8:53 AM, Timo Sirainen <tss at iki.fi> wrote:
>>> On 9.4.2012, at 15.50, Charles Marcus wrote:
>>>>> LMTP adds a new Delivered-To:<rcpt-to at address>  header when there is
>>>>> a single RCPT TO. You can force a single RCPT TO from Postfix side by
>>>>> setting lmtp_destination_recipient_limit=1. LMTP doesn't
>>>>> add/remove/change X-Original-To: header.
>>>> Ok, thanks Timo... but...
>>>>
>>>> Are you saying that this 'Delivered-To:' header can somehow be 
>>>> leveraged to provide the same info as the x-original-to header?
>>> I guess X-Original-To is the same address as what Postfix sees as the 
>>> original RCPT TO address before alias expansion and such? In that 
>>> case, see my today's mail in Postfix list..
>> Hi Timo,
>>
>> I just tried to find your email from that day, but don't see it in the 
>> archives...
>>
>> Was this ever resolved (getting x-original-to support in LMTP, like it 
>> is for the LDA)?
>>
>> If not, since it seemed like it wasn't going to be much work, any 
>> chance you can revisit it soon?
> Hello,
>
> I have tried to keep tabs on the various discussions going on related to 
> the X-Original-To header when using Dovecot LMTP. Until now I have not 
> needed a solution, but I am now finally about to migrate my old server.
>
> Old setup: Postfix + SpamAssassin (after-queue content filter via pipe) 
> + virtual transport, and Courier-IMAP.
> New setup: Postfix + amavisd-new (after-queue content filter via smtp, 
> with ClamAV and SpamAssassin) + Dovecot LMTP, and Dovecot for IMAP.
>
> Charles, have you found a way that works for you?

No, and I simply haven't switched to LMTP yet, for this and one other
reason (political, not technical)...

As for the rest below... wow... all I can say is, it sure would be nice
if Timo/Wietse could just add the few lines of code that Timo said would
be needed to properly support it natively.

> I was experimenting some with my test server and came up with a way that 
> utilizes some additional internal smtp content filter forwarding, which 
> produces some overhead. It should be light compared with the load from 
> ClamAV and SpamAssassin, however.
>
> I'm not yet sure how amavisd-new funneling would handle multiple local 
> recipients with different settings without passing the mail through 
> multiple time, at least once per local user, let alone without first 
> performing address mapping in postfix (for alias expansion). I have 
> configured per-user SpamAssassin bayes filtering, and may introduce a 
> whitelist based on address book entries (Roundcube.)
>
>
> This solution I'm currently testing will pass each message through 
> amavisd-new one time each per local and remote recipient, and will only 
> add the X-Original-To header to the specific local recipient each 
> envelope is intended for. No external users will receive the header, and 
> no local user will see which other local users (e.g. via BCC) have 
> potentially received the same message.
>
> Flow:
> all mail in (both external and tls-authenticated internal) -> smtp (1) 
> -> smtp-split (2) -> smtp-to-me (3a) | smtp-to-external (3b) -> 
> smtp-amavis (4) -> dovecot-lmtp (5)
>
> 1) I rely on default_destination_recipient_limit=1 in main.cf to split 
> each incoming mail into one stream per recipient.
> 2) smtp-split will receive one stream per recipient. Default 
> content_filter=smtp-to-me, followed by option 
> "smtpd_recipient_restrictions=permit_auth_destination,check_recipient_access,pcre:/usr/local/etc/postfix/filter-to-external.pcre,permit_mynetworks,reject" 
> means I stop processing restrictions if my server is the destination. If 
> my server is not the destination, the FILTER in check_recipient_access 
> will override the preceding smtp-to-me filter.
>
> Both 1) and 2) smtpd instances include option 
> receive_override_options=no_address_mappings, to wait with mapping to 
> internal recipient until we can add X-Original-To header for my server's 
> users only.
>
> 3a) For mail to my server, smtp-to-me will add X-Original-To using a 
> pcre script, in a similar fashion to step 2's filter. This step also 
> expands the address mapping (by not specifying any 
> receive_override_options).
>    -o 
> smtpd_recipient_restrictions=check_recipient_access,pcre:/usr/local/etc/postfix/recipient_access_x-orig.pcre,permit_mynetworks,reject
>
> 3b) For mail leaving my server, smtp-to-external will not add any 
> processing besides implied expanding of the address mapping.
>
> 4) Mail is funneled through amavisd-new, once per final recipient. Mails 
> leaving the server (sent from smtp-to-external) will be scanned by ClamV 
> only. Mails with my server as the destination (sent from smtp-to-me) 
> will go through ClamV, and SpamAssassin (together with per-user bayes 
> filtering).
>
> 5) Nothing special is done here. The final destination address is sent 
> to LMTP for delivery.
>
> Contents of /usr/local/etc/postfix/recipient_access_x-orig.pcre:
> /(.+)/        prepend X-Original-To: <$1>
>
> Contents of /usr/local/etc/postfix/filter-to-external.pcre:
> /^/        FILTER smtp-to-external:[127.0.0.1]:<port>
>
>
> Room for improvement:
> Postfix seem to know the orig_to even after processing in amavisd-new, 
> however I have yet to find a way to use this option.
> I can move the amavisd-new filter to before the X-Original-To header 
> addition, however for amavisd-new to utilize per-user bayes, I currently 
> need to do the address mapping in postfix before sending the content to 
> amavisd-new. This may be possible to circumvent either with alias lookup 
> in amavisd-new, or if I can find another way to use the 
> postfix-available "orig_to" to populate X-Original-To header after 
> scanning in amavisd-new.
>
> I've tried to split the mails into one per recipient after address 
> mapping and amavisd-new, instead of before as my solution above, without 
> default_destination_recipient_limit=1 in main.cf. Instead I tried the 
> options as part of smtp (and even smtpd) services in master.cf, 
> unfortunately without success. I keep ending up with multiple 
> X-Original-To with all local recipients (including BCC) in all internal 
> copies delivered.
>
>
> Have anyone successfully tackled this conundrum with other solutions?
>
> Regards,
> Tobias
>