[Dovecot] Dovecot LDA/LMTP vs postfix virtual delivery agent and the x-original-to header

[Tanstaafl]

I'm resurrecting this again because I'm getting pretty close to possibly
being ready to install a brand new dovecot server (finally), but I still
need for dovecots LMTP to add the x-original-to header.

So... was this completed quietly, or is support for it still not there?

Thanks,

Charles

On Tue Apr 28 2015 15:27:07 GMT-0400 (Eastern Standard Time), Charles
Marcus <CMarcus at Media-Brokers.com> wrote:
> On 4/28/2015 1:40 PM, Tobias Franzén <lists.zxinn at otaking.se> wrote:
>> On 2014-01-08 14:32, Charles Marcus wrote:
>>> On 2012-04-09 8:53 AM, Timo Sirainen <tss at iki.fi> wrote:
>>>> On 9.4.2012, at 15.50, Charles Marcus wrote:
>>>>>> LMTP adds a new Delivered-To:<rcpt-to at address>  header when there is
>>>>>> a single RCPT TO. You can force a single RCPT TO from Postfix side by
>>>>>> setting lmtp_destination_recipient_limit=1. LMTP doesn't
>>>>>> add/remove/change X-Original-To: header.
>>>>> Ok, thanks Timo... but...
>>>>>
>>>>> Are you saying that this 'Delivered-To:' header can somehow be 
>>>>> leveraged to provide the same info as the x-original-to header?
>>>> I guess X-Original-To is the same address as what Postfix sees as the 
>>>> original RCPT TO address before alias expansion and such? In that 
>>>> case, see my today's mail in Postfix list..
>>> Hi Timo,
>>>
>>> I just tried to find your email from that day, but don't see it in the 
>>> archives...
>>>
>>> Was this ever resolved (getting x-original-to support in LMTP, like it 
>>> is for the LDA)?
>>>
>>> If not, since it seemed like it wasn't going to be much work, any 
>>> chance you can revisit it soon?
>> Hello,
>>
>> I have tried to keep tabs on the various discussions going on related to 
>> the X-Original-To header when using Dovecot LMTP. Until now I have not 
>> needed a solution, but I am now finally about to migrate my old server.
>>
>> Old setup: Postfix + SpamAssassin (after-queue content filter via pipe) 
>> + virtual transport, and Courier-IMAP.
>> New setup: Postfix + amavisd-new (after-queue content filter via smtp, 
>> with ClamAV and SpamAssassin) + Dovecot LMTP, and Dovecot for IMAP.
>>
>> Charles, have you found a way that works for you?
> 
> No, and I simply haven't switched to LMTP yet, for this and one other
> reason (political, not technical)...
> 
> As for the rest below... wow... all I can say is, it sure would be nice
> if Timo/Wietse could just add the few lines of code that Timo said would
> be needed to properly support it natively.
> 
>> I was experimenting some with my test server and came up with a way that 
>> utilizes some additional internal smtp content filter forwarding, which 
>> produces some overhead. It should be light compared with the load from 
>> ClamAV and SpamAssassin, however.
>>
>> I'm not yet sure how amavisd-new funneling would handle multiple local 
>> recipients with different settings without passing the mail through 
>> multiple time, at least once per local user, let alone without first 
>> performing address mapping in postfix (for alias expansion). I have 
>> configured per-user SpamAssassin bayes filtering, and may introduce a 
>> whitelist based on address book entries (Roundcube.)
>>
>>
>> This solution I'm currently testing will pass each message through 
>> amavisd-new one time each per local and remote recipient, and will only 
>> add the X-Original-To header to the specific local recipient each 
>> envelope is intended for. No external users will receive the header, and 
>> no local user will see which other local users (e.g. via BCC) have 
>> potentially received the same message.
>>
>> Flow:
>> all mail in (both external and tls-authenticated internal) -> smtp (1) 
>> -> smtp-split (2) -> smtp-to-me (3a) | smtp-to-external (3b) -> 
>> smtp-amavis (4) -> dovecot-lmtp (5)
>>
>> 1) I rely on default_destination_recipient_limit=1 in main.cf to split 
>> each incoming mail into one stream per recipient.
>> 2) smtp-split will receive one stream per recipient. Default 
>> content_filter=smtp-to-me, followed by option 
>> "smtpd_recipient_restrictions=permit_auth_destination,check_recipient_access,pcre:/usr/local/etc/postfix/filter-to-external.pcre,permit_mynetworks,reject" 
>> means I stop processing restrictions if my server is the destination. If 
>> my server is not the destination, the FILTER in check_recipient_access 
>> will override the preceding smtp-to-me filter.
>>
>> Both 1) and 2) smtpd instances include option 
>> receive_override_options=no_address_mappings, to wait with mapping to 
>> internal recipient until we can add X-Original-To header for my server's 
>> users only.
>>
>> 3a) For mail to my server, smtp-to-me will add X-Original-To using a 
>> pcre script, in a similar fashion to step 2's filter. This step also 
>> expands the address mapping (by not specifying any 
>> receive_override_options).
>>    -o 
>> smtpd_recipient_restrictions=check_recipient_access,pcre:/usr/local/etc/postfix/recipient_access_x-orig.pcre,permit_mynetworks,reject
>>
>> 3b) For mail leaving my server, smtp-to-external will not add any 
>> processing besides implied expanding of the address mapping.
>>
>> 4) Mail is funneled through amavisd-new, once per final recipient. Mails 
>> leaving the server (sent from smtp-to-external) will be scanned by ClamV 
>> only. Mails with my server as the destination (sent from smtp-to-me) 
>> will go through ClamV, and SpamAssassin (together with per-user bayes 
>> filtering).
>>
>> 5) Nothing special is done here. The final destination address is sent 
>> to LMTP for delivery.
>>
>> Contents of /usr/local/etc/postfix/recipient_access_x-orig.pcre:
>> /(.+)/        prepend X-Original-To: <$1>
>>
>> Contents of /usr/local/etc/postfix/filter-to-external.pcre:
>> /^/        FILTER smtp-to-external:[127.0.0.1]:<port>
>>
>>
>> Room for improvement:
>> Postfix seem to know the orig_to even after processing in amavisd-new, 
>> however I have yet to find a way to use this option.
>> I can move the amavisd-new filter to before the X-Original-To header 
>> addition, however for amavisd-new to utilize per-user bayes, I currently 
>> need to do the address mapping in postfix before sending the content to 
>> amavisd-new. This may be possible to circumvent either with alias lookup 
>> in amavisd-new, or if I can find another way to use the 
>> postfix-available "orig_to" to populate X-Original-To header after 
>> scanning in amavisd-new.
>>
>> I've tried to split the mails into one per recipient after address 
>> mapping and amavisd-new, instead of before as my solution above, without 
>> default_destination_recipient_limit=1 in main.cf. Instead I tried the 
>> options as part of smtp (and even smtpd) services in master.cf, 
>> unfortunately without success. I keep ending up with multiple 
>> X-Original-To with all local recipients (including BCC) in all internal 
>> copies delivered.
>>
>>
>> Have anyone successfully tackled this conundrum with other solutions?
>>
>> Regards,
>> Tobias
>>
>