Pigeonhole redirect is adding a message-id header when it already exists

[John Stoffel]

>>>>> "Michael" == Michael Peddemors <michael at linuxmagic.com> writes:

> This should almost be an RFC discussion, rather than a dovecot 
> discussion, for clarity on what to do with a malformed Message-Id.

Seems simple, just rename it to "Message-Id-Orig: ..." and insert your
own.  Fixes the problem, still let's you trace the message, etc. 



> For the record, if you start modifying it by deleting the bad message 
> id, and adding your own, you can start breaking other things, such as 
> DKIM signing etc..

> IMHO, Dovecot should simply refuse to accept or deliver a message with a 
> 'bad' message id, so that the sending system can identify and correct 
> the problem.

> That way Dovecot doesn't need to address/modify the email message.

> 	-- Michael --

> On 2022-10-01 21:35, Sébastien Riccio wrote:
>> Hi,
>> 
>> After reading a bit the code and trying to understand it, here is what I 
>> think happens here:
>> 
>> Given a bogus Message-ID, for example (notice it's missing angle 
>> brackets < >:
>> 
>> Message-ID: 
>> 1883biz_pay_after_purchase:0:0_572392900$ae7ed6e4d53b424c84aaf83b30c507e7
>> 
>> Dovecot is parsing Message-ID headers and is looking for the angle 
>> bracket as the begining of the Message-ID:
>> https://github.com/dovecot/core/blob/d2ff32792ac052610cea7d65f30de1ee139cb55c/src/lib-mail/message-id.c#L75 
>> <https://github.com/dovecot/core/blob/d2ff32792ac052610cea7d65f30de1ee139cb55c/src/lib-mail/message-id.c#L75>
>> 
>> As none is found it will act as if there was no Message-ID header in the 
>> mail (even that the header is present).
>> 
>> Then, pigeonhole's redirect function  is told to generate a new 
>> Message-ID if none was previously detected:
>> https://github.com/dovecot/pigeonhole/blob/5a3f4bd672cc2fb9e755a4b09c4753ac86e15f99/src/lib-sieve/cmd-redirect.c#L569 
>> <https://github.com/dovecot/pigeonhole/blob/5a3f4bd672cc2fb9e755a4b09c4753ac86e15f99/src/lib-sieve/cmd-redirect.c#L569>
>> 
>> The result is the mail being forwarded, in this case, is now having dual 
>> Message-ID and is not RFC 5322 compliant anymore and can be rejected for 
>> this reason (hi, gmail?)
>> https://www.spamresource.com/2022/08/gmail-weird-rfc-5322-bounces-and-what.html 
>> <https://www.spamresource.com/2022/08/gmail-weird-rfc-5322-bounces-and-what.html>
>> 
>> Some thoughts:
>> 
>> - First, to be honest, I'm not sure gmail would accept the original mail 
>> with the bogus Message-ID sent directly to their servers, but if it was 
>> refused, I would assume that these senders would have fixed the issue on 
>> their side so their message are delivered (unless there is some 
>> whitelisting going on?)
>> 
>> - What options could we have to resolve this?
>> 
>> a) Having dovecot core to remove the Message-ID header line from the 
>> mail if it is not going to consider it valid ? (So there is no dupe 
>> headers when pigeonhole adds one?)
>> b) Having pigeonhole check, when adding a new valid Message-ID, if there 
>> is already one existing, and remove the bogus one ?
>> For now, to workaround this, I'm trying to find a way in the mail flow 
>> on our servers to keep only the top most Message-ID when more than one 
>> exists.
>> Maybe using: https://www.postfix.org/postconf.5.html#smtp_header_checks 
>> <https://www.postfix.org/postconf.5.html#smtp_header_checks> but I'm not 
>> sure how to achieve it yet or even if it's possible.
>> 
>> Kind regards
>> 
>> *Sébastien RICCIO*
>> 
>> *SYSTEM ADMINISTRATOR*
>> 
>> *P*  +41 840 888 888
>> 
>> *F***+41 840 888 000
>> 
>> *M****sriccio at swisscenter.com <mailto:sriccio at swisscenter.com>*
>> 
>> *
>> *
>> 
>> 
>> 
>> ------ Message d'origine ------
>> De "michael.zork at feierfighter.de <mailto:michael.zork at feierfighter.de>" 
>> <michael.zork at feierfighter.de <mailto:michael.zork at feierfighter.de>>
>> À "dovecot at dovecot.org <mailto:dovecot at dovecot.org>" 
>> <dovecot at dovecot.org <mailto:dovecot at dovecot.org>>
>> Date 01.10.2022 14:49:13
>> Objet Re: Re[6]: Pigeonhole redirect is adding a message-id header when 
>> it already exists
>> 
>>> Hi there,
>>> I can confirm this behavior. A few months ago I introduced a milter 
>>> which is checking for multiple headers when the RFC says that there 
>>> just should be one of them For example "Message-Id".
>>> I found the described problem in an email coming from Alibaba, which 
>>> had an invalid "Message-Id" header. It didn't contain an "@" sign or 
>>> similar. It was RFC-invalid.
>>> This email was sent from Alibaba to a German email provider. There was 
>>> a redirect at that email provider, pointing to my mailserver.
>>> My server rejected the email because there were 2 "Message-Id" 
>>> headers: The original invalid "Message-Id" header from Alibaba, and a 
>>> new "Message-Id" header from the German provider, which seems to have 
>>> been added during the redirect. There were "Dovecot-sieve" headers in 
>>> that mail, so my guess was that it happened because of 
>>> Dovecot-sieve/pigeonhole implementation.
>>> I contacted the email provider, asking for help. Asking if it really 
>>> is a bug in pigeonhole (or maybe some other system at that provider, 
>>> who knows). And I contacted Alibaba, so they fix the invalid 
>>> "Message-Id". I got responses from both, but until now, as far as I 
>>> can see, it has not been fixed.
>>> The best fix would be (if it really is a bug in pigeonhole), if 
>>> pigeonhole fixes the problem, then it's fixed for all users of 
>>> Dovecot. I guess Alibaba is not the only sender with an invalid 
>>> "Message-ID" header, but that's the only one I saw.
>>> Michael
>>> 
>>> 
>>> Am 01-Oct-2022 14:00:45 +0200 schrieb sriccio at swisscenter.com 
>>> <mailto:sriccio at swisscenter.com>:
>>> 
>>> >You wrote in the original email the message was rejected. Sorry I
>>> don't have login access to my gmail test account anymore since the
>>> google @#$%@#$% wanted to have me add a phone number.
>>> 
>>> In my original post I said that gmail was rejecting the forwards
>>> because
>>> of duplicate headers, and that the duplicate header seems to be a
>>> Message-ID added by pigeonhole when it's "not happy" with the
>>> original
>>> mail Message-ID.
>>> 
>>> I probably failed to explain the issue clearly and sorry for that.
>>> 
>>> Thank you anyway for trying to help :)
>>> 



> -- 
> "Catch the Magic of Linux..."
> ------------------------------------------------------------------------
> Michael Peddemors, President/CEO LinuxMagic Inc.
> Visit us at http://www.linuxmagic.com @linuxmagic
> A Wizard IT Company - For More Info http://www.wizard.ca
> "LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd.
> ------------------------------------------------------------------------
> 604-682-0300 Beautiful British Columbia, Canada

> This email and any electronic data contained are confidential and intended
> solely for the use of the individual or entity to which they are addressed.
> Please note that any views or opinions presented in this email are solely
> those of the author and are not intended to represent those of the company.