Pigeonhole redirect is adding a message-id header when it already exists

Mon Oct 3 14:51:26 UTC 2022

This should almost be an RFC discussion, rather than a dovecot 
discussion, for clarity on what to do with a malformed Message-Id.

For the record, if you start modifying it by deleting the bad message 
id, and adding your own, you can start breaking other things, such as 
DKIM signing etc..

IMHO, Dovecot should simply refuse to accept or deliver a message with a 
'bad' message id, so that the sending system can identify and correct 
the problem.

That way Dovecot doesn't need to address/modify the email message.

	-- Michael --

On 2022-10-01 21:35, Sébastien Riccio wrote:
> Hi,
> 
> After reading a bit the code and trying to understand it, here is what I 
> think happens here:
> 
> Given a bogus Message-ID, for example (notice it's missing angle 
> brackets < >:
> 
> Message-ID: 
> 1883biz_pay_after_purchase:0:0_572392900$ae7ed6e4d53b424c84aaf83b30c507e7
> 
> Dovecot is parsing Message-ID headers and is looking for the angle 
> bracket as the begining of the Message-ID:
> https://github.com/dovecot/core/blob/d2ff32792ac052610cea7d65f30de1ee139cb55c/src/lib-mail/message-id.c#L75 
> <https://github.com/dovecot/core/blob/d2ff32792ac052610cea7d65f30de1ee139cb55c/src/lib-mail/message-id.c#L75>
> 
> As none is found it will act as if there was no Message-ID header in the 
> mail (even that the header is present).
> 
> Then, pigeonhole's redirect function  is told to generate a new 
> Message-ID if none was previously detected:
> https://github.com/dovecot/pigeonhole/blob/5a3f4bd672cc2fb9e755a4b09c4753ac86e15f99/src/lib-sieve/cmd-redirect.c#L569 
> <https://github.com/dovecot/pigeonhole/blob/5a3f4bd672cc2fb9e755a4b09c4753ac86e15f99/src/lib-sieve/cmd-redirect.c#L569>
> 
> The result is the mail being forwarded, in this case, is now having dual 
> Message-ID and is not RFC 5322 compliant anymore and can be rejected for 
> this reason (hi, gmail?)
> https://www.spamresource.com/2022/08/gmail-weird-rfc-5322-bounces-and-what.html 
> <https://www.spamresource.com/2022/08/gmail-weird-rfc-5322-bounces-and-what.html>
> 
> Some thoughts:
> 
> - First, to be honest, I'm not sure gmail would accept the original mail 
> with the bogus Message-ID sent directly to their servers, but if it was 
> refused, I would assume that these senders would have fixed the issue on 
> their side so their message are delivered (unless there is some 
> whitelisting going on?)
> 
> - What options could we have to resolve this?
> 
> a) Having dovecot core to remove the Message-ID header line from the 
> mail if it is not going to consider it valid ? (So there is no dupe 
> headers when pigeonhole adds one?)
> b) Having pigeonhole check, when adding a new valid Message-ID, if there 
> is already one existing, and remove the bogus one ?
> For now, to workaround this, I'm trying to find a way in the mail flow 
> on our servers to keep only the top most Message-ID when more than one 
> exists.
> Maybe using: https://www.postfix.org/postconf.5.html#smtp_header_checks 
> <https://www.postfix.org/postconf.5.html#smtp_header_checks> but I'm not 
> sure how to achieve it yet or even if it's possible.
> 
> Kind regards
> 
> *Sébastien RICCIO*
> 
> *SYSTEM ADMINISTRATOR*
> 
> *P*  +41 840 888 888
> 
> *F***+41 840 888 000
> 
> *M****sriccio at swisscenter.com <mailto:sriccio at swisscenter.com>*
> 
> *
> *
> 
> 
> 
> ------ Message d'origine ------
> De "michael.zork at feierfighter.de <mailto:michael.zork at feierfighter.de>" 
> <michael.zork at feierfighter.de <mailto:michael.zork at feierfighter.de>>
> À "dovecot at dovecot.org <mailto:dovecot at dovecot.org>" 
> <dovecot at dovecot.org <mailto:dovecot at dovecot.org>>
> Date 01.10.2022 14:49:13
> Objet Re: Re[6]: Pigeonhole redirect is adding a message-id header when 
> it already exists
> 
>> Hi there,
>> I can confirm this behavior. A few months ago I introduced a milter 
>> which is checking for multiple headers when the RFC says that there 
>> just should be one of them For example "Message-Id".
>> I found the described problem in an email coming from Alibaba, which 
>> had an invalid "Message-Id" header. It didn't contain an "@" sign or 
>> similar. It was RFC-invalid.
>> This email was sent from Alibaba to a German email provider. There was 
>> a redirect at that email provider, pointing to my mailserver.
>> My server rejected the email because there were 2 "Message-Id" 
>> headers: The original invalid "Message-Id" header from Alibaba, and a 
>> new "Message-Id" header from the German provider, which seems to have 
>> been added during the redirect. There were "Dovecot-sieve" headers in 
>> that mail, so my guess was that it happened because of 
>> Dovecot-sieve/pigeonhole implementation.
>> I contacted the email provider, asking for help. Asking if it really 
>> is a bug in pigeonhole (or maybe some other system at that provider, 
>> who knows). And I contacted Alibaba, so they fix the invalid 
>> "Message-Id". I got responses from both, but until now, as far as I 
>> can see, it has not been fixed.
>> The best fix would be (if it really is a bug in pigeonhole), if 
>> pigeonhole fixes the problem, then it's fixed for all users of 
>> Dovecot. I guess Alibaba is not the only sender with an invalid 
>> "Message-ID" header, but that's the only one I saw.
>> Michael
>>
>>
>> Am 01-Oct-2022 14:00:45 +0200 schrieb sriccio at swisscenter.com 
>> <mailto:sriccio at swisscenter.com>:
>>
>>     >You wrote in the original email the message was rejected. Sorry I
>>     don't have login access to my gmail test account anymore since the
>>     google @#$%@#$% wanted to have me add a phone number.
>>
>>     In my original post I said that gmail was rejecting the forwards
>>     because
>>     of duplicate headers, and that the duplicate header seems to be a
>>     Message-ID added by pigeonhole when it's "not happy" with the
>>     original
>>     mail Message-ID.
>>
>>     I probably failed to explain the issue clearly and sorry for that.
>>
>>     Thank you anyway for trying to help :)
>>

-- 
"Catch the Magic of Linux..."
------------------------------------------------------------------------
Michael Peddemors, President/CEO LinuxMagic Inc.
Visit us at http://www.linuxmagic.com @linuxmagic
A Wizard IT Company - For More Info http://www.wizard.ca
"LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd.
------------------------------------------------------------------------
604-682-0300 Beautiful British Columbia, Canada

This email and any electronic data contained are confidential and intended
solely for the use of the individual or entity to which they are addressed.
Please note that any views or opinions presented in this email are solely
those of the author and are not intended to represent those of the company.