Re: To field was not correct indexed by FTS

21 Jul 2020


      On Tue, Jul 21, 2020 at 08:18:21 +0900, TACHIBANA Masashi wrote:
...
Hi,
Thank you for your reply.
I hope this bug to be fixed soon. :)
It is a tricky one because as far as I can tell, the From header is invalid.
So, we have to figure out the best way to handle such input.
The input is invalid because '@' is not allowed by the RFC to be in the name
portion of an email address without quoting.  In other words, these are ok:
test@example.com
&lt;test@example.com>
Test &lt;test@example.com>
"test@example.com" &lt;test@example.com>
but this one is bad:
test@example.com &lt;test@example.com>
In your case, the name is encoded but not quoted so even though the email
contains:
=?UTF-8?B?dXNlcjJAZXhhbXBsZS5jb20=?= &lt;user2@example.com>
the parser sees:
user2@example.com &lt;user2@example.com>
Which is invalid.
Dovecot parsing needs to be improved, but as I already mentioned it is an
improvement to handling of malformed input.  So, as a workaround, if you
control the software/system that generates these emails, try to change it so
it doesn't generate headers that are invalid.  As far as I can tell, the
parser in Dovecot handles those just fine.
Jeff.
...
Thank you.
Tachibana
----- Original Message -----
...
On Mon, Jul 20, 2020 at 09:43:12 -0400, Josef 'Jeff' Sipek wrote:
...
On Mon, Jul 20, 2020 at 20:24:13 +0900, TACHIBANA Masashi wrote:
...
Thanks for the report.  I reproduced it locally, but I'm not sure what is
causing it yet.
Alright, it is actually a parsing bug.  The problem seems to be when the
code encounters a "name <address>" format where the name is a valid address.
This accidentally makes the parser stop parsing - making any additional
addresses ignored.
E.g.,
test@example.com &lt;test@example.com>, Name &lt;other@example.com>
The parser stops after the first address's name thinking it is just a bare
address (without a name).  That is, after the first loop iterating while
parsing, the "cursor" is at the first space and not at the comma.
                                   /- parsing should stop here
                                   v
test@example.com &lt;test@example.com>, Name &lt;other@example.com>
                ^
                `- parsing stops here
This is obviously wrong.  I'll have to dig in more to figure out what it
will take to fix this.
Thanks again,
Jeff.
--
FreeBSD 12.1
--
TACHIBANA Masashi  QUALITIA CO., LTD.
mailto:tachibana@qualitia.co.jp
株式会社クオリティア
https://www.qualitia.co.jp/
--
Hegh QaQ law'
quvHa'ghach QaQ puS