To field was not correct indexed by FTS

[Josef 'Jeff' Sipek]

On Tue, Jul 21, 2020 at 08:18:21 +0900, TACHIBANA Masashi wrote:
> Hi,
> 
> Thank you for your reply.
> 
> I hope this bug to be fixed soon. :)

It is a tricky one because as far as I can tell, the From header is invalid.
So, we have to figure out the best way to handle such input.

The input is invalid because '@' is not allowed by the RFC to be in the name
portion of an email address without quoting.  In other words, these are ok:

	test at example.com
	<test at example.com>
	Test <test at example.com>
	"test at example.com" <test at example.com>

but this one is bad:

	test at example.com <test at example.com>

In your case, the name is encoded but not quoted so even though the email
contains:

	=?UTF-8?B?dXNlcjJAZXhhbXBsZS5jb20=?= <user2 at example.com>

the parser sees:

	user2 at example.com <user2 at example.com>

Which is invalid.

Dovecot parsing needs to be improved, but as I already mentioned it is an
improvement to handling of malformed input.  So, as a workaround, if you
control the software/system that generates these emails, try to change it so
it doesn't generate headers that are invalid.  As far as I can tell, the
parser in Dovecot handles those just fine.

Jeff.

> Thank you.
> 
> Tachibana
> 
> ----- Original Message -----
> > On Mon, Jul 20, 2020 at 09:43:12 -0400, Josef 'Jeff' Sipek wrote:
> > > On Mon, Jul 20, 2020 at 20:24:13 +0900, TACHIBANA Masashi wrote:
> > ...
> > > Thanks for the report.  I reproduced it locally, but I'm not sure what is
> > > causing it yet.
> > 
> > Alright, it is actually a parsing bug.  The problem seems to be when the
> > code encounters a "name <address>" format where the name is a valid address.
> > This accidentally makes the parser stop parsing - making any additional
> > addresses ignored.
> > 
> > E.g.,
> > 
> >     test at example.com <test at example.com>, Name <other at example.com>
> > 
> > The parser stops after the first address's name thinking it is just a bare
> > address (without a name).  That is, after the first loop iterating while
> > parsing, the "cursor" is at the first space and not at the comma.
> > 
> >                                        /- parsing should stop here
> >                                        v
> >     test at example.com <test at example.com>, Name <other at example.com>
> >                     ^
> >                     `- parsing stops here
> > 
> > This is obviously wrong.  I'll have to dig in more to figure out what it
> > will take to fix this.
> > 
> > Thanks again,
> > 
> > Jeff.
> > 
> > -- 
> > FreeBSD 12.1
> > 
> --
> TACHIBANA Masashi  QUALITIA CO., LTD.
> mailto:tachibana at qualitia.co.jp
> 
> 株式会社クオリティア
> https://www.qualitia.co.jp/
> 
> 

-- 
Hegh QaQ law'
quvHa'ghach QaQ puS