v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed)

[Shawn Heisey]

On 10/11/2020 4:27 PM, PGNet Dev wrote:
> I'm running,
> 
> 	dovecot --version
> 		2.3.11.3 (502c39af9)
> 
> 	solr -version
> 		8.6.3

<snip>

> Attempting to test/debug from from cmd line,
> 
> 	doveadm fts lookup -u myuser at example.com body "tambien"
> 
> causes a PANIC

I am a committer on the lucene-solr project.  So I know that product 
very well.  I am less confident about dovecot, but I do use it.  I do 
not use the fts-solr plugin, because my mail host in AWS does not have 
enough memory for that.

If you are using something like the following schema:

https://raw.githubusercontent.com/dovecot/core/master/doc/solr-schema-7.7.0.xml

That schema does not have anything that would fold accented characters. 
I do see "normalizer-icu" in your dovecot config ... if this filters 
messages before they get to Solr during indexing, then maybe the Solr 
config does not need to do the folding.

Solr does have a set of ICU filters, which I would recommend using 
rather than the lowercase filter, because they are aware of all of 
Unicode.  Those filters are not present in the main Solr distribution, 
but they are in the Solr binary package under "contrib".

I do not have a setup where I can test this.  If I did, I would have 
done that testing.

I cannot say much about the panic you're getting when using the doveadm 
command.  The stacktrace says it is happening in dovecot code, not Solr 
code.  And it looks like the panic had nothing to do with FTS or Solr 
... what I see points to mailbox storage code.

Thanks,
Shawn