v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed) [proposed patch]

[PGNet Dev]

On 11/1/20 1:56 AM, John Fawcett wrote:
> At the moment I don't see other corrections needed in dovecot apart from
> command line doveadm fts which is not a show stopper. Via doveadm search
> I confirm - on my simple config - that search for accented or non
> accented characters works correctly as it does via imap connection.

thx.  hopefully it'll get considered for a next release soon.

> Only thing I cannot vouch for is bringing dovecot fts library and config
> into the equation because my setup delegates almost everything to solr.

do i understand correctly that you're solr-indexing your dovecot mail store withOUT using dovecot fts plugin, and that -- with your aforementioned patch -- doveadm successfully uses the resulting indexes?

i hadn't yet seriously considered _circumventing_ fts plugin; if this^ does get resolved soonish, then it's not a big deal.  if not, an fts-plugin-less setup would be interesting to know more abt!

> Can you get evidence of things not working? For example tests run with
> soft_commit configured - that's important since without it the updates
> don't show up immediately in searches, that do show that the update is
> happening in solr via solr log, but then search is not working on
> accented characters, despite it working on other text in the same
> message? The solr logs also show whether the text was found or not via
> the "hits=" value in the logged searches, for example:
> 
> 2020-11-01 08:32:42.231 INFO  (qtp24119573-21) [   x:dovecot]
> o.a.s.c.S.Request [dovecot]  webapp=/solr path=/select
> params={q={!lucene+q.op%3DAND}body:también&fl=uid,score&sort=uid+asc&fq=%2Bbox:b1626f0fe8d9145e54100000c54a863a+%2Buser:john at voipsupport.it&rows=3202&wt=xml}
> hits=3 status=0 QTime=3
> 
> But if no hits are found, then dovecot cannot be expected to display
> results. It still may be an indexing problem though.

my current config has soft_commit enabled,

   fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250

i'll see abt getting some clearer test results ...