sending to the list as well
On 02/11/2020 17:40, PGNet Dev wrote:
On 11/1/20 10:35 AM, John Fawcett wrote:
Yes, getting more data about any potential problem would be useful.
Just to clarify: I have a fully working search setup for some time now over various dovecot releases, so no patches needed to get it working.
My setup does use fts plugin and fts-solr plugin, but it does not use lib-fts functionality (that has many features for example it was stopping you indexing excluded words like tambien). On my setup without lib-fts everything goes to solr which does the work of indexing without all the features of lib-fts.
withOUT libfts
- fts_solr = url=https://solr.presence-group.net:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 + fts_solr = url=https://solr.presence-group.net:8984/solr/dovecot/ soft_commit=yes batch_size=250
and unmodified dovecot-provided schema/config,
/bin/cp -af /usr/share/doc/dovecot/solr-config-7.7.0.xml /path/to/solr/data/dovecot/conf/solrconfig.xml /bin/cp -af /usr/share/doc/dovecot/solr-schema-7.7.0.xml /path/to/solr/data/dovecot/conf/schema.xml
i suspect my config's now more similar to yours.
checking,
doveadm fts rescan -u testuser@example.com doveadm index -u testuser@example.com -q '*'
as before
doveadm fts lookup -u testuser@example.com body "tésting"
panics,
doveadm(testuser@example.com): Panic: file mail-storage.c: line 2112 (mailbox_get_open_status): assertion failed: (box->opened) doveadm(testuser@example.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x7f7829b81cc6] -> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x7f7829b81de2] -> /usr/lib64/dovecot/libdovecot.so.0(+0x10025b) [0x7f7829b8b25b] -> /usr/lib64/dovecot/libdovecot.so.0(+0x100297) [0x7f7829b8b297] -> /usr/lib64/dovecot/libdovecot.so.0(+0x59bc6) [0x7f7829ae4bc6] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x4779e) [0x7f7829c9879e] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x5849) [0x7f78296ea849] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_backend_lookup+0x51) [0x7f782930b7c1] -> /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so(+0x3280) [0x7f78270d0280] -> doveadm(+0x343cd) [0x55aa57edc3cd] -> doveadm(+0x34fe0) [0x55aa57edcfe0] -> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x22d) [0x55aa57edde2d] -> doveadm(doveadm_cmd_run_ver2+0x4e8) [0x55aa57eee8d8] -> doveadm(doveadm_cmd_try_run_ver2+0x3e) [0x55aa57eee92e] -> doveadm(main+0x1d4) [0x55aa57ecccf4] -> /lib64/libc.so.6(__libc_start_main+0xf2) [0x7f7829746042] -> doveadm(_start+0x2e) [0x55aa57ecd1ce] Aborted
but search, even for accented characters,
doveadm search -u testuser@example.com subject "tésting" 42d73837f133a05fad4d0000f8839f03 1 813ef60e984f1b5f5fc200005439fba4 293
doveadm search -u testuser@example.com body "tésting" ba899d0cfe33a05fbe4d0000f8839f03 1 813ef60e984f1b5f5fc200005439fba4 293
appears to work.
next, to get tokenization -- at least email/url (UAX29URLEmailTokenizer) -- and lowercase & icu normalization working and verified.
the panic on doveadm fts lookup is to be expected and solved by my previous patch.
I think you've now got a config very similar to mine. One last check I did was to search for the same string with the grave accent. e.g. tèsting, so that should NOT be found. That was just proof that it was actually searching for accents and not folding them to e.
I saw Aki's advice about removing use_tls because of issues, and I know I'm repeating myself here :-) but so far I have seen no concrete evidence of issues on your original setup. Doesn't mean there aren't, but until there is evidence, can't do much towards solving them.
John
John