On 19/10/2020 17:00, PGNet Dev wrote:
On 10/19/20 1:18 AM, John Fawcett wrote:
I would recommend you to redo the tests after correcting the configuration. To be doubly sure you can include accented and unique non accented text in the same email and search for both. If the non accented text is found you know you've searching against the updated index and the fact that accented text is not found is not simply because the index updates are not visible.
temp changing,
fts_solr = url=https://solr.example.com:8984/solr/dovecot/ use_libfts soft_commit=yes batch_size=250 - fts_enforced = body + fts_enforced = yes - fts_filters = normalizer-icu snowball stopwords + fts_filters = lowercase snowball stopwords
now,
doveadm fts rescan -u testuser@example.com doveadm index -u testuser@example.com -q '*'
, no errors.
on receipt of a test email with
subject: aausdfrhyétdwgyatrdf aausdfrhyetdwgyatrdf body: aausdfrhyétdwgyatrdf aausdfrhyetdwgyatrdf
i see a solr auto-scan,
2020-10-19 14:41:50.628 INFO (searcherExecutor-15-thread-1-processing-x:dovecot) [ x:dovecot] o.a.s.c.SolrCore [dovecot] Registered new searcher autowarm time: 1 ms 2020-10-19 14:41:50.634 INFO (qtp1533985074-18) [ x:dovecot] o.a.s.u.p.LogUpdateProcessorFactory [dovecot] webapp=/solr path=/update params={}{commit=} 0 155 2020-10-19 14:41:51.571 INFO (qtp1533985074-24) [ x:dovecot] o.a.s.u.p.LogUpdateProcessorFactory [dovecot] webapp=/solr path=/update params={}{add=[135785/c92f64f79f0d1ed01e6d5b314f04886c/testuser@example.com (1680991596381732864)]} 0 9
==> /var/log/solr/solr_gc.log <== [2020-10-19T07:41:51.612-0700][146823.946s] GC(343) Pause Young (Normal) (G1 Evacuation Pause) [2020-10-19T07:41:51.613-0700][146823.947s] GC(343) Using 2 workers of 2 for evacuation [2020-10-19T07:41:51.635-0700][146823.969s] GC(343) Pre Evacuate Collection Set: 0.3ms [2020-10-19T07:41:51.636-0700][146823.970s] GC(343) Merge Heap Roots: 0.2ms [2020-10-19T07:41:51.636-0700][146823.970s] GC(343) Evacuate Collection Set: 18.9ms [2020-10-19T07:41:51.637-0700][146823.971s] GC(343) Post Evacuate Collection Set: 2.0ms [2020-10-19T07:41:51.637-0700][146823.971s] GC(343) Other: 1.5ms [2020-10-19T07:41:51.637-0700][146823.971s] GC(343) Eden regions: 238->0(244) [2020-10-19T07:41:51.638-0700][146823.972s] GC(343) Survivor regions: 4->2(31) [2020-10-19T07:41:51.638-0700][146823.972s] GC(343) Old regions: 189->189 [2020-10-19T07:41:51.638-0700][146823.973s] GC(343) Archive regions: 2->2 [2020-10-19T07:41:51.639-0700][146823.973s] GC(343) Humongous regions: 10->9 [2020-10-19T07:41:51.639-0700][146823.973s] GC(343) Metaspace: 61564K(78028K)->61564K(78028K) NonClass: 55348K(65024K)->55348K(65024K) Class: 6216K(13004K)->6216K(13004K) [2020-10-19T07:41:51.640-0700][146823.974s] GC(343) Pause Young (Normal) (G1 Evacuation Pause) 441M->200M(512M) 27.372ms [2020-10-19T07:41:51.640-0700][146823.974s] GC(343) User=0.01s Sys=0.01s Real=0.03s
==> /var/log/solr/solr.log <== 2020-10-19 14:41:51.702 INFO (searcherExecutor-15-thread-1-processing-x:dovecot) [ x:dovecot] o.a.s.c.SolrCore [dovecot] Registered new searcher autowarm time: 0 ms 2020-10-19 14:41:51.705 INFO (qtp1533985074-18) [ x:dovecot] o.a.s.u.p.LogUpdateProcessorFactory [dovecot] webapp=/solr path=/update params={}{commit=} 0 127
search in TBird
subject: aausdfrhyetdwgyatrdf => FOUND body: aausdfrhyétdwgyatrdf => FOUND
subject: aausdfrhyetdwgyatrdf => FOUND body: aausdfrhyétdwgyatrdf => (emtpy)
on header search, I'm _not_ seeing any additional activity in solr.log
so, either i'm looking in the wrong place, haven't turned on appropriate logging, or i'm still not searching via solr ...
separately,
doveadm fts lookup ...
still panics; Aki's bug will hopefully deal with that
Depending how solr has been setup you could see the logging in the web server access log. My access log is where I configured it in /var/log/httpd/servername.access_log, yours may be different.
For searches I see things like this (one for each folder searched)
2a01:488:67:1000:523:f8eb:0:1 - john [19/Oct/2020:17:16:39 +0200] "GET /solr/dovecot/select?wt=xml&fl=uid,score&rows=3176&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7dbody:aausdfrhy%c3%a9tdwgyatrdf&fq=%2Bbox:b1626f0fe8d9145e54100000c54a863a+%2Buser:john@voipsupport.it HTTP/1.1" 200 910 "-" "-"
For index updates I see things like this:
2a01:488:67:1000:523:f8eb:0:1 - john [19/Oct/2020:17:10:01 +0200] "POST /solr/dovecot/update HTTP/1.1" 200 156 "-" "-"
BTW I've noticed that Thunderbird does not always send the search to the server even with the "search on server" flag set, which is why I look at the access_log in solr if I want to be really sure it's going to solr.
John