On 28/05/2018 13:04, Timo Sirainen wrote:
On 28 May 2018, at 13.28, kfx <kadafax@gmail.com <mailto:kadafax@gmail.com>> wrote:
Especially what is in the "fts" header vs. "next uid" header? Does the UID in "fts" header keep changing every time you save a new mail? I suppose it will.
Diff between 2 emails: next uid = 30104 | next uid = 30105 last_indexed_uid = 30103 | last_indexed_uid = 30104
So Dovecot thinks it has indexed everything.
You could also monitor (e.g. tcpdump/wireshark) the network traffic between Dovecot <-> Solr what happens when a new mail arrives. I suspect Dovecot sends it to Solr, which for whatever reason just ignores the update.
################### TCPDUMP #################################### POST /solr/dovecot/update HTTP/1.1 Host: localhost:8983 Date: Mon, 28 May 2018 10:18:05 GMT Transfer-Encoding: chunked Connection: Keep-Alive Content-Type: text/xml
<add><doc><field name="uid">37581</field><field name="box">e0c58a3093235153110800003ea484a8</field><field name="user">username</field><field name="id">37581/e0c58a3093235153110800003ea484a8/username</field><field name="body">Search Pattern: Kai8oovi .. </field></doc></add>
And Dovecot sends the mail.
######### SOLR'S RESPONSE ############### <?xml version="1.0" encoding="UTF-8"?> <response>
<lst name="responseHeader"> <int name="status">0</int> <int name="QTime">0</int> </lst> </response>
And Solr receives it. Your tcpdump doesn't show <commit softCommit="true" waitSearcher="true"/> being sent though. Do you see it being sent anywhere?
Yes:
################### TCPDUMP ################################### POST /solr/dovecot/update HTTP/1.1 Host: localhost:8983 Date: Mon, 28 May 2018 10:18:05 GMT Expect: 100-continue Content-Length: 47 Connection: Keep-Alive Content-Type: text/xml
<commit softCommit="true" waitSearcher="true"/> ################### /TCPDUMP ###################################
Does it make the mails visible if you run it yourself? Or if you run hard commit? :
# curl http://127.0.0.1:8983/solr/update?commit=true <html>
[0/0]
<head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Not Found</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/update. Reason: <pre> Not Found</pre></p> </body> </html>
# curl http://127.0.0.1:8983/solr/dovecot/update?commit=true <?xml version="1.0" encoding="UTF-8"?> <response>
<lst name="responseHeader"> <int name="status">0</int> <int name="QTime">0</int> </lst> </response>
# doveadm search -u username mailbox INBOX body Kai8oovi ==> No result ('Kai8oovi' is the search pattern, it should returns 4 results)