Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"

[Timo Sirainen]

On 21 May 2018, at 14.11, kadafax at gmail.com wrote:
> 
> Le 21/05/2018 à 12:38, Aki Tuomi a écrit :
>> can you try turning on pluign { fts_enforced = yes } and repeat your test?
> 
> Same (wrong) result:
> 1. Send an email with "too6Ouka" in the body
> 
> 2. Search against "too6Ouka":
> # doveadm search -u username mailbox INBOX body too6Ouka
> --> No result
> 
> 3. Force re-index:
> # doveadm fts rescan -u username
> 
> 4. Search again against "too6Ouka":
> # doveadm search -u username mailbox INBOX body too6Ouka
> --> e09cce0283e8695ab760000002deed92 29055
> 
> Don't know if relevant, but on a side note, if I send a second message
> with "too6Ouka" in the body, followed by:
> # doveadm -v index -u username Inbox
> --> doveadm(username): Info: INBOX: Cache is already up to date
> And a search against the pattern immediately return only one result
> instead of two.

So it looks like after "doveadm fts rescan" you can do initial indexing successfully. Afterwards the index isn't updated at all, unless you rebuild it entirely. fts_autoindex=yes appears to work as expected, because after fts rescan you did a search which automatically updated the index. So this doesn't have anything to do with fts_autoindex, just that your index updates don't seem to work at all.

What does it show in the header if you do:
doveadm dump /var/vmail/username/dovecot.index

Especially what is in the "fts" header vs. "next uid" header? Does the UID in "fts" header keep changing every time you save a new mail? I suppose it will. You could also monitor (e.g. tcpdump/wireshark) the network traffic between Dovecot <-> Solr what happens when a new mail arrives. I suspect Dovecot sends it to Solr, which for whatever reason just ignores the update.