FTS Dealing badly with Tika failures

[Andreas Thienemann]

Hi,

I've seen that latest Dovecot has the ability to use the Apache Tika 
Server to extract metadata from attachments.

That is a pretty useful feature and looks to be an improvement over piping 
the attachment through a shellscript as done with fts_decoder.

Having this feature enabled though I sometimes do see the situation that 
the tika service does not reply in time and it looks like the whole FTS 
transaction gets aborted:

dovecot: indexer-worker(eggs at localhost): Error: fts_tika: PUT 
http://localhost:9998/tika/ failed: Request timed out (Request queued 
60.145 secs ago, 1 attempts in 60.134 secs, 60.034 in http ioloop, 0.000 
in other ioloops, connected 1739.235 secs ago)

dovecot: indexer-worker(eggs at localhost): Error: Mailbox INBOX.junk: Mail 
search failed: Internal error occurred. Refer to server log for more 
information. [2017-10-09 03:46:09]

dovecot: indexer-worker(eggs at bawue.de): Error: Mailbox INBOX.junk: 
Transaction commit failed: FTS transaction commit failed: transaction 
context (attempted to index 12084 messages (UIDs 2..17850))


Is that actually sensible behavior? I would consider attachment metadata 
to be nice to have, but not necessarily mandatory. So if the attachment 
indexing fails, I would hope that at least the mailtext got indexed. But 
it seems like we abort the whole transaction.

Am I reading the error log correctly?

cheers,
  Andreas