<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">In some cases (exact condition still unknown) dovecot sends binary data (attachments) to SOLR for indexing. This reduces index and overall FTS efficiency dramatically. <div class=""><br class=""></div><div class="">In extreme condition (below an example of 20MB) dovecot’s hardwired timeout of 60s gets triggered during HTTP exchange with SOLR on just a single file. This results in an unfinished index which, by initial indexing, gets restarted over and over. With multiple affected mailboxes even on moderate usage this can cause an IO overload of the whole system.</div><div class=""><br class=""></div><div class="">Message example (doveadm fetch text): <a href="https://filebin.ca/5oy5Wc1QrBK3/fetch-text.obfuscated.txt" class="">https://filebin.ca/5oy5Wc1QrBK3/fetch-text.obfuscated.txt</a></div><div class="">Corresponding raw log data: <a href="https://filebin.ca/5oy6yqLSCr3H/rawlog.obfuscated.txt" class="">https://filebin.ca/5oy6yqLSCr3H/rawlog.obfuscated.txt</a></div><div class=""><br class=""></div><div class="">(Both files were processed with perl <a href="https://www.dovecot.org/tools/doveadm-obfuscate.pl" class="">doveadm-obfuscate.pl</a>; the script doesn’t replace non-latin characters so they were replaced with ‘R’ manually)</div><div class=""><br class=""></div><div class="">Workaround: there is a useful <a href="https://www.mail-archive.com/dovecot@dovecot.org/msg82296.html" class="">patch by John Fawcett </a> that allows to set the FTS indexing message body maximum size. It works perfectly, but affected messages are getting completely ignored by FTS.</div><div class=""><br class=""></div><div class="">This bug report is a summarised result of <a href="https://www.mail-archive.com/dovecot@dovecot.org/msg82599.html" class="">this discussion</a>. </div></div></div></body></html>