Searching 30 GB mailbox

[Sami Ketola]

> On 2. Dec 2021, at 10.57, Einar Bjarni Halldórsson <einar at isnic.is> wrote:
> root at ht-mailstore01:/data/mail/hostmasterlog/mdbox/mailboxes/INBOX/dbox-Mails # doveadm dump . | grep -c hdr.MESSAGE-ID
> 4464736
> 
> In dovecot config I have `fts_enforced = yes` and after the search for Message-ID finishes and returns UID, I can do a FETCH .. ENVELOPE on it and it responds immediately.
> I was under the impression that fts_enforced forces all searches, headers and body, to go to solr. Then all dovecot would have to do was to return the UID returned by solr.
> Unless solr doesn't return UID and dovecot has to take the result from solr and lookup the UID, and with a full 1 GB cache file it always has to scan the whole index? 
> 
> Nothing seems to be completely broken, I always receive a result, it's just that it takes 30 seconds when I really want it to be ~5 seconds at most.
> 
> I guess if we can't find a solution and 30 seconds becomes a real problem, we'll split the mailbox up by years. It should help with the size of the cache. It makes the searching code a little more complicated since it has to figure out the sent date before it can search, but it's doable.

If you have fts_enforced = yes then that SEARCH probably goes to solr even if message-id is found from cache. So the delay is probably on solr side.

Can you try to remove fts_enforced for that one particular user?

Sami