Searching 30 GB mailbox

[Sami Ketola]

> On 1. Dec 2021, at 0.30, Einar Bjarni Halldórsson <einar at isnic.is> wrote:
> 
> Hi,
> 
> We're migrating our mailboxes from Zimbra to dovecot, running on FreeBSD 12.2. We're using mdbox and solr for fts. dovecot version 2.3.17.
> 
> One of our mailboxes gets a bcc of every outgoing mail from our system and it currently holds ~4.5 million messages taking 30 GB.
> 
> One of the ways we use the mailbox is that when outgoing mail is sent, we store the Message-ID in our database, and in our admin interface we can open the mail and it's fetched from IMAP by doing a 'SEARCH HEADER Message-ID <MSG_ID>'.
> 
> In Zimbra this was reasonably fast, but after migrating the mailbox to dovecot it's 30 seconds which I think we can reduce.
> 
> After turning on mail_debug I see this in our log https://pastebin.com/kP9H7gSB
> 
> Looks like solr finishes searching instantly, then there's ~30 seconds of silence in the logs, and then more log lines about fetching data from mail cache.
> 
> The same thing happens if I use telnet and just issue a 'SEARCH HEADER...' command.
> 
> How can I find out what dovecot is doing in those 30 seconds?


Did you have Messsage-ID listed in mail_always_cache_fields when you imported the mailbox?
Do you have it now? 

If not, please add it and then use doveadm fetch to populate that in cache.

I think the correct setting is

mail_always_cache_fields = hdr.message-id

But you also might want to add some other fields

Sami