Slow reading of large dovecot-uidlist files

[Timo Sirainen]

On 09 Apr 2016, at 21:04, Bostjan Skufca <bostjan at a2o.si> wrote:
> 
> Hi there,
> 
> (context: I was optimizing Roundcube mailbox list server response, and in
> that 300-400ms response time, around 170ms is spent on single fgets() call
> which is waiting IMAP repsonse to "SELECT MyMailbox" command)
> 
> I straced dovecot and of the whole request/response process, around 30ms is
> spent for everything else, and overwhelming majority of time (150-170ms) is
> spent for reading dovecot-uidlist file for given mailbox.
> 
> I skimmed over src/lib-storage/index/maildir/maildir-uidlist.c
> and src/src/lib/istream.c, but I am out of ideas on how to optimize
> mentioned file reading elegantly to make it faster.
> 
> Is there any way to cache parsed content of uidlist file(s) or some other
> obvious quicktrick I am missing to speed this process up?
> 
> Does anyone have any other ideas, how to speed this up?

Switch to mdbox or sdbox format to get better performance. I'm sure there are ways to optimize Maildir too, but it doesn't seem worth the effort since it'll always be slower than mdbox/sdbox.