Slow reading of large dovecot-uidlist files
skdovecot at smail.inf.fh-brs.de
Tue Apr 12 12:04:42 UTC 2016
-----BEGIN PGP SIGNED MESSAGE-----
On Tue, 12 Apr 2016, Bostjan Skufca wrote:
> On 12 April 2016 at 10:23, A.L.E.C <alec at alec.pl> wrote:
>> I don't know dovecot's code, but I suppose it uses uidlist file to get
>> mailbox statistics that it returns as EXISTS, RECENT, UNSEEN, UIDNEXT,
>> UIDVALIDITY, etc, which are required by IMAP standard. I don't know,
>> maybe they could be stored in more optimized way, but I think in most
>> cases this data is also needed for SORT/THREAD/FETCH which is sent after
>> SELECT in many cases - so it will be needed anyway. There are cases
>> (e.g. mailbox synchronization) when you indeed do only SELECT.
> (Oh, Alec, hi :)
> Can someone (who is more intimate with Dovecot's internals) comment on what
> dovecot is actually doing, where this time is being spent? That would be
> My uidfile is 4MB "large" and reading it takes about 3ms on my system, so
> there is still ~160ms (98%) of dovecot's delay that is unaccounted for.
You could try to trace Dovecot and see if there are a lot of syscalls when
a new connection starts up, e.g. with strace or dtruss.
> BTW: I would have thought that the whole process goes like this:
> - user logs in, dovecot imap process starts
> - "SELECT MyMailbox" is issued for the first time
> - dovecot reads uidlist file for that mailbox and caches it along with
> timestamp metadata
does your mailboxes change without Dovecot? Do you have some script or
something like that that causes the mtime of the directory change? If
Dovecot thinks an external program changed the mailbox, it validates all
messages in cache and from store.
> - on subsequent requests, uidlist file is stat-ed to detect any changes
> (changed by i.e. another process that handles same user's connection, from
> another device) and trigger rereading when needed, otherwise cache is used
> - I would imagine having certain (fairly low) TTL on the cache to prevent
> too much memory consumption would be nice, too.
> Would that make sense?
> (I am just trying to figure out the way how to optimize response time to
> SELECT and thus make rendering mailbox listing in Roundcube faster, or,
> better, acceptable.)
>> Aleksander 'A.L.E.C' Machniak
>> Kolab Groupware Developer [http://kolab.org]
>> Roundcube Webmail Developer [http://roundcube.net]
>> PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
-----END PGP SIGNATURE-----
More information about the dovecot