Slow reading of large dovecot-uidlist files

[Bostjan Skufca]

On 12 April 2016 at 10:23, A.L.E.C <alec at alec.pl> wrote:

> I don't know dovecot's code, but I suppose it uses uidlist file to get
> mailbox statistics that it returns as EXISTS, RECENT, UNSEEN, UIDNEXT,
> UIDVALIDITY, etc, which are required by IMAP standard. I don't know,
> maybe they could be stored in more optimized way, but I think in most
> cases this data is also needed for SORT/THREAD/FETCH which is sent after
> SELECT in many cases - so it will be needed anyway. There are cases
> (e.g. mailbox synchronization) when you indeed do only SELECT.
>

(Oh, Alec, hi :)

Can someone (who is more intimate with Dovecot's internals) comment on what
dovecot is actually doing, where this time is being spent? That would be
awesome:)

My uidfile is 4MB "large" and reading it takes about 3ms on my system, so
there is still ~160ms (98%) of dovecot's delay that is unaccounted for.

BTW: I would have thought that the whole process goes like this:
- user logs in, dovecot imap process starts
- "SELECT MyMailbox" is issued for the first time
- dovecot reads uidlist file for that mailbox and caches it along with
timestamp metadata
- on subsequent requests, uidlist file is stat-ed to detect any changes
(changed by i.e. another process that handles same user's connection, from
another device) and trigger rereading when needed, otherwise cache is used
- I would imagine having certain (fairly low) TTL on the cache to prevent
too much memory consumption would be nice, too.

Would that make sense?

(I am just trying to figure out the way how to optimize response time to
SELECT and thus make rendering mailbox listing in Roundcube faster, or,
better, acceptable.)


Best,
b.



>
> --
> Aleksander 'A.L.E.C' Machniak
> Kolab Groupware Developer        [http://kolab.org]
> Roundcube Webmail Developer  [http://roundcube.net]
> ---------------------------------------------------
> PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl
>