On 05/04/15 17:45, Timo Sirainen wrote:
On 04 May 2015, at 16:59, Nagy, Attila <bra@fsn.hu> wrote:
Hi,
On 05/04/15 10:59, Timo Sirainen wrote:
On 28 Apr 2015, at 23:49, Nagy, Attila <bra@fsn.hu> wrote:
Hi,
imapc does a lot of UID FETCH $UID (BODY.PEEK[]), which is nice, because it works even with the dumbest IMAP server, altough it really kills performance, especially on high latency lines.
I wonder: if IMAP servers can effectively handle boundless fetches (like a list with all wanted UIDs, or simply 1:* if all are needed), do you see this as a good addition to develop? Set mail_prefetch_count = 10 or 100 or something and it'll do larger FETCHes. The higher the value, the more memory/disk space is used for storing the received mails.
I'm aware of that, but it doesn't, or at least not always. For example Thunderbird issues this: 13 UID fetch 333574:333601,333630:333801 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type Reply-To)]) Dovecot does this to the IMAP backend 18 UID FETCH 333574 (BODY.PEEK[]) Oh, there were several bugs related to that. Fixed:
http://hg.dovecot.org/dovecot-2.2/rev/8f20aa806bcc http://hg.dovecot.org/dovecot-2.2/rev/d350a23207c2 http://hg.dovecot.org/dovecot-2.2/rev/8c49fb6d789b I've just tried 2.2.18, which has the above modifications, but the effect is the same. I have these in the config: # doveconf -n | egrep 'prefe|imapc' imapc_features = rfc822.size fetch-headers imapc_host = 10.3.34.12 mail_location = imapc:~/imapc mail_prefetch_count = 100
With an empty imapc directory a POP3 login yields these on the backend IMAP server: 2 LOGIN "user" "pass" 1 LIST "" "" 3 SELECT "INBOX" 4 NOOP 5 UID FETCH 1:* (FLAGS) 6 UID FETCH 2 (RFC822.SIZE) 7 UID FETCH 3 (RFC822.SIZE) 8 UID FETCH 4 (RFC822.SIZE) 9 UID FETCH 5 (RFC822.SIZE) 10 UID FETCH 6 (RFC822.SIZE) 11 UID FETCH 7 (RFC822.SIZE) 12 UID FETCH 8 (RFC822.SIZE) [...] so on, for every message...
The response times for each of the FETCHes are around 13-16 ms, so logging into dovecot for the first time takes more than a minute with 5000 mails in the INBOX and a half day with around a million...
It would be good to have a: 6 UID FETCH 1:* (RFC822.SIZE) or even a: 5 UID FETCH 1:* (FLAGS RFC822.SIZE) if needed (dovecot first fills up its cache, or other cases when the full list is needed)
If dovecot would issue the latter for this 5000-mail inbox, the first fetch (and POP login) would take only 500 ms instead of the 75 seconds that it takes now... Could you please tweak this some more? :)
Thanks,