On 07/02/2015 04:22 PM, Nagy, Attila wrote:
On 05/04/15 17:45, Timo Sirainen wrote:
On 04 May 2015, at 16:59, Nagy, Attila <bra@fsn.hu> wrote:
Hi,
On 05/04/15 10:59, Timo Sirainen wrote:
On 28 Apr 2015, at 23:49, Nagy, Attila <bra@fsn.hu> wrote:
Hi,
imapc does a lot of UID FETCH $UID (BODY.PEEK[]), which is nice, because it works even with the dumbest IMAP server, altough it really kills performance, especially on high latency lines.
I wonder: if IMAP servers can effectively handle boundless fetches (like a list with all wanted UIDs, or simply 1:* if all are needed), do you see this as a good addition to develop? Set mail_prefetch_count = 10 or 100 or something and it'll do larger FETCHes. The higher the value, the more memory/disk space is used for storing the received mails.
I'm aware of that, but it doesn't, or at least not always. For example Thunderbird issues this: 13 UID fetch 333574:333601,333630:333801 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type Reply-To)]) Dovecot does this to the IMAP backend 18 UID FETCH 333574 (BODY.PEEK[]) Oh, there were several bugs related to that. Fixed:
http://hg.dovecot.org/dovecot-2.2/rev/8f20aa806bcc http://hg.dovecot.org/dovecot-2.2/rev/d350a23207c2 http://hg.dovecot.org/dovecot-2.2/rev/8c49fb6d789b I've just tried 2.2.18, which has the above modifications, but the effect is the same. I have these in the config: # doveconf -n | egrep 'prefe|imapc' imapc_features = rfc822.size fetch-headers imapc_host = 10.3.34.12 mail_location = imapc:~/imapc mail_prefetch_count = 100
With an empty imapc directory a POP3 login yields these on the backend IMAP server: 2 LOGIN "user" "pass" 1 LIST "" "" 3 SELECT "INBOX" 4 NOOP 5 UID FETCH 1:* (FLAGS) 6 UID FETCH 2 (RFC822.SIZE) 7 UID FETCH 3 (RFC822.SIZE) 8 UID FETCH 4 (RFC822.SIZE) 9 UID FETCH 5 (RFC822.SIZE) 10 UID FETCH 6 (RFC822.SIZE) 11 UID FETCH 7 (RFC822.SIZE) 12 UID FETCH 8 (RFC822.SIZE) [...] so on, for every message...
The response times for each of the FETCHes are around 13-16 ms, so logging into dovecot for the first time takes more than a minute with 5000 mails in the INBOX and a half day with around a million...
It would be good to have a: 6 UID FETCH 1:* (RFC822.SIZE) or even a: 5 UID FETCH 1:* (FLAGS RFC822.SIZE) if needed (dovecot first fills up its cache, or other cases when the full list is needed)
If dovecot would issue the latter for this 5000-mail inbox, the first fetch (and POP login) would take only 500 ms instead of the 75 seconds that it takes now... Could you please tweak this some more? :) Running 2.2.25 I can observe the same behaviour: Dovecot fetches each message's RFC822.SIZE independently, instead of doing a much more efficient fetch 1:* for all messages. This delays the first login way too long, most clients just time out which Dovecot is busy fetching the sizes and try again and again and again, eating up all resources for some time. Any chance that this will be fixed?
Thanks,