[Dovecot] compressed mboxes very slow
Stan Hoeppner
stan at hardwarefreak.com
Sun May 8 00:37:53 EEST 2011
On 5/6/2011 3:07 PM, Kamil Jońca wrote:
>
> I have some archive mails in gzipped mboxes. I could use them with
> dovecot 1.x without problems.
> But recently I have installed dovecot 2.0.12, and they are slow. very
> slow.
>
> Creating index files takes about 10 minutes for ~20M file with 560
> messages for bzipped mbox, for gzipped is little better but still
> unusable :(
What other software, if any, was also upgraded/changed when you upgraded
to Dovecot 2.0.12? Libraries? Filesystem? Daemons? What OS/version?
Was the OS upgraded? Is this a new machine as well as new software?
If so how did you copy the files to the new system? Could they have
been mildly corrupted along the way? Did this bad behavior start
directly after the upgrade or did 2.0.12 run the zipped mbox files at
acceptable speed for a while? Did you add/enable any new Dovecot
plugins that you weren't running in 1.2.x?
> Stracing dovecot process shows that every ~ 20 messages it rereads
> complete mbox file.
Can you be a bit more specific here? What do you mean by "rereads
complete mbox file"? I'm not a dev, but that sounds suspiciously like
an error handling mechanism. I.e. an error occurred while processing,
or the file may have changed while processing, so we start over. Could
you have a buggy inotify/dnotify or something along those lines? Do you
now have something else running say, at the filesystem level, that that
is making Dovecot think the file has changed even though it hasn't? Are
you zipping these mbox files via a cron job that is running every few
seconds instead of every few hours or days?
Something is apparently causing Dovecot to reread these files regularly,
and I'd guess it's probably not a Dovecot bug. Did you run strace when
accessing a non-compressed mbox file for comparison?
--
Stan
More information about the dovecot
mailing list