[Dovecot] compressed mboxes very slow

[Kamil Jońca]

Stan Hoeppner <stan at hardwarefreak.com> writes:

> On 5/6/2011 3:07 PM, Kamil Jońca wrote:
>>
>> I have some archive mails in gzipped mboxes. I could use them with
>> dovecot 1.x without problems.
>> But recently I have installed dovecot 2.0.12, and they are slow. very
>> slow.
>>
>> Creating index files takes about 10 minutes for ~20M file with 560
>> messages for bzipped mbox, for gzipped is little better but still
>> unusable :(
>
> What other software, if any, was also upgraded/changed when you
> upgraded to Dovecot 2.0.12?  Libraries?  Filesystem?  Daemons?  What

IIRC only dovecot - I simply upgrade debian package via aptitude.

> they have been mildly corrupted along the way?  Did this bad behavior
> start directly after the upgrade or did 2.0.12 run the zipped mbox
Yes, immediately after upgrade.

> files at acceptable speed for a while?  Did you add/enable any new
> Dovecot plugins that you weren't running in 1.2.x?
>
No, the only thing was converting old config to dovecot-2 config.

>> Stracing dovecot process shows that every ~ 20 messages it rereads
>> complete mbox file.
>
> Can you be a bit more specific here?  What do you mean by "rereads
> complete mbox file"?  I'm not a dev, but that sounds suspiciously like

Sorry, my fault, more correctly is to say: "regularly".

> an error handling mechanism.  I.e. an error occurred while processing,
> or the file may have changed while processing, so we start over.

I'm almost sure that file is not changed.


> Could you have a buggy inotify/dnotify or something along those lines?
How to check it?

> Do you now have something else running say, at the filesystem level,
> that that is making Dovecot think the file has changed even though it
> hasn't?  Are you zipping these mbox files via a cron job that is
> running every few seconds instead of every few hours or days?
No.  These files were compressed once by mutt, and then only read as
archive via dovecot.

>
> Something is apparently causing Dovecot to reread these files
> regularly, and I'd guess it's probably not a Dovecot bug.  Did you run
> strace when accessing a non-compressed mbox file for comparison?

http://strony.aster.pl/kjonca/dovecot.log.gz <- uncompressed mbox
http://strony.aster.pl/kjonca/dovecot.gz.log.gz <- gzipped mbox

KJ


-- 
http://blogdebart.pl/2009/12/22/mamy-chorych-dzieci/
KRETYNIZM - ułomność predysponująca często do wampiryzmu
	(J.Collin de Plancy "Słownik wiedzy tajemnej")