Stan Hoeppner stan@hardwarefreak.com writes:
On 5/6/2011 3:07 PM, Kamil Jońca wrote:
I have some archive mails in gzipped mboxes. I could use them with dovecot 1.x without problems. But recently I have installed dovecot 2.0.12, and they are slow. very slow.
Creating index files takes about 10 minutes for ~20M file with 560 messages for bzipped mbox, for gzipped is little better but still unusable :(
What other software, if any, was also upgraded/changed when you upgraded to Dovecot 2.0.12? Libraries? Filesystem? Daemons? What
IIRC only dovecot - I simply upgrade debian package via aptitude.
they have been mildly corrupted along the way? Did this bad behavior start directly after the upgrade or did 2.0.12 run the zipped mbox Yes, immediately after upgrade.
files at acceptable speed for a while? Did you add/enable any new Dovecot plugins that you weren't running in 1.2.x?
No, the only thing was converting old config to dovecot-2 config.
Stracing dovecot process shows that every ~ 20 messages it rereads complete mbox file.
Can you be a bit more specific here? What do you mean by "rereads complete mbox file"? I'm not a dev, but that sounds suspiciously like
Sorry, my fault, more correctly is to say: "regularly".
an error handling mechanism. I.e. an error occurred while processing, or the file may have changed while processing, so we start over.
I'm almost sure that file is not changed.
Could you have a buggy inotify/dnotify or something along those lines? How to check it?
Do you now have something else running say, at the filesystem level, that that is making Dovecot think the file has changed even though it hasn't? Are you zipping these mbox files via a cron job that is running every few seconds instead of every few hours or days? No. These files were compressed once by mutt, and then only read as archive via dovecot.
Something is apparently causing Dovecot to reread these files regularly, and I'd guess it's probably not a Dovecot bug. Did you run strace when accessing a non-compressed mbox file for comparison?
http://strony.aster.pl/kjonca/dovecot.log.gz <- uncompressed mbox http://strony.aster.pl/kjonca/dovecot.gz.log.gz <- gzipped mbox
KJ
-- http://blogdebart.pl/2009/12/22/mamy-chorych-dzieci/ KRETYNIZM - ułomność predysponująca często do wampiryzmu (J.Collin de Plancy "Słownik wiedzy tajemnej")