Some time ago I complained about very slow access to compressed mboxes. Unfortunately it looks like that it is very little interest in it, so I have to investigate some things by myself.
Firstly: some rationale. Why do I prefer use mbox/maildir over mdbox. Short answer "bus factor" for support mdbox (not only dovecot) Longer answer: if something goes wrong withm maildir/mbox i can use other tools (mutt, or formail or even text editor) and with mdbox ...
I am not ISP, I use dovecot as a "gateway" to my (rather huge) mail archive. Most of these mails are rather valuable for me, so I prefer use something "well-known-and-tested". (I can't do like most ISP's do: write in "Terms of Service" that mail can be lost or damaged and we give no warranty :) )
So then:
Below my patch. It contains 2 changes:
- when buffer is compressed, we try to save last marked offset.
- Increase temporary buffer for decompression.
without these changes 1.5 GB of bzip compressed mbox with ~20K messages can't be open in 1.5 day After applying 1. change it can be open in ~1.5 h With both changes it was a few minutes.
Maybe it is a good idea to add config parameter to specify size of decompress buffer?
Patch is against v2.0.18
-- Gdyby ktoś miał zbędny Toshiba G450 - to chętnie przejmę ;) < asuffield> a workstation is anything you can stick on somebodies desk and con them into using -- in #debian-devel