[Dovecot] Compressing existing maildirs

[Stan Hoeppner]

On 12/29/2011 2:49 AM, Jan-Frode Myklebust wrote:
> On Wed, Dec 28, 2011 at 03:56:33PM -0800, Dovecot-GDH wrote:
>> The cleanest (though not necessarily simplest) way to go about this would be to use dsync to create a new maildir and incrementally direct traffic to a separate Dovecot instance.
>>
>> Unless you have a legacy application that relies on maildir, switching to mdbox would be a good idea too.
> 
> We just got rid of the legacy app that worked directly against the
> maildirs, which is the reason we now can turn on compression. I
> intend to switch to mdbox, but first I need to free up some disks by
> compressing the existing maildirs (12 TB maildirs, should probably
> compress down to less than half).

How much additional space do you expect the conversion process to
compressed mdbox to consume?  It shouldn't need much.  Using dsync, the
conversion will be done one mailbox at a time and the existing emails
will be compressed when written into the new mdbox mailbox.

After you've converted a few mailboxes by hand and have confirmed you're
happy with the results, simply add commands to your bulk conversion
script to delete each user maildir and contents after the new mdbox
mailbox has been created and populated.  Using this method shouldn't
require much more additional filesystem space than that equal to your
largest single user maildir.

Given your 12TB of mailstore, I'd convert users in small batches over a
period of weeks or a month, depending on your total mailbox count.
Firing up a conversion script and having it run non-stop until all 12TB
are converted is probably asking for trouble due to many factors I
shouldn't need to put down here.  Time your first few manual
conversions.  Divide that average time into your daily off-peak hours so
you know approximately how many mailboxes you can convert during
off-peak hours.  Run your script daily against these small sets of
mailboxes until the entire process is complete.

-- 
Stan