[Dovecot] Doubts about dsync, mdbox, SIS

[Jan-Frode Myklebust]

On Thu, Feb 02, 2012 at 08:46:55AM +0100, Alessio Cecchi wrote:
> 
> How many users there are in this installation?

Quite a few :-) This is for an ISP.

> >The active servers are using Maildir, and has:
> >
> >	$ df -h /usr/local/atmail/users/
> >	Filesystem            Size  Used Avail Use% Mounted on
> >	/dev/atmailusers       14T   12T  2.2T  85% /usr/local/atmail/users
> >	$ df -hi /usr/local/atmail/users/
> >	Filesystem            Inodes   IUsed   IFree IUse% Mounted on
> >	/dev/atmailusers        145M    113M     33M   78% /usr/local/atmail/users
> >
> >very little of this is compressed (zlib plugin enabled during christmas).
> 
> This is the old storage in Maildir format?

Correct.

> 
> >I'm surprised that the destination server is so large, was expecting zlib and
> >mdbox and SIS would compress it down to much less than what we're seeing
> >(12TB ->  5TB):
> >
> >	$ df -h /srv/mailbackup
> >	Filesystem            Size  Used Avail Use% Mounted on
> >	/dev/mapper/mailbackupvg-mailbackuplv
> >                       5.7T  4.8T  882G  85% /srv/mailbackup
> 
> This is the new storage in mdbox format?

Correct.

> What size you would expect?

With Maildir I see message-files shrink to about 20%* of original size
after turning on zlib with zlib_save_level=6. I was expecting better
compression with mdbox (and zlib_save_level=9), and I would expect SIS to
help even further.

mdbox+SIS+zlib_save_level=9 variant taking up 40% the space of a mixed**
compressed/non-compressed Maildir storage isn't very impressive to me --
and the mdbox backup isn't even complete (it's only the 25% most active users).

Yes, I see there might be holes in my logic, expecting compressed messages to
compress further after move to mdbox. But also I have expectation that
most of the messages are not already compressed on the Maildir side.
Sorry, expectations and guesses, not hard facts.

[*] based on a couple of samples, not thourough research
[**] Only messages saved after we enabled zlib on December 25. are compressed.


  -jf