[Dovecot] Performance of Maildir vs sdbox/mdbox

[Javier Miguel Rodríguez]

     Spanish edu site here, 80k users, 4,5 TB of email, 6.000 iops 
(indexes) + 9.000 iops (mdboxes) in working hours here.

     We evaluated mdbox against Maildir and we found that with these 
setting dovecot 2 perfoms better than Maildir:

mdbox_rotate_interval = 1d
mdbox_rotate_size=60m
zlib_save_level = 9 # 1..9
   zlib_save = gz # or bz2

     We detected 40% less iops with this setup *in working hours (more 
info below)*. Zlib saved some writes (15-30%). With mdbox, deletion of a 
message is written to indexes (use SSD for this), and a nightly cronjob 
deletes the real message from the mdbox, this saves us some iops in 
working hours. Also, backup software is MUCH happier handling hundreds 
of thousands files (mdbox) versus tens of millions (maildir)

     Mdbox has also drawbacks: you have to be VERY careful with your 
indexes, they contain data that can not be rebuilt from mdboxes. The 
nightly cronjob "purging" the mdboxes hammers the SAN. Full backup time 
is reduced, but incremental backup space & time increases: if you delete 
a message, after "purging" it from the mdbox the mdbox file changes 
(size and date), so the incremental backup has to copy it again.

     Regards

     Javier