[Dovecot] mbox vs. maildir storage block waste

[Timo Sirainen]

Uh..

On 13.11.2012, at 1.02, Timo Sirainen wrote:

> On 13.11.2012, at 0.44, Robin wrote:
> 
>> On 11/11/2012 5:26 PM, Christoph Anton Mitterer wrote:
>>> Have you made systematic tests? I.e. compared times for all of these
>>> with those from the different dovecot backends.
>> 
>> The choice of Dovecot backends made no substantial difference.  I used maildir, sdbox, and mdbox.  I also added SiS (with mdbox).  Initial tests were on local multi-spindle RAID5 storage,
> 
> With local disks the tests often measure only the local RAM/CPU speed, unless you're testing thousands of users.

..measuring disk I/O most importantly.

>> but to handicap Dovecot, I pushed it over NFS (also Linux 3.2 on a local GigE segment).  It wasn't slow enough to make dbmail competitive, even though you have to start turning off performance optimisation features in Dovecot to avoid NFS bugs.
> 
> NFS makes a better test case if you're measuring single user performance. Much of it is probably due to the index file access latency, although not all. In some cases Dovecot's prefetching mails can help (maildir, sdbox backends with local disks currently, nothing preventing it from working in other use cases though, even with Dovecot-SQL backend).


Prefetching is done only with mail_prefetch_count setting. Someone in blog.dovecot.org mentioned that it was bad for performance with local disk+maildir. Linux apparently doesn't do this with NFS. It would of course be possible to just have the prefetching create a new thread/process to download the mail locally and read it (similar to what the object storage plugin does).