[Dovecot] Performance of Maildir vs sdbox/mdbox

[Stan Hoeppner]

On 1/19/2012 1:18 PM, Timo Sirainen wrote:
> On 19.1.2012, at 6.39, Stan Hoeppner wrote:
> 
>>> You're going to run into NFS caching troubles with the above split
>>> setup. I don't recommend it. You will see error messages about index
>>> corruption with it, and with dbox it can cause metadata loss.
>>> http://wiki2.dovecot.org/NFS http://wiki2.dovecot.org/Director
>>
>> Would it be possible to fix this NFS mdbox index corruption issue in
>> this split scenario by using a dual namespace and disabling indexing on
>> the INBOX?  The goal being no index file collisions between LDA and imap
>> processes.  Maybe something like:
>>
>> namespace {
>>  separator = /
>>  prefix = "#mbox/"
>>  location = mbox:~/mail:INBOX=/var/mail/%u:INDEX=MEMORY
>>  inbox = yes
>>  hidden = yes
>>  list = no
>> }
>> namespace {
>>  separator = /
>>  prefix =
>>  location = mdbox:~/mdbox
>> }
>>
>> Client access to new mail might be a little slower, but if it eliminates
>> the index corruption issue and allows the split architecture, it may be
>> a viable option.
> 
> That assumes that mails are only being delivered to INBOX (i.e. no Sieve or +mailbox addressing). I suppose you could do that if you can live with that limitation. Slightly better for performance would be to not actually keep INBOX mails in mbox format but use snarf plugin to move them to mdbox.
> 
> And of course the above still requires that for imap/pop3 access the user is redirected to the same server every time. I don't really see it helping much.

I spent a decent amount of time last night researching the NFS cache
issue.  It seems there is no way to completely disable NFS client
caching (in lie of rewriting the code oneself--a daunting tak), which
would seem to be the real solution to the mdbox index corruption problem.

So I went looking for alternatives and came up with the idea above.
Obviously it's far from an optimal solution and introduces some
limitations, but I thought it was worth tossing out for discussion.

Timo, it seems that when you designed mdbox you didn't have NFS based
clusters in mind.  Do you consider mdbox simply not suitable for such an
NFS cluster deployment?  If one has no choice but an NFS cluster
architecture, what Dovecot mailbox format do you recommend?  Stick with
maildir?

In this case the OP has Netapp storage.  Netapp units support both NFS
exports as well as iSCSI LUNs.  If the OP could utilize iSCSI instead of
NFS, switching to GFS2 or OCFS, do you see these cluster filesystem as
preferable for mdbox?

-- 
Stan