[Dovecot] distributed mdbox

[Stan Hoeppner]

On 3/23/2012 7:13 AM, Jim Lawson wrote:
> On 3/23/12 3:13 AM, Stan Hoeppner wrote:
> 
>>> Speaking as an admin who has run Dovecot on top of GFS both with and
>>> without the director, I would never go back to a cluster without the
>>> director.  The cluster performs *so* much better when glocks can be
>>> cached on a single node, and this can't happen if a single user has IMAP
>>> processes on separate nodes.
>>>
>>> No, you don't strictly need the director if you have GFS, but if you can
>>> manage it, you'll be a lot happier.
>> Did/do you see the Director/glock benefit with both maildir and mdbox
>> Jim?  Do you see any noteworthy performance differences between the two
>> formats on GFS, with and without Director?  BTW, are you hitting FC or
>> iSCSI LUNs?
>>
> 
> Actually, we're all mbox.  This primarily has to do with how users do
> self-service mail recovery from backup: one folder = one file.

Yeah, mbox isn't as dead as some people contend, but it just doesn't
have legs for newer deployment architectures.

> I'd like to move to mdbox, but it would mean the recovery scripts will
> need to understand which files are associated with which folders, as
> well as restoring the associated index files.  That's a to-do.

That's an easy weekend project. ;)

> We're using fibrechannel (IBM v7000) storage, but I would expect to see
> the same thing with iSCSI.  It's mostly about different nodes contending
> over locks on the same files (although I'm sure cache locality helps a
> great deal, too.)  If you end up with imap processes for the same folder
> on different nodes, or mail delivery happening on one node and imap on
> the other, you will feel the lag in your IMAP client.  "Oh, my INBOX has
> been unresponsive for 10 seconds, I must be getting a lot of mail right
> now!"  That's an exaggeration, but not by much.

I was asking about your SAN storage unrelated to the locking issue.
Just a curiosity thing.  Note my email domain. ;)  I'm an FC fan but
iSCSI seems to be more popular in many circles, actually pretty much
market wide these days.  So when I come across another SAN user I'm
naturally curious as to what hardware they use.

Just so nobody gets the wrong idea, I wasn't advocating against Director
earlier in the thread.  I think it's fantastic and solves some critical
scalability problems.  As in your case, it allows one to use his mail
storage format of choice with a cluster filesystem while mostly avoiding
the locking headaches.  In the past one pretty much had to use maildir
with a cluster FS to avoid the locking performance killed.  But one had
to suffer the higher IOPS load on the storage.  Not always a good
tradeoff, especially for busy mail systems.

I assume you do still have some minor locking/performance issues with
the INBOX, even with Director, when LDA and the user MUA are both
hitting the INBOX index and mbox files.  You'll still see this with
mdbox, but probably to a lesser degree if you use a smallish
mdbox_rotate_size value.  To mitigate this INBOX locking you could go
with a dual namespaces, using maildir or sdbox for the INBOX and mdbox
for the other user mail folders.

-- 
Stan