On 3/23/2012 7:13 AM, Jim Lawson wrote:
On 3/23/12 3:13 AM, Stan Hoeppner wrote:
Speaking as an admin who has run Dovecot on top of GFS both with and without the director, I would never go back to a cluster without the director. The cluster performs *so* much better when glocks can be cached on a single node, and this can't happen if a single user has IMAP processes on separate nodes.
No, you don't strictly need the director if you have GFS, but if you can manage it, you'll be a lot happier. Did/do you see the Director/glock benefit with both maildir and mdbox Jim? Do you see any noteworthy performance differences between the two formats on GFS, with and without Director? BTW, are you hitting FC or iSCSI LUNs?
Actually, we're all mbox. This primarily has to do with how users do self-service mail recovery from backup: one folder = one file.
Yeah, mbox isn't as dead as some people contend, but it just doesn't have legs for newer deployment architectures.
I'd like to move to mdbox, but it would mean the recovery scripts will need to understand which files are associated with which folders, as well as restoring the associated index files. That's a to-do.
That's an easy weekend project. ;)
We're using fibrechannel (IBM v7000) storage, but I would expect to see the same thing with iSCSI. It's mostly about different nodes contending over locks on the same files (although I'm sure cache locality helps a great deal, too.) If you end up with imap processes for the same folder on different nodes, or mail delivery happening on one node and imap on the other, you will feel the lag in your IMAP client. "Oh, my INBOX has been unresponsive for 10 seconds, I must be getting a lot of mail right now!" That's an exaggeration, but not by much.
I was asking about your SAN storage unrelated to the locking issue. Just a curiosity thing. Note my email domain. ;) I'm an FC fan but iSCSI seems to be more popular in many circles, actually pretty much market wide these days. So when I come across another SAN user I'm naturally curious as to what hardware they use.
Just so nobody gets the wrong idea, I wasn't advocating against Director earlier in the thread. I think it's fantastic and solves some critical scalability problems. As in your case, it allows one to use his mail storage format of choice with a cluster filesystem while mostly avoiding the locking headaches. In the past one pretty much had to use maildir with a cluster FS to avoid the locking performance killed. But one had to suffer the higher IOPS load on the storage. Not always a good tradeoff, especially for busy mail systems.
I assume you do still have some minor locking/performance issues with the INBOX, even with Director, when LDA and the user MUA are both hitting the INBOX index and mbox files. You'll still see this with mdbox, but probably to a lesser degree if you use a smallish mdbox_rotate_size value. To mitigate this INBOX locking you could go with a dual namespaces, using maildir or sdbox for the INBOX and mdbox for the other user mail folders.
-- Stan