[Dovecot] So, what about clustering and load balancing?

[Stan Hoeppner]

Timo Sirainen put forth on 2/13/2010 11:40 AM:
> On 13.2.2010, at 17.48, Stefan Foerster wrote:
> 
>> 1) Use a HA shared storage, export either a cluster filesystem or NFS,
>> and have your dovecot servers mount that file system. Load balance
>> these servers (Cisco ACE, ldirectord, ...) and there you go.
> 
> With NFS if you want to avoid random errors, you still need to do the load balancing in a way that user's mails are never accessed simultaneously from two servers at the same time. See recent "quick question" thread about this. Cluster filesystems should be able to handle this better, although for performance it's probably still a good idea to do this.

Timo, are you saying cluster filesystems aren't suitable for clustered dovecot
storage due to performance reasons?

I've managed a 500 user environment where *EVERYTHING* was hitting on two 2Gb/s
FC SAN storage arrays, one IBM and one Nexsan, through a single Qlogic FC
switch, and these were fairly low end FC arrays.  We had 4 Citrix blades, 3 ESX
blades, and an Exchange blade.  On the ESX blades we were running two AD DCs
plus windows CIFS serving, A LAMP Moodle database server, a dedicated Linux
syslog server, a MS SQL server for Citrix licensing and the organization
accounting application, a Citrix balancing director server, Novel Zen server,
Novell iFolder server, a Linux print server, etc, etc.

All the ESX VM guest storage resided on the SAN and thus they all booted from
SAN.  All bare metal blades booted from SAN as well.  The only blades with
onboard disks were the ESX blades which booted the VMware kernel from mirrored
local disks.  Now, granted none of these systems were sharing the same
filesystem on exported LUNs and sharing metadata over ethernet.  But the overall
load on the SAN controllers was fairly high.

I can't see how the metadata sharing of say GFS2 is going to create any serious
performance impact on a cluster of dovecot servers using GFS2 and a shared SAN
array especially if using maildir.  If the load balancing is implemented
correctly and a given user is only hitting one dovecot server at any one point
in time, there should be few, if any, shared file locks.  Thus, no negative
impact due to shared locking.

A single one of these:
http://www.nexsan.com/sataboy/tech.php

configured with 2GB of cache and 14 x 300GB 10K rpm SATA drives setup at as a
single RAID 10 would yield ~2.1TB of fast redundant mail storage and ~370MB/s of
(controller limited) read throughput using a single controller.  Double that
figure if you go with active-active dual controllers.  Sustained IOPs with 14x
10K rpm drives is about 30K IOPs, which is not bad at all considering this is
the lowest end array Nexsan sells.  Double that to 60K IOPs for dual controllers.

I'd bet I could take this single low end Nexsan array, a low end 8 port Qlogic
4Gb FC switch, and 6 average dual socket servers with 8GB of RAM each and a
single port 4Gb FC adapter, a decent gigabit ethernet switch, using Linux with
GFS2 and Dovecot, and build a cluster that could easily handle a few hundred to
a few thousand concurrent IMAP users and 30K+ total mailiboxen assuming a 500MB
mailbox limit per user.  This hardware configuration should be attainable for
well less than $50K USD.  This doesn't included the front end load balancer.

-- 
Stan