[Dovecot] Basic clustered filesystem advice

[List]

On 9/17/13 3:23 PM, Andreas Gaiser wrote:
> Does anybody know about GlusterFS & Dovecot?
>
>
> ...Andreas
>
>> Time marches on, and I need to continue the service migration. I'd still
>> like to use Dovecot (we're migrating away from Cyrus).  I'm assuming the
>> only other alternative without existing shared storage is to use DRBD
>> and a cluster file system to provide the replication, and to ensure
>> Director is enabled.  Are there any things to watch for surrounding
>> this?
We tested glusterfs 3.2 a while ago using four storage nodes, four 
Dovecot/Postfix machines, and a number of email client bots that 
generated upwards of a 180k inbound messages per hour, and upwards of 
360k pop/imap connections per hour.  Unfortunately we did not grab any 
metrics on how long it took for a POP/IMAP session to 
open/read/delete/close or how long SMTP transactions took, we simply 
wanted to see how much load would be generated which was reasonable for 
the machines we used.  All storage and mail machines were virtual 
(vmware) and consisted of 2vcpu with 8gigs mem running Centos 6.1.  We 
tested both NFS and the gluster native client and didn't see much 
difference in perceived load on the system.  We did not run into any of 
the issues that are common with running Dovecot over NFS during our 
testing, which we attribute to a proper configuration for NFS and solid 
NTP.  We ran an extended test that lasted for about two months and 
nothing really hiccuped or failed to function so I would call it a 
success to that extent.

We also tested stretching the glusterfs cluster between our two data 
centers which are 100 miles apart as the fiber lays.  Our latency is 
very low and stable between sites, and resulted in a small increase in 
load on the cluster.  I would not recommend this concept over anything 
but the most stable and fault tolerant WAN imaginable, but it seemed to 
work reasonably well for the duration of the testing we did (about a day 
long test).

If I were to do it again obviously I would grab metrics and compare it 
to access times for a basic single server system on local disk and an 
NFS backed system using multiple servers, but alas we were just propping 
it up for fun and see how far we could abuse it.  If one could assume 
that Glusterfs does scale linearly with more nodes you could continue to 
add capacity to the storage layer and grow the cluster, but that's 
another level of testing all together.