[Dovecot] Best Cluster Storage

[Luben Karavelov]

 On Thu, 13 Jan 2011 10:33:34 -0200, Henrique Fernandes 
 <sf.rique at gmail.com> wrote:
 I use ocfs2 with 3 dovecots. one only for mailman.

> We have problens with IO. Have about 4k active users.

> We are now testing more ocfs2 clusters, becasue one of yours theorys 
> is that
> iff all mail resides in only one ocfs2 cluster, it takes too long to 
> find
> the file. ocfs2 i guess does not support index. using ocfs2 1.4


 My last production environment using OCFS2 was with quite recent 
 ocfs2/dovecot -
 linux 2.6.35 and dovecot 1.2.15 with dbox mail storage. We got a lot of
 problems - high IO, fragmentation and exponential grow of access time 
 etc. We
 tested also with directory indexes but this hasn't helped a lot.

 Finaly we scrapped the ocfs2 setup and moved to less advanced setup:
 We created distinct volumes for every worker on the SAN, formated it 
 with with
 XFS. The volumes got mounted on different mountpoints on workers. We 
 setup a
 Pacemaker as cluster manager on the workers, so if worker dies its 
 volume
 gets mounted on another worker and its service IP is brought up there.

 As a result we are using a fraction of the IO compared with OCFS, the 
 wait time
 on the workers dropped significantly, the service got better.

 You have different options to distribute mailboxes through the workers. 
 In owr
 setup the load is distributed by domain, because we are servicing 
 hundreds of
 domains. So every domain MX/pop3/imap was changed to the service IP of 
 the
 worker. If there are a lot of mailboxes in one domain you should put a 
 balancer
 that knows on which server the mailbox is located and forward the 
 requests
 there.

> So now, we are gettins smallers luns from your storages and mounting
> 3 ocfs2
> clusters that way we think the DLM will work better.

> Sorry if i did not answer your question.

> Anyway, we had some tests with NFS and it wasn't good also. We 
> prefere
> sticky with ocfs2.

 My test with NFS3/NFS4 were not good also, so it was not considered an 
 option.


> We are balacing with IPVS, not using dovecot director.


 With IPVS you could not stick the same mailbox to the same server - 
 this is
 important with ocfs setup because of filesystem caches and the locks.
 We were using nginx as proxy/balancer that could stick the same mailbox 
 to
 the same backend - we did this before there was director service in 
 dovecot
 but now you could use the director.

 Best regards

-- 
 Luben Karavelov