[Dovecot] Best Cluster Storage
Luben Karavelov
karavelov at spnet.net
Wed Jan 26 21:21:02 EET 2011
On Thu, 13 Jan 2011 10:33:34 -0200, Henrique Fernandes
<sf.rique at gmail.com> wrote:
I use ocfs2 with 3 dovecots. one only for mailman.
> We have problens with IO. Have about 4k active users.
> We are now testing more ocfs2 clusters, becasue one of yours theorys
> is that
> iff all mail resides in only one ocfs2 cluster, it takes too long to
> find
> the file. ocfs2 i guess does not support index. using ocfs2 1.4
My last production environment using OCFS2 was with quite recent
ocfs2/dovecot -
linux 2.6.35 and dovecot 1.2.15 with dbox mail storage. We got a lot of
problems - high IO, fragmentation and exponential grow of access time
etc. We
tested also with directory indexes but this hasn't helped a lot.
Finaly we scrapped the ocfs2 setup and moved to less advanced setup:
We created distinct volumes for every worker on the SAN, formated it
with with
XFS. The volumes got mounted on different mountpoints on workers. We
setup a
Pacemaker as cluster manager on the workers, so if worker dies its
volume
gets mounted on another worker and its service IP is brought up there.
As a result we are using a fraction of the IO compared with OCFS, the
wait time
on the workers dropped significantly, the service got better.
You have different options to distribute mailboxes through the workers.
In owr
setup the load is distributed by domain, because we are servicing
hundreds of
domains. So every domain MX/pop3/imap was changed to the service IP of
the
worker. If there are a lot of mailboxes in one domain you should put a
balancer
that knows on which server the mailbox is located and forward the
requests
there.
> So now, we are gettins smallers luns from your storages and mounting
> 3 ocfs2
> clusters that way we think the DLM will work better.
> Sorry if i did not answer your question.
> Anyway, we had some tests with NFS and it wasn't good also. We
> prefere
> sticky with ocfs2.
My test with NFS3/NFS4 were not good also, so it was not considered an
option.
> We are balacing with IPVS, not using dovecot director.
With IPVS you could not stick the same mailbox to the same server -
this is
important with ocfs setup because of filesystem caches and the locks.
We were using nginx as proxy/balancer that could stick the same mailbox
to
the same backend - we did this before there was director service in
dovecot
but now you could use the director.
Best regards
--
Luben Karavelov
More information about the dovecot
mailing list