Hello,
I'd like to figure out how to set up a site running dovecot on multiple computers for load balancing reasons.
I'm currently running dovecot-1.0.14 on a 8GB RAM, 2 quad-core 2.66Ghz Xeon single server which runs FreeBSD 6.2-STABLE SMP.
. the mailbox format was mbox and I successfully migrated to Maildir only . mailboxes are on an NFS NetApp server, attribute caching is off . indexes are locally stored . the number of users is around 2500, some of them having huge mailboxes (~ 1 or 2 GB) . a large amount of users is using IMAP but there are also many POP3 users . the passdb/userdb is openLDAP . for now, I neither use namespaces, nor quotas nor deliver, nor any plugin, but I plan to use namespaces and ACL to implement shared mailboxes.
The same machine is running postfix-2.4.6 with amavisd-new-2.5.4 (no spamassassin) and clamav-0.93.1. I'm using procmail-3.22 as the LDA.
I installed dovecot-1.1.2 on non-standard ports for test and plan to switch to it as I read on this list it performs better, load wise.
This setup (1.0.14) works great except that, compared to the time mailboxes were in mbox format, the load average (as shown by 'top') goes higher from a significant amout on a regular basis and sometimes goes over the top.
On "normal" circumstances, the number of processes is around 1400 and the load oscillates between 1 to 10 and 30 to 60. Most of the time, I'd say the load average is around 20.
Every friday a message (no attachement, decent size) is sent to all users : in such circumstances (but only sometimes, not every time : let's say half the time such a message is sent), the load goes so high that I have to stop dovecot to let procmail deliver the message or even reboot the machine.
Sometimes, the load climbs up to something like 150, then goes back to the 'normal' case described above.
I know multi-master replication is on the roadmap. I know some dovecot sites use in the meantime several dovecot servers. I'd like to know how those sites do the load balancing, the main problem beeing to be able to direct each user to the same dovecot server each time as stated in the Wiki (http://wiki.dovecot.org/NFS).
I don't think that DNS round robin would do the trick because some UA (for instance Thunderbird) often open up to 5 connexions for the same user, unless maybe such UA makes a single 'gethostbyname()', thus connecting 5 times to the same physical server ?
What are the options to achieve such a setup ? Any successful experiences ?
Thank you
-- Thomas Hummel | Institut Pasteur hummel@pasteur.fr | Pôle informatique - systèmes et réseau