My installation is only serving 1/10 of your size, but long time ago we migrated off mysql for userdatabase, and over to LDAP. The MySQL data source had issues (not dovecot related), and didn't seem like the right tool for the job.
Initially we kept mysql as the authoritative database over our users, and mirrored the user details over to LDAP/389ds -- which we pointed dovecot and postfix to. Then eventually we migrated completely out of MySQL as user database. LDAP/389ds gives us easy multimaster replication, easy integration with dovecot, postfix, etc., client side support for failover between servers, and it is very fast. I don't think we've ever had any issue with the userdb after migrating to LDAP.
our two 389ds servers are doing about 80 ldap bind() authentications per second (plus dovecot auth cache is masking a lot more), 300 searches/s and are using about 20% of a single cpu core each.
So, I would very much recommend you look into if something similar can work for you.
-jf
On Mon, Nov 11, 2013 at 03:24:46PM +1000, Edwardo Garcia wrote:
My company have 36 dovecots, one biggest ISP in country 3 million user, agree with Nick poster, we had stop use dovecot load balance because too bad effect on primary database, now use single localhost, we have script run every 30 second to test login, if fail sleep 30 second, try again, fail and down ethernet interface so hardware load balancer see server not answer and can not use, nagios soon tell us of problem, very very bad and stupid way, but only option is safe, we have look at alternative to dovecot for this and still look, not happy with unreliable softwares to immitate feature.
big network mean big time locate and fix problem when arise so you be good to say no extra point of failure. Too many cog in chain eventually lead to problem.
Timo pleaz reconsider feature
On Sun, Nov 10, 2013 at 4:21 PM, Nick Edwards nick.z.edwards@gmail.comwrote:
On 11/9/13, Timo Sirainen tss@iki.fi wrote:
On 9.11.2013, at 5.11, Nick Edwards nick.z.edwards@gmail.com wrote:
On 11/9/13, Michael Kliewe mkliewe@gmx.de wrote:
Hi Timo,
I would also, like others, see you mainly working on Dovecot as an IMAP server. As far as I can see there are many things on the roadmap, and I hope many more will be added (for example a built-in health-checker for director backends).
Only if you have enough personal resources and Dovecot as an IMAP server will not "loose your attention", I would love to see your expertise in making a better MTA.
Yes, some of us have been waiting for some years now, for a configurable change to alter the method of dovecots method of failover, which is just load balancing between servers rather than true failover, like postix, I see now why it gets no importance.
Ah, you’re talking about SQL connections. Had to look up from old mails what you were talking about. It hasn’t changed, because I think the current behavior with load balancing + failover is more useful than failover-only. And you can already do failover-only with an external load balancer. Sure, Dovecot could also implement it, but it’s not something I especially want to spend time on implementing.
My employer has 18 pop3 servers, one imap customer access (imap here has so little use we cant justify a redundant machine, not for 11, yes, eleven only users after 2 years of offering imap , and 2 imap (webmail).
Sp, each server has a replicated mysql database
If I use your "better" method, I have 18 machines polling themselves and the MASTER server, this needlessly slams the daylights out of the master as I'm sure even you can imagine.
We have 4 customer relay smtp servers and 4 inbound smtp servers, postifx, using its failover and "better" method, means they only hit the master server when the local mysql unix socket is not listening, ie, mysqld is stopped - the master server NEVER sees them.
How is your method, "better" than true failover like method used by postfix, your methods is load balancing, it is not failover, and causes problems on larger networks
I'm sure in some cases most people using it are happy and wont have performance increases noticeable, but if you are going to offer a backup for auth, it really shoulds be able to configure, if we want it to DoS our master, or only talk to master when it cant talk local, so I think it should be matter you need to consider, else you are only half arsed doing it, and like implying we should go introduce a further point of failure, by using yet more third party softwares