[Dovecot] highly available userdb (Was: Re Dovecot MTA)

[Jan-Frode Myklebust]

My installation is only serving 1/10 of your size, but long time ago we
migrated off mysql for userdatabase, and over to LDAP. The MySQL data
source had issues (not dovecot related), and didn't seem like the right
tool for the job.

Initially we kept mysql as the authoritative database over our users, and
mirrored the user details over to LDAP/389ds -- which we pointed dovecot and
postfix to. Then eventually we migrated completely out of MySQL as user
database. LDAP/389ds gives us easy multimaster replication, easy
integration with dovecot, postfix, etc., client side support for
failover between servers, and it is very fast. I don't think we've ever
had any issue with the userdb after migrating to LDAP.

our two 389ds servers are doing about 80 ldap bind() authentications per
second (plus dovecot auth cache is masking a lot more), 300 searches/s
and are using about 20% of a single cpu core each.

So, I would very much recommend you look into if something similar can
work for you.



  -jf

On Mon, Nov 11, 2013 at 03:24:46PM +1000, Edwardo Garcia wrote:
> My company have 36 dovecots, one biggest ISP in country 3 million user,
> agree with Nick  poster, we had stop use dovecot load balance because too
> bad effect on primary database, now use single localhost, we have script
> run every 30 second to test login, if fail sleep 30 second, try again, fail
> and down ethernet interface so hardware load balancer see server not answer
> and can not use, nagios soon tell us of problem, very very bad and stupid
> way, but only option is safe, we have look at alternative to dovecot for
> this and still look, not happy with unreliable softwares to immitate
> feature.
> 
> big network mean big time locate and fix problem when arise so you be good
> to say no extra point of failure. Too many cog in chain eventually lead to
> problem.
> 
> Timo pleaz reconsider feature
> 
> 
> On Sun, Nov 10, 2013 at 4:21 PM, Nick Edwards <nick.z.edwards at gmail.com>wrote:
> 
> > On 11/9/13, Timo Sirainen <tss at iki.fi> wrote:
> > > On 9.11.2013, at 5.11, Nick Edwards <nick.z.edwards at gmail.com> wrote:
> > >
> > >> On 11/9/13, Michael Kliewe <mkliewe at gmx.de> wrote:
> > >>> Hi Timo,
> > >>>
> > >>> I would also, like others, see you mainly working on Dovecot as an IMAP
> > >>> server. As far as I can see there are many things on the roadmap, and I
> > >>> hope many more will be added (for example a built-in health-checker for
> > >>> director backends).
> > >>>
> > >>> Only if you have enough personal resources and Dovecot as an IMAP
> > server
> > >>> will not "loose your attention", I would love to see your expertise in
> > >>> making a better MTA.
> > >>
> > >> Yes, some of us have been waiting for some years now, for a
> > >> configurable change to alter the method of dovecots method of
> > >> failover, which is just load balancing between servers rather than
> > >> true failover, like postix, I see now why it gets no importance.
> > >
> > > Ah, you’re talking about SQL connections. Had to look up from old mails
> > what
> > > you were talking about. It hasn’t changed, because I think the current
> > > behavior with load balancing + failover is more useful than
> > failover-only.
> > > And you can already do failover-only with an external load balancer.
> > Sure,
> > > Dovecot could also implement it, but it’s not something I especially
> > want to
> > > spend time on implementing.
> > >
> >
> > My employer has 18 pop3 servers, one imap customer access (imap here
> > has so little use we cant justify a redundant machine, not for 11,
> > yes, eleven only users after 2 years of offering imap , and 2 imap
> > (webmail).
> >
> > Sp, each server has a replicated mysql database
> >
> > If I use your "better" method, I have 18 machines polling themselves
> > and the MASTER server, this needlessly slams the daylights out of  the
> > master as I'm sure even you can imagine.
> >
> > We have 4 customer relay smtp servers and 4 inbound smtp servers,
> > postifx, using its failover and "better" method, means they only hit
> > the master server when the local mysql unix socket is not listening,
> > ie, mysqld  is stopped -  the master server NEVER sees them.
> >
> > How is your method, "better" than true failover like method used by
> > postfix, your methods is load balancing, it is not failover, and
> > causes problems on larger networks
> >
> > I'm sure in some cases most people using it are happy and wont have
> > performance increases noticeable, but if you are going to offer a
> > backup for auth, it really shoulds be able to configure, if we want it
> > to DoS our master, or only talk to master when it cant talk local, so
> > I think it should be matter you need to consider, else you are only
> > half arsed doing it, and like implying we should go introduce a
> > further point of failure, by using yet more third party softwares
> >