Hi John

Thanks for you input.

So you basically state that („physically“) separating the director servers from keepalive/haproxy servers is the only option? I would like to avoid setting up two additional machines for that whenever possible, as any node more in the chain potentially is another point of failure… ;)

I’m curious to hear of any others how they did their dovecot IMAP HA setup, maybe raising som new ideas. :)

BTW: Why was never such a simple thing added to the direcotors code to .i.ex. just specifiy which is the IP of a director server itsels? Example with a new configuriony option „my_director_sever“:

both directors:

director_servers: 192.168.1.10 192.168.1.20

on director-2:

my_director_sever: 192.168.1.20

cheers, Steven

-- https://steven.varco.ch/

Am 14.03.2021 um 20:14 schrieb Paterakis E. Ioannis jpat@uoc.gr:

On 14/3/2021 6:52 μ.μ., Steven Varco wrote:

...

Hi All

I’m trying to establish a dovecot HA setup with two loadbalancers, running keepalived for sharing a virtual public IP. On the same machines I’m running a dovecot director which proxies the requests to two underlying mail servers (on seperate machines).

Now I’m hitting the issue with the way director determines his „Self IP“ by trying to bind to all configured director_servers IPs, taking the first one possible.

Each Director has to listen only on the static IP address of each machine. Then you have to configure the 2 directors in the HAproxies. The floating ip with keepalived will work along with the 2 HAproxies.

...

However this approach only works, when the sysctl setting is: net.ipv4.ip_nonlocal_bind=0 On the other side keepalived needs net.ipv4.ip_nonlocal_bind=1 in order to bind the VIP.

You don't have to mess with these settings.

...

Other possible solutions I could think about:

• Configure each director as „independent“ by setting only one IP in director_servers. => With this aporach you would loose the user to mailserver mapping, although only in a a case of a failover on the loadbalancer, which might can be neglected (or are there any other fallbacks?)

The two directors have a connection to each other, so both know at the same time where's a user mapped. You don't have to worry about that. The user->dovecot mapping will work without any problems even if there is a failover.

...

• Putting director on seperated intermediate machines and proxing the requests through haproxy on the keepalived servers (keepalived -> haproxy -> director -> IMAP => Besides the disadvantage of having another bunch of servers in the chain, also some special configuration on the directory servers might be neccessary to assure director works neatly with haproxy.

The identical scenario will be to have keepalived along with haproxy on same machine, and directors on another. But can work with all three on the same as well. I use the keepalived, haproxy on two machines, with 2 directors underneath each one on different machine/hardware for the high availability's sake, and below them there are 3 dovecot servers.

...

So 2021, what is the „correct“ (best practive) way of having a reduntant HA setup for dovecot?

Cheers :-)

John

On 15/03/2021 20:54 Paterakis E. Ioannis jpat@uoc.gr wrote:

On 15/3/2021 6:09 μ.μ., Steven Varco wrote:

...

Hi John

Thanks for you input.

So you basically state that („physically“) separating the director servers from keepalive/haproxy servers is the only option? I would like to avoid setting up two additional machines for that whenever possible, as any node more in the chain potentially is another point of failure… ;)

Nope, it's not the only option. You can always have all three daemons (keepalived/haproxy/director) on each machine. Keepalived will handle the floating ip job, haproxies will have no problems with the floating ip, the directors will always be binded to the static ips of the machines and have their setup in the haproxies. That's all.

But, if you plan to make a Highly available environment, u have to consider splitting your services to different VMs, and them to different hypervisors in order to be as Highly available as you can....

John

The point of dovecot director is that it acts as a proxy that always routes users to same backend. You can use keepalived, if it supports external commands, to maybe tell director which backends are up / down.

You should have separate server for director(s) and each backend.

Aki

On 03/15/2021 8:43 PM, Paterakis E. Ioannis wrote:

It's not keepalived's work to tell the directors which backend is up/down. You can use poolmon for that. keepalived will make sure the floating ip will always be assigned on an alive haproxy. Then it's haproxies' work to check the aliveness of directors. Then It's Directors job to assign the users to the same dovecot backend all the time, and so on....

What is the purpose of HAProxy in this director setup? It seems like an unecessary extra layer of proxying in your example.

We run a setup with keepalived directors, and a bunch of dovecot IMAP servers, and this works well.

The directors have two IPs each, one static and one floating (keepalived). The IPs listed in the "director_servers" setting are the static IPs. The floating IPs are listed in DNS.

If you simply configure dovecot to bind to all interfaces, and instead use iptables to limit IMAP/POP/director connections to the interfaces you want, there is no need to set net.ipv4.ip_nonlocal_bind=1.

With all that said, I do agree that there should be a way to explicitly set the director's announce/listen address, instead of using the net_try_bind() method.

If you need this feature, I doubt it would be very hard to patch by adding a new configuration option, and then modifying this code to check said option value, and use it (if present) instead of trying to determine the IP:

https://github.com/dovecot/core/blob/fb6aa64435e0ffd66b81cd4895127187f28fa20...

• Eirik

Hi Sebastiaan

Many thanks for that hint, it does work just perfectly, after I configured an address on the service director inet_listener! :) Now each instance only binds to its own IP, which is exactly what I was looking for.

I think it would be helpfull, if this could be added as a note to the docs in the director configuration settings.

And to answer the questsions from others regarding my mentioned „haproxy" on the server: haproxy is not directly in connection with doevecot at all, instead I use it for proxying the SMTP submission port to postifix until I can upgrade dovecot to 2.3 (which I believe should habe the smtp sumbmission service included). And it is also used for proxing to other services, like http for webmail, etc.

Regarding the dovecot setup, I just use keepalived -> dovecot director -> dovecot IMAP

Sorry for the confusion, might should have mentioned that in the beginning… ;)

thanks, Steven

-- https://steven.varco.ch/

Am 16.03.2021 um 13:39 schrieb Sebastiaan Hoogeveen s.hoogeveen@nederhost.nl:

Hi Steven,

On 14/03/2021, 17:53, "dovecot on behalf of Steven Varco" wrote:

...

Now I’m hitting the issue with the way director determines his „Self IP“ by trying to bind to all configured director_servers IPs, taking the first one possible.

However this approach only works, when the sysctl setting is: net.ipv4.ip_nonlocal_bind=0 On the other side keepalived needs net.ipv4.ip_nonlocal_bind=1 in order to bind the VIP.

This can be fixed by specifying the IP address of the director in the inet_listener section of the director service, like this:

service director { ### other configuration here ### inet_listener { address = 172.20.1.4 port = 9090 } }

The listener address will be used as the 'self IP' of the director. This also means that each director will have a slightly different configuration file, but that should usually not be a problem.

I got this from skimming the source, afaict it is not documented anywhere so I'm not sure if this behavior can always be relied on in future releases (it does seem logical to me though).

Kind regards,

--
Sebastiaan Hoogeveen

NederHost KvK: 34099781