[Dovecot] dovecot director service
hi, we are using the dovecot-rc1 version, and we are testing the director service, as there is little documentation on this service, a pair of questions..
if i'm not wrong, the proxy_maybe it's not implemented, yet? will be soon? this limitation does not allow use the same two directors servers like mail backend servers? the fairlure of one director server it's automatically detected or require manual intervention?
thanks and sorry for so many questions...
Xavier
On 7.7.2010, at 10.59, Xavier Pons wrote:
if i'm not wrong, the proxy_maybe it's not implemented, yet? will be soon?
I tried implementing it once, but it got difficult, so I'm not sure when it'll be implemented.
this limitation does not allow use the same two directors servers like mail backend servers?
You can install two Dovecot instances on the same server (set different base_dir for them), one as director and another as mail server. A bit annoying to have two of them, but it should work.
the fairlure of one director server it's automatically detected or require manual intervention?
It's automatically detected. I've only simulated it with software, but you could also try it by actually pulling the network plug out. :)
El 07/07/2010 13:35, Timo Sirainen escribió: ....
this limitation does not allow use the same two directors servers like mail backend servers? You can install two Dovecot instances on the same server (set different base_dir for them), one as director and another as mail server. A bit annoying to have two of them, but it should work. I suppouse, this require to have different IPs address for every instance? the fairlure of one director server it's automatically detected or require manual intervention? It's automatically detected. I've only simulated it with software, but you could also try it by actually pulling the network plug out. :) If I have different dovecot instances on same host, one like proxy and another like mail backend, the director service will detect both failures of director service like of backend servers? or only detect the first one?
thanks
Xavier
Am 08.07.2010 09:31, schrieb Xavier Pons:
El 07/07/2010 13:35, Timo Sirainen escribió: ....
this limitation does not allow use the same two directors servers like mail backend servers? You can install two Dovecot instances on the same server (set different base_dir for them), one as director and another as mail server. A bit annoying to have two of them, but it should work. I suppouse, this require to have different IPs address for every instance? the fairlure of one director server it's automatically detected or require manual intervention? It's automatically detected. I've only simulated it with software, but you could also try it by actually pulling the network plug out. :) If I have different dovecot instances on same host, one like proxy and another like mail backend, the director service will detect both failures of director service like of backend servers? or only detect the first one?
You will have to build some kind of HA environment in front of the proxys. If you just use DNS round-robin for connects to the proxy server (the instance) client connection attempts to this particular server will fail. The director service is only for user-stickyness (and HA) between the 1st and the 2nd tier. Most common use would be in a NFS storage backend.
Regards, Oliver
El 08/07/2010 10:00, Oliver Eales escribió:
Am 08.07.2010 09:31, schrieb Xavier Pons:
El 07/07/2010 13:35, Timo Sirainen escribió: ....
this limitation does not allow use the same two directors servers like mail backend servers? You can install two Dovecot instances on the same server (set different base_dir for them), one as director and another as mail server. A bit annoying to have two of them, but it should work. I suppouse, this require to have different IPs address for every instance? the fairlure of one director server it's automatically detected or require manual intervention? It's automatically detected. I've only simulated it with software, but you could also try it by actually pulling the network plug out. :) If I have different dovecot instances on same host, one like proxy and another like mail backend, the director service will detect both failures of director service like of backend servers? or only detect the first one?
You will have to build some kind of HA environment in front of the proxys. If you just use DNS round-robin for connects to the proxy server (the instance) client connection attempts to this particular server will fail. The director service is only for user-stickyness (and HA) between the 1st and the 2nd tier. Most common use would be in a NFS storage backend.
Yes, we will have two hardware balancers in front of proxies. Thus, the director service will detect failures of backend servers and not forward sessions them? how detects if a backend server it's alive or not? The command 'doveadm director status', gives information about status of backend mailservers or of director servers? Sorry for so many questions ...
Thanks Xavier
Xavier,
On 7/8/10 1:29 AM, "Xavier Pons" <xavier.pons@uib.es> wrote:
Yes, we will have two hardware balancers in front of proxies. Thus, the director service will detect failures of backend servers and not forward sessions them? how detects if a backend server it's alive or not?
IIRC, it does not detect failures of backend servers. It's up to you to detect outages and react appropriately.
The folks that sponsored Director development apparently have a monitoring script that removes downed nodes by running something like 'ssh directorhost doveadm director remove backendhost', and then re-adds them when they come back up.
I'm not sure how I'm going to handle this myself, as our monitoring system only checks every 5 minutes, and our existing load balancers check and add/remove nodes every 20 seconds or so. 5 minutes would be a long time to have Dovecot trying to send users to a non-functional server.
The command 'doveadm director status', gives information about status of backend mailservers or of director servers?
Just the backend servers - it shows backend server addresses and how many users they're each assigned, or details on a specific user mapping.
I am not aware of a way to get Dovecot to output the director ring status. That would be nice though, to be able to list the directors and how many connections they're each proxying.
You might read though this thread, which starts here: http://www.dovecot.org/list/dovecot/2010-May/049189.html And continues later here: http://www.dovecot.org/list/dovecot/2010-June/049425.html
-Brad
On 8.7.2010, at 10.48, Brandon Davidson wrote:
Yes, we will have two hardware balancers in front of proxies. Thus, the director service will detect failures of backend servers and not forward sessions them? how detects if a backend server it's alive or not?
IIRC, it does not detect failures of backend servers. It's up to you to detect outages and react appropriately.
The folks that sponsored Director development apparently have a monitoring script that removes downed nodes by running something like 'ssh directorhost doveadm director remove backendhost', and then re-adds them when they come back up.
Yes. It was also much simpler to implement that way :) Maybe it wouldn't be hugely difficult to implement internally though:
- notify-connection is already used to send notifications about what user connections exist. the same api could be used to send a notification that 'backend x is down'. but a single "connection refused" probably shouldn't bring down the host without at least a few tries within several seconds.. removal would be done by setting vhost_count=0
- adding hosts back up automatically would require more code in director where it would automatically try to connect to them every once in a while. probably save the pre-removal vhost count and then add it back with that vhost count.
- the proxy itself should also have some code to deal with connect failures by doing another passdb lookup a while after sending the 'host is down' notification and then reconnecting to the new host.
- then some settings how long to try to connect to a backend until it's declared dead. either a hanging connect(), hanging existing connections or connect() refused attempts tried for that long time.
I am not aware of a way to get Dovecot to output the director ring status. That would be nice though, to be able to list the directors and how many connections they're each proxying.
Yeah. Any good naming ideas for the doveadm director command? :)
El 08/07/2010 13:26, Timo Sirainen escribió:
On 8.7.2010, at 10.48, Brandon Davidson wrote:
Yes, we will have two hardware balancers in front of proxies. Thus, the director service will detect failures of backend servers and not forward sessions them? how detects if a backend server it's alive or not? IIRC, it does not detect failures of backend servers. It's up to you to detect outages and react appropriately.
The folks that sponsored Director development apparently have a monitoring script that removes downed nodes by running something like 'ssh directorhost doveadm director remove backendhost', and then re-adds them when they come back up. Yes. It was also much simpler to implement that way :) Maybe it wouldn't be hugely difficult to implement internally though:
- notify-connection is already used to send notifications about what user connections exist. the same api could be used to send a notification that 'backend x is down'. but a single "connection refused" probably shouldn't bring down the host without at least a few tries within several seconds.. removal would be done by setting vhost_count=0
- adding hosts back up automatically would require more code in director where it would automatically try to connect to them every once in a while. probably save the pre-removal vhost count and then add it back with that vhost count.
- the proxy itself should also have some code to deal with connect failures by doing another passdb lookup a while after sending the 'host is down' notification and then reconnecting to the new host.
- then some settings how long to try to connect to a backend until it's declared dead. either a hanging connect(), hanging existing connections or connect() refused attempts tried for that long time.
I think this new funcionalities would be perfect (necessary ;-) ) for a complete load balanced/high availability mail system.
I am not aware of a way to get Dovecot to output the director ring status. That would be nice though, to be able to list the directors and how many connections they're each proxying. Yeah. Any good naming ideas for the doveadm director command? :) something like doveadm director servers ?!?!
Xavier
On 7/9/10 12:01 AM, "Xavier Pons" <xavier.pons@uib.es> wrote:
I think this new funcionalities would be perfect (necessary ;-) ) for a complete load balanced/high availability mail system.
Timo, what you described sounds great.
Pretty much anything built into Dovecot would be an improvement over an external script from my point of view. With a script I have to deal with all kinds of questions like, which director do I have my script log in to? What happens if it goes down? What happens if the monitoring host is down?
I'd probably end up trying to put the director port behind the load-balancer and figuring out some way to get my script to talk Director protocol to add/remove mail servers, and that would just be ugly ;)
Yeah. Any good naming ideas for the doveadm director command? :) something like doveadm director servers ?!?!
I'm not sure either. Maybe 'doveadm director ring'?
I thought of suggesting that 'status' report the ring status, and the current output go to something like 'mailhosts'. After a moment of consideration, I realized that all the current director commands (add,remove,status) act on the mailhost list, not the director list, and so in that sense 'doveadm director' is really more like 'doveadm director-mailhosts' to begin with.
-Brad
On Fri, 2010-07-09 at 03:03 -0700, Brandon Davidson wrote:
I think this new funcionalities would be perfect (necessary ;-) ) for a complete load balanced/high availability mail system.
Timo, what you described sounds great.
Pretty much anything built into Dovecot would be an improvement over an external script from my point of view.
Yeah, some day I guess..
With a script I have to deal with all kinds of questions like, which director do I have my script log in to?
Any one of them.
What happens if it goes down? What happens if the monitoring host is down?
Add redundancy :)
I'd probably end up trying to put the director port behind the load-balancer and figuring out some way to get my script to talk Director protocol to add/remove mail servers, and that would just be ugly ;)
Well, you could make the doveadm interface available via TCP port, but that really should be firewalled well. Hmm. It wouldn't be difficult to patch doveadm director to also support connecting to host:port rather than unix socket.
Yeah. Any good naming ideas for the doveadm director command? :) something like doveadm director servers ?!?!
I'm not sure either. Maybe 'doveadm director ring'?
That sounds pretty good to me.
Timo,
On 7/11/10 12:06 PM, "Timo Sirainen" <tss@iki.fi> wrote:
Pretty much anything built into Dovecot would be an improvement over an external script from my point of view.
Yeah, some day I guess..
Well, I would definitely make use of it if you ever get around to coding it.
With a script I have to deal with all kinds of questions like, which director do I have my script log in to?
Any one of them.
Sure, but that adds additional complexity to the script (selecting a host, retrying, etc)... or sticking the director/doveadm interface behind a load balancer if possible.
What happens if it goes down? What happens if the monitoring host is down?
Add redundancy :)
Sure, but it's easier for me if I don't have to worry about it ;)
Well, you could make the doveadm interface available via TCP port, but that really should be firewalled well. Hmm. It wouldn't be difficult to patch doveadm director to also support connecting to host:port rather than unix socket.
That would be pretty awesome. Rignt now I could try to talk Director protocol from the management node to inject commands, but the Directors only accept connections from other hosts listed in the ring, right? So having doveadm function over TCP would be a big plus.
-Brad
On 11.7.2010, at 23.30, Brandon Davidson wrote:
Well, you could make the doveadm interface available via TCP port, but that really should be firewalled well. Hmm. It wouldn't be difficult to patch doveadm director to also support connecting to host:port rather than unix socket.
That would be pretty awesome. Rignt now I could try to talk Director protocol from the management node to inject commands, but the Directors only accept connections from other hosts listed in the ring, right?
Yes, but you'd also have to act as a proper director then. It's basically the same as running a director in the management node, which just doesn't happen to accept any client connections.
So having doveadm function over TCP would be a big plus.
Done. doveadm director status -a host:port
Works also with doveadm who and doveadm penalty (and anything else in future that communicates usually via UNIX sockets).
On Mon, 2010-07-12 at 00:34 +0100, Timo Sirainen wrote:
So having doveadm function over TCP would be a big plus.
Done. doveadm director status -a host:port
Except .. there's no way to actually tell director that a specific port is a doveadm port, not a director port, unless you hardcode it.. Wonder if I should have added some kind of a "type" setting for the listeners..
El 12/07/2010 16:53, Timo Sirainen escribió:
On Mon, 2010-07-12 at 00:34 +0100, Timo Sirainen wrote:
So having doveadm function over TCP would be a big plus. Done. doveadm director status -a host:port Except .. there's no way to actually tell director that a specific port is a doveadm port, not a director port, unless you hardcode it.. Wonder if I should have added some kind of a "type" setting for the listeners..
Hi, this funcionality it's implemented in the actual hg repository?
doveadm director status -a ipDirector1:991 and get: doveadm(root): Fatal: ipDirector1:991 not a compatible director-doveadm socket
I'm trying to implement a healthcheck/monitor script for the directors servers and mail servers.
Thanks
Xavier
On 22.7.2010, at 11.46, Xavier Pons wrote:
Except .. there's no way to actually tell director that a specific port is a doveadm port, not a director port, unless you hardcode it.. Wonder if I should have added some kind of a "type" setting for the listeners..
Hi, this funcionality it's implemented in the actual hg repository?
doveadm director status -a ipDirector1:991
This is.
and get: doveadm(root): Fatal: ipDirector1:991 not a compatible director-doveadm socket
Like I said above, there's no way for director to know that you're trying to connect with doveadm instead of director. You'd have to modify the source code to handle port 991 as doveadm connection. I guess there could be a setting..
director_doveadm_port = 991
Kind of annoying, but I guess I could add that.
On Thu, 2010-07-22 at 12:35 +0100, Timo Sirainen wrote:
director_doveadm_port = 991
Kind of annoying, but I guess I could add that.
El 22/07/2010 17:08, Timo Sirainen escribió:
On Thu, 2010-07-22 at 12:35 +0100, Timo Sirainen wrote:
director_doveadm_port = 991
Kind of annoying, but I guess I could add that. Added: http://hg.dovecot.org/dovecot-2.0/rev/ab85d17947db
Hi Timo,
like you say, I have added director_doveadm_port = 990 to my director configuration, but how I must add the inet_listener for this port? I have tried with:
service director { unix_listener login/director { mode = 0666 } fifo_listener login/proxy-notify { mode = 0666 } unix_listener director-userdb { mode = 0666 }
director service port
inet_listener { port = 991 } #director-doveadm service port inet_listener { port = 990 } }
but I get the error:
[root@imap2 ~]# doveadm director status -a imap2:990 doveadm(root): Fatal: imap2:990 not a compatible director-doveadm socket
and the log says: Jul 23 09:50:14 imap2 dovecot: director: Error: director(10.180.188.12/in): Wrong protocol in socket (director-doveadm vs director)
then also I have tried with:
service director-doveadm { inet_listener { port = 990 } }
but when restart the daemon get:
doveconf: Error: service(director-doveadm): executable is empty doveconf: Fatal: Error in configuration file /etc/dovecot-proxy/dovecot.conf: service(director-doveadm): executable is empty
How is the correct format to define the inet director-doveadm port?
thanks
Xavier
On Fri, 2010-07-23 at 09:55 +0200, Xavier Pons wrote:
El 22/07/2010 17:08, Timo Sirainen escribió:
On Thu, 2010-07-22 at 12:35 +0100, Timo Sirainen wrote:
director_doveadm_port = 991
Kind of annoying, but I guess I could add that. Added: http://hg.dovecot.org/dovecot-2.0/rev/ab85d17947db
Hi Timo,
like you say, I have added director_doveadm_port = 990
It wasn't actually working. This fixes it: http://hg.dovecot.org/dovecot-2.0/rev/3ef50adb7abf
director service port
inet_listener { port = 991 } #director-doveadm service port inet_listener { port = 990 } }
This is correct.
On 8.7.2010, at 8.31, Xavier Pons wrote:
El 07/07/2010 13:35, Timo Sirainen escribió: ....
this limitation does not allow use the same two directors servers like mail backend servers? You can install two Dovecot instances on the same server (set different base_dir for them), one as director and another as mail server. A bit annoying to have two of them, but it should work. I suppouse, this require to have different IPs address for every instance?
No, they can just listen on different ports.
participants (4)
-
Brandon Davidson
-
Oliver Eales
-
Timo Sirainen
-
Xavier Pons