On 19.5.2010, at 16.16, luben karavelov wrote:
On Wed, 19 May 2010 10:51:06 +0200, Timo Sirainen tss@iki.fi wrote:
The company here in Italy didn't really like such idea, so I thought about making it more transparent and simpler to manage. The result is a new "director" service, which does basically the same thing, except without SQL database. The idea is that your load balancer can redirect connections to one or more Dovecot proxies, which internally then figure out where the user should go. So the proxies act kind of like a secondary load balancer layer.
As I understand, the first load balancer is just IP balancer, not POP3/IMAP balancer, isn't it?
Right.
I have implemented similar scheme here with imap/pop3 proxy (nginx) in front of dovecot servers. What i have found to work best (for my conditions) as hashing scheme is some sort of weighted constant hash.
I guess you meant consistent hash? Yeah, I thought about that too first but simpler hash seemed .. simpler. :)
In this way you do not need to synchronize a state between balancers and proxies. If you add or remove servers very few clients get reallocated - num active clients/num servers. .. This scheme has some disadvantages also - on certain circumstances, different sessions to one mailbox could be handled by different servers in parallel.
That's the main thing I wanted to prevent with the director service, so I don't think consistent hashing would have made implementing it easier. Although it might have helped make the caching work a bit better when servers were added/removed.
So my choice was to trade correctness (no parallel sessions to different servers) for simplicity (no state synchronization between servers).
I would have liked to avoid the state sync too, but I like more the idea of having it work 100% perfectly in all conditions. :)
Also, I have a question. Your implementation, what kind of sessions does it balance? I suppose imap/pop3. Is there a plan for similar redirecting of LMTP connections based on delivery address?
It's a pretty generic service. Currently imap and pop3 use it, but it would be pretty easy to make LMTP proxy support it too. The main difference is that for imap/pop3 it needs to emulate being an auth socket, while for lmtp it would need to emulate being a userdb socket. I'd guess it would need less than 50 lines of code total. I guess I should do it before v2.0.0.