[Dovecot] Clustering (replication and proxying) plans for the future

[Ethan Sommer]

Timo Sirainen wrote:
> Whenever modifying the mailbox a global lock is required. Here's also
> the problem that if the network is down between two servers, they both
> think that they got the global lock. If they did any conflicting
> changes, they'll have to be resolved when both of them again see each
> others.
>   
Some clustering systems (eg GFS) get around this by requiring that you 
have >= 3 servers (preferably an odd number) and having elections which 
require a majority (quorum) to proceed. The third server could just be a 
tie breaker machine and doesn't actually have to run dovecot. (kind of 
like the Vice President in the Senate.) It makes the system somewhat 
less redundant (eg, you can't have a "remote office" server and expect 
it to function if the link is down,) but it is "safer" and "simpler" in 
many ways.

Thus you could have 3 servers, if one goes down, the other two would 
behave normally. (so you can do maintenance, etc) If two servers go 
down, then you would queue up mail and wait for a quorum to be regained. 

With 5 servers, 2 servers could go down without requiring the other 3 to 
stop functioning, etc.


Another option, if you wanted to take on something smaller first would 
be to only allow master/slave clustering somewhat like LDAP and use some 
form of reliable heartbeat (eg serial cable) to detect if the master is 
down and the slave should take the master role. You could still try to 
split the read requests among the two servers for scalability. I know 
that we would _love_ to have a system like this, even if it is less 
flexible.

> All this replication stuff could also be used to implement a remote mail
> storage (ie. smarter proxying). The received indexes/transactions could
> simply be kept in memory, and all the mails that aren't already cached
> in memory could just be requested from the master whenever needed. I'm
> not sure if there's much point in doing this over dummy proxying though.
>   
I would imagine that having a smarter proxy might not help much for full 
fledged mail client (eg Thunderbird) but would have a significant impact 
on webmail applications which are constantly asking for the same thing 
(eg SELECT INBOX) over and over again.


Ethan Sommer
UNIX Systems Administrator
Gustavus Adolphus College