[Dovecot] Replication plans

[Bill Boebel]

On Fri, May 18, 2007 1:42 am, Troy Benjegerdes <hozer at hozed.org> said:

> I'm going to throw out a warning that it's my feeling that replication
> has ended many otherwise worthwhile projects. Once you go down that
> rabbit hole, you end up finding out the hard way that you just can't
> avoid the stability, performance, complexity, and whatever problems.
> ..
> I've found myself pretty much in the same "all roads lead to the
> filesystem" situation. I don't want to replicate just imap, I want to
> replicate the build directory with my source code, my email, and my MP3
> files.

One of the problems with the clustered file system approach seems to be that accessing Dovecot's index, cache and control files are slow over the network.  For speed, ideally you want your index, cache and control on local disk... but still replicated to another server.

So what about tackling this replication problem from a different angle...  Make it Dovecot's job to replicate the index and control files between servers, and make it the file system's job to replicate just the mail data.  This would require further disconnecting the index and control files from the mail data, so that there is less syncing required.  i.e. remove the need to check directory mtimes and to compare directory listings against the index; and instead assume that the indexes are always correct.  Periodically you could still check to see if a sync is needed, but you'd do this must less frequently.

I agree that there are already great solutions available for replicated storage, so this would allow us to take advantage of these solutions for the bulk of our storage without impacting the speed of IMAP.

Bill