On Fri, May 18, 2007 at 11:41:46AM +0900, Christian Balzer wrote:
On Thu, 17 May 2007 19:17:25 +0300 Timo Sirainen tss@iki.fi wrote:
On Thu, 2007-05-17 at 10:04 -0500, Troy Benjegerdes wrote:
But that's currently not *really* replicated. The real question I guess is why not use a cluster/distributed/san filesystem like AFS, GFS, Lustre, GPFS to handle the actual data, and specify that replication only works for maildir or other single file per message mailbox formats.
This already works, but people don't seem to want to use it. There are apparently some stability, performance, complexity and whatever problems. And if you're planning on replicating to a remote location far away they're really slow. But I haven't personally tried any of them, so all I can say is that people are interested in getting replication that doesn't require clustered filesystems, so I'll build it for them. :)
I for one would rather pay you for not re-inventing the wheel, but if people with actual access to funds are willing to pay you for this then I guess "take the money and run" is the thing to do. :-p
I'm going to throw out a warning that it's my feeling that replication has ended many otherwise worthwhile projects. Once you go down that rabbit hole, you end up finding out the hard way that you just can't avoid the stability, performance, complexity, and whatever problems.
If you take the money and run, just be aware of the complexity and customer expectations you are getting yourself into ;)
Yes, all these FS based approaches currently have one or more of the issues Timo lists. The question of course is, will a replicated dovecot be less complex, slow, etc. For people with money, there are enterprise level replicated file systems and/or hardware replicated SANs (remote locations, too). For people w/o those funds there are the above approaches (which despite all their shortcomings can work, right now) and of course one frequently overlooked but perfectly fitting solution, DRBD. For the ZFS fanbois, there ware ways to make it clustered/replicated as well (some storageworks add-on or geom tricks).
The point (for me) would be to not just replicate IMAP (never mind that most of our users use POP, me preferring not to use the dovecot LDA, etc), but ALL of the services/infrastructure that make up a mail system. Which leads quickly back to HA/cluster/SAN/DRBD for me.
I've found myself pretty much in the same "all roads lead to the filesystem" situation. I don't want to replicate just imap, I want to replicate the build directory with my source code, my email, and my MP3 files.
So maybe the right thing to do here is have dovecot do the locking and proxying bit, and initially use librsync for the actual replication. The rsync bit could be replaced with plugins for various filesystems.