On Sat, Mar 24, 2012 at 08:19:48AM +0100, Attila Nagy wrote:
On 03/23/12 22:25, Timo Sirainen wrote:
Well, dsync is a very useful tool, but with continuous replication it tries to solve a problem which should be handled -at least partially- elsewhere. Storing stuff in plain file systems and duplicating them to another one just doesn't scale.
I don't see why this shouldn't scale. Mailboxes are after all changed relatively infrequently. One idea for making it more scalable might be to treat indexes/metadata and messages differently. Make index/metadata updates synchronous over the clusters/locations (with re-sync capability in case of lost synchronisation), while messages are store in one "altstorage" per cluster/location.
For a two-location solution, message-data should be stored in:
mail_location = mdbox:~/mdbox
ALTcache=mdbox:~/mdbox-remoteip-cache
ALT=dfetch://remoteip/ <-- new protocolIf a message is in the index, look for it in that order:
local mdbox
ALTcache
ALTif it finds the message in ALT, make a copy into ALTcache (or local mdbox?).
Syncronizing messages could be a very low frequency job, and could be handled by simple rsync of ALT to ALTcache. No need for specialized tool for this job. Syncronizing ALTcache to local mdbox could be done with a reversed doveadm-altmove, but might not be necessary.
Of course this is probably all very naive.. but you get the idea :-)
-jf