[Dovecot] Replication plans

[Steffen Kaiser]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 22 May 2007, Timo Sirainen wrote:

> a) Accept that replication process can lose changes, and require a full
> resync when it gets back up. This of course means that all users'
> mailboxes need to be scanned, which can be slow.

If you generate (in idle time or so and at admin request) an Unique ID, 
say MD5 hash, of the mail and use something like "IMAP-UID/MD5-hash" as 
sync data, it shouldn't be that problematic. The change of the data is 
probed by the ID, the keyword-file plain/text, the keywords per message 
using a "IMAP-UID/keyword-letters" pair. Even for large mailboxes, a diff 
should be possible in reasonable time. However, in multi-master 
environments, you'll need to track Expunges, so you know who to resolve 
conflicts, when server A has a message with UID U, but server B has not:

a) server B might not got it so far, or
b) an user had deleted the message on server B already.

The same for UID clashes. If the MD5hash differs for a particular UID on 
two servers, it might be caused by a reset of the UIDs, right?

> But again this would mean that the above is done for all mailboxes.

Why not check a "diff" state beforehand?

> replication, but in error conditions that would almost guarantee that
> some messages get lost. I want to avoid that.

But in this case, you'll need to wait for any replication to any slave is 
done. This is not possible with the laptop-scenario??

> By "much simpler replication" I mean asynchronous replication. Perhaps
> asynchronous could be an option also if it seems that synchronous
> replication adds too much latency (especially if your replication is
> simply for an offsite backup), but I'd want synchronous to be the
> recommended method.

Hmm, wouldn't it then better to have any transaction synchroneous by 
default, then? Though, your approach by defaulting to async, if a slave is 
not responding, makes this requirement a bit strange. Either-or??

Wouldn't it then better to offer two systems right at the beginning:

a synchroneous one, where everything is in sync, incl. keywords and ACLs 
and a async one?

That reminds me of something:

The masters sends all changes to a slave, but the slave storres them for 
later replay. -- Normally, both are in sync and no problem can occur. Now, 
either in idle time or when a particular mailbox is accessed, the slave 
makes the changes to the mailbox.

This way, you have all changes on all slaves in case the master's hd 
fails, but the latency is cut down to the normal network bandwidth. The 
async mode would "simply" spool the changes locally and hands them to the 
slave at later time.

Bye,

- -- 
Steffen Kaiser
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iQEVAwUBRlKjGS9SORjhbDpvAQIQzQf/TWN2WRv8dbZWtK6+p4nVlcxbiR5I83hV
fwRrakDnSiz+lSxlPCkFmhSNALtUqCKZq98fGSIUnqfOJ/HzKcbGbyvMOEboyplk
bPAnoodttnPkJdxpOeaN3N1WzCJp/NUs30ZEAaN8GF5CekRwX7dlxCiFSE9owEkQ
oet9CvJQdg0eVwSUnOL0pscYjXwg6oia51BYS/fYJ9pkbiq71pJ/CiySo/ZtvTg6
kq5AWDamM0qqWvFJniXFlfOO/bxexbbNozJ5xa51lKUN5KdPwIlDWzo7IzwceowQ
h6enjc8T89A0zPO7uawtpcqvszTM6t7zmpGhDD7qJWxVfScnoghDFg==
=oP3J
-----END PGP SIGNATURE-----