[Dovecot] File synchronization friendly indexes

[Timo Sirainen]

On 16.2.2004, at 03:15, Aredridel wrote:

> I've been doing some tests of file synchronization with Unison (think
> "two way rsync"), and run into one problem: Synching between servers 
> has
> conflicts on the index file.

You didn't say what you were trying to accomplish with this.

A goal that would work is to use two separate servers which sync each 
others once in a while. Clients would treat both of them as completely 
separate servers. Might be useful for eg. laptop users where 
laptop-IMAP-server sync with some external one. Or using a better 
client would work too :)

But I'd guess you were trying to add some automatic redundancy where 
user could be using either one of the servers and not notice a 
difference. There are some major problems, especially related to 
handling UIDs. IMAP requires that each new message gets an UID larger 
than the last one. If both of your servers generates mail with UID 10, 
what should happen when they're syncing? Only solution is to give both 
mails new UIDs, ie. 11 and 12. This might or might not be a problem. 
Some clients store some (user-given) data related to message locally, 
attached to the UID. If UID changes, you lose that.

> I can manually exclude, but what seems to
> me to be a really wonderful feature would be to, preferably, make the
> indexes somewhat atomic, so that files, once written, don't change 
> (this
> is a feature of maildirs in general),

But indexes do change.. They contain for example quick access to 
message flags which are changed. Some data could be stored so they 
don't change, but that would pretty much require each message to have 
it's own index file, which would hurt performance badly.

> and barring that, that they
> include the machine name as part of the filename, so there's no
> conflicts.

Just excluding them from syncing would sound better to me. You could 
quite easily change the filename though.

> Also, on a completely unrelated and somewhat offtopic note: I'd love to
> hear some brainstorms on how to prevent duplicate messages when 
> maildirs
> are synched, files are moved from new on one or more ends, flags are
> changed inconsistently, and synched back.

You'd probably need a quite specialized application to do the syncing 
right. I think there was some existing ones. One that accessed maildirs 
directly, and another that did it via IMAP. They can't be perfect, but 
more or less working is possible..
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
URL: <http://dovecot.org/pipermail/dovecot/attachments/20040219/95737751/attachment-0001.bin>