[Dovecot] dsync redesign

[Micah Anderson]

Timo Sirainen <tss at iki.fi> writes:

> In case anyone is interested in reading (and maybe helping!) with a dsync redesign that's intended to fix all of its current problems, here are some possibly incoherent ramblings about it:

thank you for opening this discussion about dsync!

besides the problems I've encountered with dsync, there are a couple
things that I think would be great to build into the new vision of the
protocol. 

One would be the ability to perform *intelligent* incremental/rotated
backups. I can do this now by running a dsync backup operation and then
doing manual hardlinking or moving of the backup directories (daily.1,
daily.2, weekly.1, monthly.1, etc.), but it would be more intelligent if
this were baked into the backup process.

Secondly, being able to filter out mailboxes could result in much more
efficient syncing. Now there is the capability to operate on only
specific mailboxes, but this doesn't scale well when I am trying to
backup thousands of users and I want to omit the Spam and Trash folders
from the sync. I would have to get a mailbox list of each user, and then
iterate over each mailbox for each user, skipping the Spam and Trash
folders, forking a new 'dsync backup' for each of their mailboxes, for
each user.

Lastly, there isn't a good method for restoring backups. I can reverse
the backup process, onto the user's "live" mailbox, but that brings the
user into an undesirable state (eg. their mailbox state one day
ago). Better would be if their backup could be restored in such a way
that the user can resolve the missing pieces manually, as they know
best.

thanks again for your work on this, from my position dovecot is an
amazing piece of software, the only part that seems to have some issues
is dsync and I applaud the effort to redesign to fix things!

micah