[Dovecot] Maildirs location migration

[Thomas Hummel]

On Tue, Apr 16, 2013 at 02:00:38PM +0300, Timo Sirainen wrote:

> dsync doesn't scan through filesystem. It reads the changes from the
> index files. If there are no changes it's pretty much instant even with
> 1M mail mailbox. With changes it's still fast enough (and could be
> faster still by using incremental syncing with saved state via -s
> parameter).

Ok. Actually, I had benched an initial dsync (i.e. no mail in destination) with
a parallelized rsync of precalculated (by an home made tool) chunks of files of
the maildir. For a ~3.3G Maildir, dsync took ~1 hour vs 10 min with 4 rsync at
a time. This of course is a very unfair comparison to dsync since I was using a
cluster to parallelise rsyncs.

But as you said, dsync could be wiser, so I was thinking of using parallel
rsync to make the initial mirror and then use dsync instead of rsync in the
final step described in the dsync wiki.

I'm still not sure if I should forbid dovecot auth temporary (using auth-deny
for instance) or try the seemless way.

> > Besides, how about client side indexing while in this incoherent, not yet fully
> > sync'ed state ? Wouldn't there be corruption risk ?
> 
> The worst that can happen is:
> 
> 1) Client sees new mail 123 in old server
> 2) Client sees only mails up to 122 in the new server
> 3) Client again will see mail 123 after a while
> 
> I'm actually not sure how clients will handle that. It is an IMAP
> protocol violation. It would be possible to add a new flag to dsync
> where it would treat all new emails as conflicts and give them new UIDs,
> so in the above case it wouldn't save a mail 123 but 124.

I see. But there are other cases : 

for instance, the user deletes a mail foobar in the new server because he reconnects
after the kick. I guess dsync would merge the change and would not sync the
foobar message from the old server in the final step. But what if another , new, mail foobaz is
delivered : would'it get the nextuid which was the uid of the deleted foobar mail,
thus confusing the client local indexes ?

Thanks

-- 
Thomas Hummel 	    | Institut Pasteur
<hummel at pasteur.fr> | Groupe Exploitation et Infrastructure