Re: Live migrating index/control files

Hi,

i think you cannot do this without preventing users to log in. We were
doing similar thing few years ago and i wrote a Python script for this
purpose - you can use it if you find it usefull (script is copying
only control files as indexes are transparently recreated):

Citát Eirik Rye rye@trojka.no:

Hello,

We are running a director-based dovecot cluster where the user's
maildir and the indexes/control files are, for legacy reasons,
stored on separate NFS-backends, i.e.:

mail_location =
maildir:~/Maildir:CONTROL=/mail/control/%1u/%2u/%u:INDEX=/mail/index/%1u/%2u/%u

Both ~/Maildir and /mail/{control,index}/ are separate NFS-mounts,
and we now want to consolidate all files to the user's home
directory instead. The end goal is to stop using NFS altogether.

I have devised a four step plan for gradually migrating the
index/control files into the user's maildir, on a per-user basis:

1. Kick the user in an attempt to flush the user's index files:

doveadm director kick <username>

2. Rsync the index/control files into the user's maildir:

rsync -uav /mail/{index,control}/${user:0:1}/${user:0:2}/${user}/ /mail/${user:0:1}/${user:0:2}/${user}/Maildir/;

3. Update the (redis-based) userdb for the specific user to override
   mail_location to use the default CONTROL/INDEX locations:

{"mail": "maildir:~/Maildir"}

4. Kick the user one last time so that they reconnect using the new settings.

Unfortunately, after running this for a small batch of users, we
recieved a couple reports that their clients had started to
resynchronize and redownload their mailboxes, which is exactly what
we're trying to avoid. There are no index related error messages
related to the few users that reported these issues.

From my understanding, clients become confused when the control
files are changed or lost (dovecot-uidlist and
dovecot-uidvalidity), so I suspect the kick + rsync + kick
procedure is not robust enough for this.

Does anyone have any experience with performing such a migration
without taking systems offline for the duration? Any suggestions to
make this process more transparent for our users, so as to not
induce panic when their mailboxes abruptly appear empty?

Best regards, Eirik