Ok, see interspersed.
On Mon, 2013-04-08 at 00:53 +0300, Timo Sirainen wrote:
On 8.4.2013, at 0.10, Bob Gustafson bobgus@rcn.com wrote:
I am still on my quest for a quick way to move mail from a live Maildir system to a 'soon to be live' sdbox system.
I copy Maildir to new system using: rsync -ar --times hoho4:/home/bobgus/Maildir/ /home/bobgus/Maildir
Then I convert from Maildir to sdbox with: dsync mirror maildir:~/Maildir
Then I copy more messages from live system using rsync
^^ that is the mistake
I guess I have a basic misunderstanding of what 'dsync mirror' is doing.
My understanding is that going from Maildir to sdbox, dsync does not mess with the data in Maildir. The Maildir metadata is in one form and the sdbox metadata is in another form (in the sdbox directory).
dsync does mess with metadata in the maildir. also with dsync mirror (as opposed to dsync backup) it can also modify the contents. The main problem here is:
- dsync sees that a folder A in maildir doesn't have a GUID (because dsync is just about the only tool that uses it right now), and assigns the mailbox a new GUID
- dsync syncs the mailbox to sdbox with that GUID
- rsync comes and wipes out the maildir-uidlist that contained the GUID
- second dsync sees that folder A in maildir doesn't have a GUID, and assigns a new GUID to it
- now maildir has folder A with GUID 1, and sdbox has folder A with GUID 2
- dsync thinks they are two different folders, and duplicates them as A and A_2. the A_2 gets also copied back to maildir, because you're using dsync mirror. This is why the second dsync is slow, it's actually doing all the work again, and actually doing twice the work since it's copying the mails from sdbox to maildir as well.
v2.2 dsync is somewhat smarter and can figure out that they are actually the same folder A and it simply changes the other's GUID instead of duplicating all data.
Ok, how is this for a scheme:
- Initially rsync Maildir to destination system
- Do initial 'dsync backup' from Maildir to sdbox on destination system
- Do a 2nd rsync of new Maildir data from live system to destination system, but don't copy the old maildir-uidlist still in the Maildir of the live system. rsync -ar --times --exclude=dovecot-uidlist <live sys Maildir> <dest system Maildir>
I could also use the --ignore-existing option
In theory, rsync would not touch the dovecot-uidlist file, would not touch the existing message files, but would copy over the new messages received during the time 'dsync backup' was doing its previous run.
- Do a 2nd 'dsync backup' from the Maildir to the sdbox (which hasn't changed since the 1st 'dsync backup')
No new email messages enter the sdbox system to be 'mirrored' to the Maildir system.
I thought of using the 'dsync backup' command, but the sentence "Any changes done in destination are discarded." seems to indicate that each time 'dsync backup' is done, it starts from the beginning. No incremental backup (but this is done in 2.2 ?)
dsync backup is incremental. it just wipes out any changes done at the other side (if there happens to be any).