[Dovecot] Safely restoring backups

[Timo Sirainen]

On Fri, 2011-10-07 at 11:09 -0400, Micah Anderson wrote:
> I'm trying to determine what the best way to restore mail with mdbox
> is. Restoring using maildir was trivial, I just used rsync
> --ignore-existing which wrote any mails that were removed and didn't
> touch things that already existed[1]. 

If a mail had changed flag, the maildir file got duplicated, which
Dovecot complained about if it noticed it.

> With mdbox things have become more
> complicated, and I haven't found a way to restore mail that doesn't
> result in many message duplicates.

Do you need to restore mails so often that this is really a problem? :)

> 1. I tried rsyncing the different backup directories back to the mail
> storage host, and then doing:
> 
> 'dsync -R backup -u $user mdbox:/path/to/to/daily.1/mdbox'[2]
> 'dsync -R backup -u $user mdbox:/path/to/to/daily.2/mdbox'
> 
> This works ok, but it gives duplicates of every mail that already exists
> for every daily/weekly I dsync. This is what the rsync --ignore-existing
> avoided. This is particularly annoying if I restore the weekly, and
> multiple daily directories, because then you get a copy for every one
> you restore. I had thought that the individual message's GIDs would keep
> them from being duplicated?

GUIDs can be used to identify messages, but there's no automatic
deduplication. It's fine to e.g. copy a message from INBOX to INBOX,
which duplicates it. Dovecot shouldn't prevent that.

> Is there a way I can restore things from backup and avoid duplicates?
> Is there another method I should try?

Here's one way, although somewhat slow (and not fully tested):

doveadm -f tab fetch -u user at domain 'mailbox guid' all | sort > guids1
doveadm -f tab fetch -o mdbox:/backups/user -u user at domain 'mailbox-guid guid' all | sort > guids2
diff -u guids1 guids2 | grep '^+[+]' | sed 's/^+//' |
  awk { system("doveadm import -u user at domain mdbox:/backups/user restored mailbox-guid "$1" guid "$2); }

> 3. why doesn't doveadm import have a -s option to subscribe? 

I suppose it could.. Added to v2.1:
http://hg.dovecot.org/dovecot-2.1/rev/afec4ceda8e1