[Dovecot] dsync backup questions

[Ben Morrow]

I can't give authoratitive answers to either of these, but...

At  6PM -0500 on  3/02/13 you (Joe Beaubien) wrote:
> 
> I'm currently trying to setup remote backups of my emails but i'm running
> into issues (mdbox format, indexes and storage in the same folder
> hierarchy).
> 
> Local backup command: dsync -u "my_user" backup /backups/my_user
> 
> (1) Recently, I noticed that the local backup takes up twice the size as
> the original mail location (8gb vs. 4gb). I purged alot of emails from the
> original location, so the size shrunk, but the local backup just keeps on
> getting bigger. I couldn't find any dsync option that would delete extra
> emails.
> 
> - Question: Why isn't the local backup synced properly and remove the extra
> emails?

Are you running 'doveadm purge' on the backed-up dbox? It looks to me as
though dsync doesn't do that. I don't know if there's any (simple) way
to do that without a running Dovecot instance attached the dbox
directory: it's not entirely clear to me whether doveadm will run
locally without contacting a doveadm-server instance running under
Dovecot, nor how to point 'doveadm purge' at an arbitrary directory.

It might be easier to dsync to a Maildir instead. This should preserve
all the Dovecot-visible metadata, and dsyncing back to the dbox for
restore should put it all back.

> (2) What is the best why to copy this local backup to a remote location
> that does NOT have the possibility to run dsync.
> 
> - Question 1: is rsync safe to use and will this data work for restore?
> 
> - Question 2: Would it be safe to simply rsync the original mail_location
> to the remote server?

AFAICT, if Dovecot is stopped on both sides of the transfer it should be
safe. If either side has a currently running Dovecot instance (or any
other Dovecot tools, like a concurrent dsync) using that dbox, it's
likely rsync will copy an inconsistent snapshot of the data, and the
result will be corrupted.

If you have truly atomic filesystem snapshots, rsyncing out of a
snapshot also *ought* to be safe, since the situation is the same as if
Dovecot crashed at the moment the snapshot was taken, though it might be
a good idea to run doveadm force-resync to make sure the result is
consistent from Dovecot's point of view.

Ben