On Tue, Feb 5, 2013 at 10:41 PM, Joe Beaubien joe.beaubien@gmail.comwrote:
On Mon, Feb 4, 2013 at 9:01 PM, Timo Sirainen tss@iki.fi wrote:
On Mon, 2013-02-04 at 00:57 +0000, Ben Morrow wrote:
I can't give authoratitive answers to either of these, but...
At 6PM -0500 on 3/02/13 you (Joe Beaubien) wrote:
I'm currently trying to setup remote backups of my emails but i'm
running
into issues (mdbox format, indexes and storage in the same folder hierarchy).
Local backup command: dsync -u "my_user" backup /backups/my_user
(1) Recently, I noticed that the local backup takes up twice the size as the original mail location (8gb vs. 4gb). I purged alot of emails from the original location, so the size shrunk, but the local backup just keeps on getting bigger. I couldn't find any dsync option that would delete extra emails.
- Question: Why isn't the local backup synced properly and remove the extra emails?
Are you running 'doveadm purge' on the backed-up dbox? It looks to me as though dsync doesn't do that. I don't know if there's any (simple) way to do that without a running Dovecot instance attached the dbox directory: it's not entirely clear to me whether doveadm will run locally without contacting a doveadm-server instance running under Dovecot, nor how to point 'doveadm purge' at an arbitrary directory.
Right. doveadm -o mail=mdbox:/backups/my_user purge
This worked (at least it seems to, the source and destination are roughly the same size)..
However, if the the original email location has already been purge, does the backup email location also need to be purged?
It might be easier to dsync to a Maildir instead. This should preserve all the Dovecot-visible metadata, and dsyncing back to the dbox for restore should put it all back.
Better sdbox than maildir.
I'd rather stick to mdbox for my remote backups. I have a single email account with over 1.5 million emails in it. With a 1-file-per-message storage, this would be slow/hell to run. Unless there is a better way.
(2) What is the best why to copy this local backup to a remote location that does NOT have the possibility to run dsync.
Question 1: is rsync safe to use and will this data work for restore?
Question 2: Would it be safe to simply rsync the original mail_location to the remote server?
AFAICT, if Dovecot is stopped on both sides of the transfer it should be safe. If either side has a currently running Dovecot instance (or any other Dovecot tools, like a concurrent dsync) using that dbox, it's likely rsync will copy an inconsistent snapshot of the data, and the result will be corrupted.
It won't be badly corrupted though. At worst Dovecot will rebuild the indexes, which takes a while. And most users probably won't get any corruption at all.
I think there was a misunderstanding of the setup. dsync is only running on the local side, the remote side is a "dumb" rsync server that I don't fully control.
what i was trying to ask with my last question is the following:
I'm trying to do remote backups of an mdbox setup, and considering that I only have dsync running on the local side (not on the remote side), is it safe to simply do an rsync of the mail_location to the remote server, or should I first do a dsync (make a local duplicate) and then rsync the duplcate out to the remote server?
(wish i had a whiteboard right about now).
Thanks,
-Joe