Up until now I've been backing up my IMAP server by taking an atomic snapshot of the entire VM, so I've not really had to worry about it. I'm moving to hosting where I have neither VM snapshots nor a filesystem that can take snapshots, so I need to think about consistent backups a bit more.
Will a simple backup of all the files of an multi-dbox give me a consistent state when I recover, or do I need to do something (e.g. lock writes to the mailbox) while I run a backup?
(I could use doveadm backup if I were backing up to local storage I controlled, but I'm pushing it to offsite buckets so I'd prefer to avoid backing up the entire thing locally, then copying that offsite).
Cheers, Steve
Been rsyncing my whole /var/vmail subdir (virtual users) for a while. So far so good. Yassine.
On Sunday, April 8, 2018, 11:56:29 PM GMT+1, Steve Atkins <steve@blighty.com> wrote: Up until now I've been backing up my IMAP server by taking an atomic snapshot of the entire VM, so I've not really had to worry about it. I'm moving to hosting where I have neither VM snapshots nor a filesystem that can take snapshots, so I need to think about consistent backups a bit more.
Will a simple backup of all the files of an multi-dbox give me a consistent state when I recover, or do I need to do something (e.g. lock writes to the mailbox) while I run a backup?
(I could use doveadm backup if I were backing up to local storage I controlled, but I'm pushing it to offsite buckets so I'd prefer to avoid backing up the entire thing locally, then copying that offsite).
Cheers, Steve
Am 2018-04-09 00:55, schrieb Steve Atkins:
Will a simple backup of all the files of an multi-dbox give me a consistent state when I recover, or do I need to do something (e.g. lock writes to the mailbox) while I run a backup?
If you are doing this backup with a fully operational mail storage: no, it won't. The reason is quite simple: mdbox saves the data of an email in several files, so there's a good chance that those files might slightly change during backup, while with Maildir all necessary data for the restore is stored in the file names.
If you want to do a consistent backup, you do need file system snapshots and should make sure that this snapshot is consistent or to turn off the mail store during backup completely.
But just doing rsync on the fly as with Maildir is just begging on your side to run into problems sooner or later...
-- Greetings,
Marc
Ah, I didn't know there was a diff. b/w maildir and mdbox in terms of storage. I should have mentionned I am using Maildir. Yassine.
On Monday, April 9, 2018, 1:35:15 PM GMT+1, Marc Stürmer <mail@marc-stuermer.de> wrote: Am 2018-04-09 00:55, schrieb Steve Atkins:
Will a simple backup of all the files of an multi-dbox give me a consistent state when I recover, or do I need to do something (e.g. lock writes to the mailbox) while I run a backup?
If you are doing this backup with a fully operational mail storage: no, it won't. The reason is quite simple: mdbox saves the data of an email in several files, so there's a good chance that those files might slightly change during backup, while with Maildir all necessary data for the restore is stored in the file names.
If you want to do a consistent backup, you do need file system snapshots and should make sure that this snapshot is consistent or to turn off the mail store during backup completely.
But just doing rsync on the fly as with Maildir is just begging on your side to run into problems sooner or later...
-- Greetings,
Marc
Le 4/9/2018 à 2:34 PM, Marc Stürmer a écrit :
Am 2018-04-09 00:55, schrieb Steve Atkins:
Will a simple backup of all the files of an multi-dbox give me a consistent state when I recover, or do I need to do something (e.g. lock writes to the mailbox) while I run a backup?
If you are doing this backup with a fully operational mail storage: no, it won't. The reason is quite simple: mdbox saves the data of an email in several files, so there's a good chance that those files might slightly change during backup, while with Maildir all necessary data for the restore is stored in the file names.
If you want to do a consistent backup, you do need file system snapshots and should make sure that this snapshot is consistent or to turn off the mail store during backup completely.
But just doing rsync on the fly as with Maildir is just begging on your side to run into problems sooner or later...
Personally, I always do a doveadm backup -u <users> to a different location (in mdbox format).
Then do a git -A, git commit, git repack *THEN* rsync the resulting .git directories to an off-site location (thus only sending the difference between one bakup and the next - in packed format).
The advantage is I can retrieve the status of any mailbox at any point in time (right now I have a year long of hour per hour backup of my servers mailboxes with a volume that is less than twice the size of the entire mailbox sizes).
Granted, it's a low volume, low trafic server, but it works !
--Ivan
(PS : Sorry Mark for the duplicate as I answered to you directly instead of the list)
On Apr 9, 2018, at 5:34 AM, Marc Stürmer <mail@marc-stuermer.de> wrote:
Am 2018-04-09 00:55, schrieb Steve Atkins:
Will a simple backup of all the files of an multi-dbox give me a consistent state when I recover, or do I need to do something (e.g. lock writes to the mailbox) while I run a backup?
If you are doing this backup with a fully operational mail storage: no, it won't. The reason is quite simple: mdbox saves the data of an email in several files, so there's a good chance that those files might slightly change during backup, while with Maildir all necessary data for the restore is stored in the file names.
If you want to do a consistent backup, you do need file system snapshots and should make sure that this snapshot is consistent or to turn off the mail store during backup completely
Thanks. Time to read up on snapshots!
Cheers, Steve
participants (4)
-
chaouche yacine
-
Ivan Warren
-
Marc Stürmer
-
Steve Atkins