I have been thinking, and reading, on how to back up my mailserver. I have not found any approach that seems ready to use.
I have run years without any backup, but would really like to have something in place. I figure I can attach a USB drive and backup to that, then from there rsync to something elsewhere. Further if that USB drive is a full mailserver image, I actually have a 'hot backup' where I only have to put the backup drive into a system and boot up at the last backup.
But this means properly copying all of /home/vmail and probably /home/sieve plus the /var/lib/mysql
Are there good tools that nicely does this? Or do I choose a time late at night (only I am sometimes in non-US timezones) to shut down all services and just use rsync?
And stopping services itself is thought provoking. What if Dovecot, amavis, mysql, or whatelse is in the middle of writing out a mail file what happens to that file and restart.
Just scary stuff and, in part, why I have never tackled this in the past.
thanks for all feedback
On 10/02/19 4:50 PM, Robert Moskowitz via dovecot wrote:
I have been thinking, and reading, on how to back up my mailserver. I have not found any approach that seems ready to use.
I have run years without any backup, but would really like to have something in place. I figure I can attach a USB drive and backup to that, then from there rsync to something elsewhere. Further if that USB drive is a full mailserver image, I actually have a 'hot backup' where I only have to put the backup drive into a system and boot up at the last backup.
But this means properly copying all of /home/vmail and probably /home/sieve plus the /var/lib/mysql
Are there good tools that nicely does this? Or do I choose a time late at night (only I am sometimes in non-US timezones) to shut down all services and just use rsync?
And stopping services itself is thought provoking. What if Dovecot, amavis, mysql, or whatelse is in the middle of writing out a mail file what happens to that file and restart.
I use dirvish, which is based on rsync, and which only syncs changed files but uses hardlinks to keep storage and bandwidth down - it leaves me with a full copy of the tree for each date, with links between. I run it on a machine at home.
I don't ususally worry about stopping services; maildir should be all right. For databases, I use the backup tools provided to back up the running database, then use dirvish on the backups (and skip the database files themselves).
I do do it at night, to save load on the server (it's an LXC container on a VPS).
I believe rdiffbackup does something similar to dirvish.
Richard
Hey,
A few years back, I was also suffered for how to backup Maildir data files and then, I came to know about the Maildir Converter <https://www.cubexsoft.com/maildir/> solution. It has transferred my all Maildir files to desktop without showing any single error. You can also try this same solution.
-- Sent from: http://dovecot.2317879.n4.nabble.com/
Hello Robert,
----- Nachricht von Robert Moskowitz via dovecot <dovecot@dovecot.org>
Datum: Sat, 9 Feb 2019 22:50:24 -0500
Von: Robert Moskowitz via dovecot <dovecot@dovecot.org>Antwort an: Robert Moskowitz <rgm@htt-consult.com>, Dovecot Mailing
List <dovecot@dovecot.org>
Betreff: How to backup maildir
An: Dovecot Mailing List <dovecot@dovecot.org>
I have been thinking, and reading, on how to back up my mailserver.
I have not found any approach that seems ready to use.I have run years without any backup, but would really like to have
something in place.
you're a really lucky guy! - I've been struck in the past for such
carelessness on the one or other machine with dataloss ;-)
I figure I can attach a USB drive and backup to that, then from
there rsync to something elsewhere. Further if that USB drive is a
full mailserver image, I actually have a 'hot backup' where I only
have to put the backup drive into a system and boot up at the last
backup.But this means properly copying all of /home/vmail and probably
/home/sieve plus the /var/lib/mysql
Are you aware of the dovecot command "dsync"? (man dsync or
https://wiki.dovecot.org/Tools/Doveadm/Sync)
This could be an approach of using dsync:
dsync backup -o plugin/quota= -f -u $user backup
maildir:/mnt/USB/dovecot-backup/Maildir/$user/mail
Are there good tools that nicely does this? Or do I choose a time
late at night (only I am sometimes in non-US timezones) to shut down
all services and just use rsync?And stopping services itself is thought provoking. What if Dovecot,
amavis, mysql, or whatelse is in the middle of writing out a mail
file what happens to that file and restart.Just scary stuff and, in part, why I have never tackled this in the past.
thanks for all feedback
----- Ende der Nachricht von Robert Moskowitz via dovecot
<dovecot@dovecot.org> -----
It really depends on how important your data is to you ... But you
should really think about a general backup-strategy!
"Mr. Google" can help you to get some ideas how YOUR backup-strategy
could look like...
Also there is much input for backing up dovecot with it's different
mail storage flavours.
- But you have to invest some effort on your own, to search, read,
evaluate and finally choose what's fitting into YOUR setup!
But as an starting point:
I'm using a for _MY setup_ modified and adopted version of Klaus
Tachtler's dovecot-backup script:
https://github.com/tachtler/dovecot-backup/blob/master/dovecot_backup.sh
... mixed it with Borg Backup: https://www.borgbackup.org/
... some further encryption, cloud storage and ... and ... and other stuff.
But as above mentioned:
YOU have to think about the grade of your paranoia level, how
importand the data is to you in case of an data loss, time and money
you are willing to invest and build upon this YOUR PERSONAL backup
strategy.
- Sadly there is no one-size-fits-all!
Last famous words:
I've looked at your vita and was wondering about your post - you were
writing RFCs, but have no clue about backing up your mail-data???
Strange ...
Nevermind! Just my 2ct - hope this helps Christoph.
-- Christoph Haas
On 2/10/19 2:24 AM, Christoph Haas via dovecot wrote:
Hello Robert,
----- Nachricht von Robert Moskowitz via dovecot <dovecot@dovecot.org>
Datum: Sat, 9 Feb 2019 22:50:24 -0500 Von: Robert Moskowitz via dovecot <dovecot@dovecot.org> Antwort an: Robert Moskowitz <rgm@htt-consult.com>, Dovecot Mailing List <dovecot@dovecot.org> Betreff: How to backup maildir An: Dovecot Mailing List <dovecot@dovecot.org>
I have been thinking, and reading, on how to back up my mailserver. I have not found any approach that seems ready to use.
I have run years without any backup, but would really like to have something in place.
you're a really lucky guy! - I've been struck in the past for such carelessness on the one or other machine with dataloss ;-)
Absolute laziness. No real excuse. Also all users were POPing until 2 years ago. Finally got everyone on IMAP just in the last year. So a server loss would have been an inconvenience. For myself, almost nothing in the IMAP store, everything in local folders that I have a separate backup procedure.
I figure I can attach a USB drive and backup to that, then from there rsync to something elsewhere. Further if that USB drive is a full mailserver image, I actually have a 'hot backup' where I only have to put the backup drive into a system and boot up at the last backup.
But this means properly copying all of /home/vmail and probably /home/sieve plus the /var/lib/mysql
Are you aware of the dovecot command "dsync"? (man dsync or https://wiki.dovecot.org/Tools/Doveadm/Sync) This could be an approach of using dsync: dsync backup -o plugin/quota= -f -u $user backup maildir:/mnt/USB/dovecot-backup/Maildir/$user/mail
My search foo is weak. This is a long documented fact. I did spend a number of hours searching and reading before opening my mouth here, fully expecting to put more than my toes in.
I will read up on dsync.
Are there good tools that nicely does this? Or do I choose a time late at night (only I am sometimes in non-US timezones) to shut down all services and just use rsync?
And stopping services itself is thought provoking. What if Dovecot, amavis, mysql, or whatelse is in the middle of writing out a mail file what happens to that file and restart.
Just scary stuff and, in part, why I have never tackled this in the past.
thanks for all feedback
----- Ende der Nachricht von Robert Moskowitz via dovecot <dovecot@dovecot.org> -----
It really depends on how important your data is to you ... But you should really think about a general backup-strategy! "Mr. Google" can help you to get some ideas how YOUR backup-strategy could look like... Also there is much input for backing up dovecot with it's different mail storage flavours.
- But you have to invest some effort on your own, to search, read, evaluate and finally choose what's fitting into YOUR setup!
I have been and have been searching. Some hits, but so far nothing was hitting the spot. But, I will blame my dyslexia that my search foo is weak.
But as an starting point: I'm using a for _MY setup_ modified and adopted version of Klaus Tachtler's dovecot-backup script: https://github.com/tachtler/dovecot-backup/blob/master/dovecot_backup.sh
... mixed it with Borg Backup: https://www.borgbackup.org/
... some further encryption, cloud storage and ... and ... and other stuff.
I will check both of these.
But as above mentioned: YOU have to think about the grade of your paranoia level, how importand the data is to you in case of an data loss, time and money you are willing to invest and build upon this YOUR PERSONAL backup strategy.
- Sadly there is no one-size-fits-all!
If there was, we would not be here, I suspect.
Last famous words: I've looked at your vita and was wondering about your post - you were writing RFCs, but have no clue about backing up your mail-data??? Strange ...
MY mail is well backed up. Locally so that I can read on a plane and the like. I have been running one flavor or another of my own mail server since '95. I switched to dovecot 6 years ago.
My home file server is backed up 4 ways around. And I have lost file servers and file server drives over the years. Upgrading my file server is the next project.
But I write RFCs. I have been using geany for XML for a few years. I 'code' in English. I have not written computer code since probably the late 80s. I never coded in C, but I did use B for a while! These days I can write simple scripts when forced to. :)
My 'spare' time these days is working with armv7 boards. I have been using Cubietech since '13, recently got an Odroid HC1 and that is what this server will be. For the most part you will find me on the Centos/Fedora lists and their arm lists. Occasionally I will put on my Kevlar suit and ask something basic on a product list.
But as a result of doing this, recently there is a new SElinux policy for permitting Dovecot to access Mysql. It has already been patched into Fedora 28 and up. Sometimes it pays to ask for help.
Nevermind! Just my 2ct - hope this helps Christoph.
Euro ct? :)
Better than US ct!
Hello Robert,
[... snip ...]
of course I'm totally with you: asking other people for help, is often
a good - if even not the only way to getting things done. It was not
my intention to insult you! I hope this did not come in to your mind ...
Personally I would have a look at the mentioned Dovecot-backup-script
as a start. It does really a very good job! Cudos to Klaus Tachtler!
Another option could be, to sync your mail via mbsync/isync or
offlineimap to your Notebook ... but as an alternative backup, it
depends on how many users are on your Dovecot-server.
In a second cycle, you can then extend or modify this script - as I
have been doing.
But you should bear in mind, that you should have at least 2-3
replicas of your data on different storage, for having a good backup.
Cheers Christoph.
-- Christoph Haas
On 2/10/19 8:21 AM, Christoph Haas wrote:
Hello Robert,
[... snip ...]
of course I'm totally with you: asking other people for help, is often a good - if even not the only way to getting things done. It was not my intention to insult you! I hope this did not come in to your mind ...
You did not insult me at all. I have taken stronger barbs over the years!
Personally I would have a look at the mentioned Dovecot-backup-script as a start. It does really a very good job! Cudos to Klaus Tachtler!
That is on top of my list. Thanks for the pointer.
Another option could be, to sync your mail via mbsync/isync or offlineimap to your Notebook ... but as an alternative backup, it depends on how many users are on your Dovecot-server.
I have 4 domains., 20+ users. Small stuff. I suspect that would only work for me, and I have my processes in place.
In a second cycle, you can then extend or modify this script - as I have been doing.
But you should bear in mind, that you should have at least 2-3 replicas of your data on different storage, for having a good backup.
The local image is for 'fast' backup. This will then be rsynced to a server in my neighbor's house (we have ethernet between us. He lunches off my ISP connection, he hosts my 'offsite' backups).
Cheers Christoph.
participants (4)
-
arohi01
-
Christoph Haas
-
Richard Hector
-
Robert Moskowitz