On Wed, May 10, 2006 at 07:29:02PM -0500, Shayne Hardesty wrote:
We switched our mail server over from mbox to maildir a few months ago and couldn't be happier - the performance under normal load is incredible. However we now have a problem with backup.. Typically we would run tar on the mail server, sending its output to another server via ssh.. Like this: tar cfp - /home | gzip | ssh other.server (cd /backups && tar xfp -).. I have tried gzipping the stream on the source side, gzipping on the destination, and running without gzip, all three take an amazingly long time to complete (> 14 hours). Ours is a modest server with about 1300 users, about 300GB of mail total. The whole thing sits on hardware RAID-10, so I'm primarily guarding against a MAJOR hardware failure, a config blunder, or a hacker wiping out files. How do you guys back up your maildir? I've considered exporing the maildir with NFS and backing it up from another server.. Any opinions on that?
any maildir backup solution that involves reading files from the filesystem will be inherently slower because there are simply lots of files. a filesystem that supports snapshots is quite possibly the most sensible choice for filesystems that contain lots of small files.
I am running maildir on ZFS, each user has their own ZFS file system, and users' mail is backed up using "zfs snapshot" and "zfs send". as these operate at the filesystem-block level, they are significantly faster than any other backup method which reads files from the filesystem.
also, since you mention it, a mirror is not going to help you in the case of a config error, or a hacker trashing files, as you simply end up with two copies of the same (possibly useless) bits on disk.
grant.