[Dovecot] Semi-OT: Backing up maildir

[grant beattie]

On Wed, May 10, 2006 at 07:29:02PM -0500, Shayne Hardesty wrote:

>  We switched our mail server over from mbox to maildir a few months ago and
> couldn't be happier - the performance under normal load is incredible.
> However we now have a problem with backup..  Typically we would run tar on
> the mail server, sending its output to another server via ssh..  Like this:
> tar cfp - /home | gzip | ssh other.server (cd /backups && tar xfp -)..  I
> have tried gzipping the stream on the source side, gzipping on the
> destination, and running without gzip, all three take an amazingly long time
> to complete (> 14 hours).  Ours is a modest server with about 1300 users,
> about 300GB of mail total.  The whole thing sits on hardware RAID-10, so I'm
> primarily guarding against a MAJOR hardware failure, a config blunder, or a
> hacker wiping out files.  How do you guys back up your maildir?  I've
> considered exporing the maildir with NFS and backing it up from another
> server..  Any opinions on that?

any maildir backup solution that involves reading files from the
filesystem will be inherently slower because there are simply lots
of files. a filesystem that supports snapshots is quite possibly the
most sensible choice for filesystems that contain lots of small files.

I am running maildir on ZFS, each user has their own ZFS file system,
and users' mail is backed up using "zfs snapshot" and "zfs send". as
these operate at the filesystem-block level, they are significantly
faster than any other backup method which reads files from the
filesystem.

also, since you mention it, a mirror is not going to help you in the
case of a config error, or a hacker trashing files, as you simply end
up with two copies of the same (possibly useless) bits on disk.

grant.