Storage Design (Regligious War, Whatever)

[Kelsey Cummings]

We've been running dovecot director setup using MAILDIR++ spools on NFS
served by 4 (aging) Netapp clustered filers and are considering
modernizing our storage.  There's nothing particularly wrong with the
existing storage except the age, density and power consumption don't
compare well when against new systems.  Refreshing the Netapps is an
option but they come at a substantial cost.

The last time we were looking one of the recommended hot DIY setups was
Linux NFS backed by XFS on N mirrors but this predated the availability
of stable ZFS in FreeBSD or Linux as well as products like FreeNAS.  A
FreeBSD NFS/ZFS filer on commodity hardware with zil/l2arc on a PCIe SSD
seems like an attractive, affordable and easily scaled out solution
which also would allow us to leverage compression at the filesystem layer.

Does anyone have any experience running ZFS spool storage?  If so, how
do you handle DR/HA for spool storage?  Thoughts on how this might
compare to using DRBD?  Any pitfalls to watch out for or general
pointers?  Suggestions on pool configuration?

-K