[Dovecot] RAID1+md concat+XFS as mailstorage

[Stan Hoeppner]

On 7/1/2012 5:48 AM, Charles Marcus wrote:
> On 2012-07-01 3:17 AM, Stan Hoeppner <stan at hardwarefreak.com> wrote:
>> In a production environment, the mirror pairs will be duplexed across
>> two SAS/SATA controllers.
>>
>> Duplexing the mirrors makes a concat/RAID1, and a properly configured
>> RAID10, inherently more reliable than RAID5 or RAID6, which simply can't
>> be protected against controller failure.
> 
> Stan, am I correct that this - dual/redundant controllers - is the
> reason that a real SAN is more reliable than just running local storage
> on a mod-high end server?

In this case I was simply referring to using two PCIe SAS HBAs in a
server, mirroring drive pairs across the HBAs with md, then
concatenating the RAID1 pairs with md --linear.  This gives protection
against all failure modes.  You can achieve the former with RAID5/6 but
not the latter.  Consider something like:

2x http://www.lsi.com/products/storagecomponents/Pages/LSISAS9200-8e.aspx
2x
http://www.dataonstorage.com/dataon-products/6g-sas-jbod/dns-1640-2u-24-bay-6g-25inch-sassata-jbod.html
48x Seagate ST9300605SS 300GB SAS 10k RPM

This hardware yields a high IOPS, high concurrency, high performance
mail store.  Drives are mirrored across HBAs and JBODs.  Each HBA is
connected to an expander/controller in both chassis, yielding full path
redundancy.  Each controller can see every disk in both enclosures.
With this setup and SCSI multipath, you have redundancy against drive,
HBA, cable, expander, and chassis failure.  You can't get any more
redundant than that.  And you can't achieve this with RAID5/6, unless
you simply mirror two of them across the HBAs/chassis.  But that
eliminates the whole reason for RAID5/6--space/cost efficiency.

People like Костырев Александр Алексеевич may wish to waste 3 more HBAs,
JBOD chassis, and 24 more drives for a 3-way md mirror, to protect
against the 1000 year scenario of two drives in the same mirror pair
failing before the first is rebuilt.

A quality SAN head with dual redundant controllers will give you all of
the above for all RAID levels, assuming you have multiple network
connections (FC or iSCSI).  You'd need two HBAs in the host, one
connected to each controller.  This is a direct connect scenario.  In a
fabric scenario, you'd have an independent FC or iSCSI switch on each
path.  And of course you need SCSI multipath configured on the host.

-- 
Stan