On 7/1/2012 5:48 AM, Charles Marcus wrote:
On 2012-07-01 3:17 AM, Stan Hoeppner stan@hardwarefreak.com wrote:
In a production environment, the mirror pairs will be duplexed across two SAS/SATA controllers.
Duplexing the mirrors makes a concat/RAID1, and a properly configured RAID10, inherently more reliable than RAID5 or RAID6, which simply can't be protected against controller failure.
Stan, am I correct that this - dual/redundant controllers - is the reason that a real SAN is more reliable than just running local storage on a mod-high end server?
In this case I was simply referring to using two PCIe SAS HBAs in a server, mirroring drive pairs across the HBAs with md, then concatenating the RAID1 pairs with md --linear. This gives protection against all failure modes. You can achieve the former with RAID5/6 but not the latter. Consider something like:
2x http://www.lsi.com/products/storagecomponents/Pages/LSISAS9200-8e.aspx 2x http://www.dataonstorage.com/dataon-products/6g-sas-jbod/dns-1640-2u-24-bay-... 48x Seagate ST9300605SS 300GB SAS 10k RPM
This hardware yields a high IOPS, high concurrency, high performance mail store. Drives are mirrored across HBAs and JBODs. Each HBA is connected to an expander/controller in both chassis, yielding full path redundancy. Each controller can see every disk in both enclosures. With this setup and SCSI multipath, you have redundancy against drive, HBA, cable, expander, and chassis failure. You can't get any more redundant than that. And you can't achieve this with RAID5/6, unless you simply mirror two of them across the HBAs/chassis. But that eliminates the whole reason for RAID5/6--space/cost efficiency.
People like Костырев Александр Алексеевич may wish to waste 3 more HBAs, JBOD chassis, and 24 more drives for a 3-way md mirror, to protect against the 1000 year scenario of two drives in the same mirror pair failing before the first is rebuilt.
A quality SAN head with dual redundant controllers will give you all of the above for all RAID levels, assuming you have multiple network connections (FC or iSCSI). You'd need two HBAs in the host, one connected to each controller. This is a direct connect scenario. In a fabric scenario, you'd have an independent FC or iSCSI switch on each path. And of course you need SCSI multipath configured on the host.
-- Stan