Javier de Miguel Rodríguez put forth on 12/13/2010 3:26 AM:
Can you give me (off-list if you desire) more info about your setup?I am interested in the number and type of spindles you are using. We are using LeftHand because of their real time replication capabilities, something very interesting to us, and each node pair is relatively cheap (8x450 GB@15K rpm sas disks per node, real time replication, 512 MB caché, about 25K € each node pair).
We can throw more hardware to this, let's see if using memory-basedindexes (via ramdisk) we get better results. Zlib compression on indexes should be great for this.
Javier, your number and type of spindles aren't the source of your I/O bottleneck. Your array controller is. Or, more specifically, its lack of enough fast network ports is the problem. The P4300 BK718A model of which you speak only has two 1GbE iSCSI ports. That's only 200 MB/s full duplex. That's less link bandwidth than a single 2 Gbit fiber channel interface. 2 Gbit FC is 2 generations and over 8 years old now. It was superseded by 4 Gbit FC and then 8 Gbit FC, which is the current standard. 8 Gbit FC provides 800 MB/s full duplex bandwidth per link.
Merely two of the 8 SAS drives in the P4300 unit can saturate the iSCSI interfaces on the controller. This has always been the downside of low end iSCSI arrays. The same eight 450GB 15k spindles backed by a decent controller with 512MB to 1GB cache and dual 4 Gbit FC ports or 10 GbE ports would give you well over double the bandwidth using the same RAID level you do now, and would increase IOPS substantially.
You don't need more or faster spindles to increase your performance and decrease interface load Javier. You need more interface throughput than your two 1 GbE ports currently provide. You either need more 1 GbE ports per P4300, say 4 of them, link bonded to your ethernet switch as a single 400 Mb/s full duplex channel (assuming the P4300 supports link bonding), or you need two 10 GbE ports per P4300, if it is upgradeable in this manner. If it is not, you need to move up to another HP model with much greater interface throughput, or a competing vendor's unit with 10 GbE or 8 Gbit FC ports. 1 Gbe iSCSI is a budget solution, not a performance solution, and I think that's what you're beginning to realize.
Nexsan sells a low end 14 drive SAS array with dual 2 GB cache active/active controllers for about the same price you're paying for the P4300, maybe a little less. It has dual 4 Gbit FC ports _and_ dual 1 GbE iSCSI ports. It stomps the P4300 in performance when using the FC ports, offering a sustained 800+ MB/s and around 4,300 IOPS to disk (50K+ to cache), but it doesn't have the advanced management features of the HP unit such as built-in snapshots, etc. IMHO VMware Consolidated Backup is a superior SAN backup solution than node-node replication on the P4000 series, which is just one more load sucking the life out of the two tiny 1 GbE ports. Using VCB with an FC SAN is a much more flexible and higher performance disaster recover solution than vendor specific hardware mirroring of two SAN arrays.
IMHO, you should look into a phased migration to an FC SAN. Qlogic's FC switches and HBAs are excellent, and affordable on a per port and per chassis basis. They are easy to setup. I taught myself to program the switches in the CLI in a day. The web gui may be even easier.
For about $50-60K USD you could have the big brother to the Nexsan unit I mentioned above. It's called the SASBeast. It contains dual active/active controllers, each with 2 GB mirrored cache, 2 8Gbit FC ports, and 2 1GbE iSCSI ports, for a total of 4 8Gb FC ports and 4 1GbE iSCSI ports. It contains 42 x 300, 450, or 600GB SAS 15k drives. It can be expanded with one additional chassis containing another 60 SAS or SATA drives for a total of 104 drives.
It offers sustained throughput of over 1.2 GB/s (about 4 times that of the P4300) and over 12,000+ IOPS to disk with 42 SAS drives, 50K+ to cache, increasing to approximately 30k IOPS to disk with a fully populated expansion chassis w/60 SAS 15K drives, 104 drives total in the array. This device's performance is controller limited, similar to the P4300, but due to the processor, not the interfaces, simply for the fact you can expand it to 104 SAS drives for less than $100K USD. If one needs maximum IOPS and bandwidth rather than total storage, one would be better off buying two SASBeasts than one unit with the expansion chassis. IIRC it's also a little cheaper.
There are many other fine SAN arrays on the market. I mention Nexsan merely because they are very affordable, fast as hell for the price, easy to manage, and I've used them. Due to the low price, they lack the more advanced features of higher priced units, such as snapshots, remote replication, etc. As I mentioned, I'm not a big fan of relying on SAN controllers to perform all my disaster recovery functions, so I do that with software, such as VCB, and a dedicated backup server with a SAN attached robotic library. Thus, the lack of this functionality in the Nexsan units is not an issue here.
-- Stan