Kádár Tamás (KTamas) put forth on 4/11/2011 12:20 PM:
Wow, thanks everyone for the replies.
Already using XFS as a filesystem with noatime. Switching to faster disks/raids/etc is not easy, we rent hosted server in Germany, to make real changes i'd have to get my own server etc.
Use relatime (the default) instead of noatime: http://xfs.org/index.php/XFS_FAQ#Q:_Is_using_noatime_or.2Fand_nodiratime_at_...
Far more importantly, using kernel 2.6.36+ with the 'delaylog' mount option can increase XFS metadata write performance by a factor of 10 to 100+ on metadata write heavy systems. If you're not already using this, update your kernel to 2.6.37.6 and mount your XFS filesystems with delaylog. Doing this is free but for your time and reboot downtime, and could land you a huge decrease in metadata IOPS hitting the physical storage, in essence giving you a free hardware upgrade.
I'll play around with disk schedulers and report back.
As I mentioned previously, changing the elevator won't address the problem if you have real hardware RAID with decent sized write cache (512MB or more) either PCIe HBA or FC/iSCSI SAN array. In the case of hardware RAID, use the 'noop' elevator.
-- Stan