[Dovecot] Better to use a single large storage server or multiple smaller for mdbox?

[Stan Hoeppner]

On 4/11/2012 11:50 AM, Ed W wrote:
> Re XFS.  Have you been watching BTRFS recently?
> 
> I will concede that despite the authors considering it production ready
> I won't be using it for my servers just yet.  However, it's benchmarking
> on single disk benchmarks fairly similarly to XFS and in certain cases
> (multi-threaded performance) can be somewhat better.  I haven't yet seen
> any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it
> scales up.  Basically what I have seen seems "competitive"

Links?

> I don't have such hardware spare to benchmark, but I would be interested
> to hear from someone who benchmarks your RAID1+linear+XFS suggestion,
> especially if they have compared a cutting edge btrfs kernel on the same
> array?

http://btrfs.boxacle.net/repository/raid/history/History_Mail_server_simulation._num_threads=128.html

This is with an 8 wide LVM stripe over 8 17 drive hardware RAID0 arrays.
 If the disks had been setup as a concat of 68 RAID1 pairs, XFS would
have turned in numbers significantly higher, anywhere from a 100%
increase to 500%.  It's hard to say because the Boxacle folks didn't
show the XFG AG config they used.  The concat+RAID1 setup can decrease
disk seeks by many orders of magnitude vs striping.  Everyone knows as
seeks go down IOPS go up.  Even with this very suboptimal disk setup,
XFS still trounces everything but JFS which is a close 2nd.  BTRFS is
way down in the pack.  It would be nice to see these folks update these
results with a 3.2.6 kernel, as both BTRFS and XFS have improved
significantly since 2.6.35.  EXT4 and JFS have seen little performance
work since.  In fact JFS has seen no commits but bug fixes and changes
to allow compiling with recent kernels.

> One of the snags of md RAID1 vs RAID6 is the lack of checksumming in the
> event of bad blocks.  (I'm not sure what actually happens when md
> scrubbing finds a bad sector with raid1..?).  For low performance
> requirements I have become paranoid and been using RAID6 vs RAID10,
> filesystems with sector checksums seem attractive...

Except we're using hardware RAID1 here and mdraid linear.  Thus the
controller takes care of sector integrity.  RAID6 yields nothing over
RAID10, except lower performance, and more usable space if more than 4
drives are used.

-- 
Stan