[Dovecot] Better to use a single large storage server or multiple smaller for mdbox?

[Ed W]

On 12/04/2012 02:18, Stan Hoeppner wrote:
> On 4/11/2012 11:50 AM, Ed W wrote:
>> Re XFS.  Have you been watching BTRFS recently?
>>
>> I will concede that despite the authors considering it production ready
>> I won't be using it for my servers just yet.  However, it's benchmarking
>> on single disk benchmarks fairly similarly to XFS and in certain cases
>> (multi-threaded performance) can be somewhat better.  I haven't yet seen
>> any benchmarks on larger disk arrays yet, eg 6+ disks, so no idea how it
>> scales up.  Basically what I have seen seems "competitive"
> Links?

http://btrfs.ipv5.de/index.php?title=Main_Page#Benchmarking

See the regular Phoronix benchmarks in particular.  However, I believe 
these are all single disk?


>> I don't have such hardware spare to benchmark, but I would be interested
>> to hear from someone who benchmarks your RAID1+linear+XFS suggestion,
>> especially if they have compared a cutting edge btrfs kernel on the same
>> array?
> http://btrfs.boxacle.net/repository/raid/history/History_Mail_server_simulation._num_threads=128.html
>
> This is with an 8 wide LVM stripe over 8 17 drive hardware RAID0 arrays.
>   If the disks had been setup as a concat of 68 RAID1 pairs, XFS would
> have turned in numbers significantly higher, anywhere from a 100%
> increase to 500%.

My instinct is that this is an irrelevant benchmark for BTRFS because 
its performance characteristics for these workloads have changed so 
significantly?  I would be far more interested in a 3.2 and then a 
3.6/3.7 benchmark in a years time

In particular recent benchmarks on Phoronix show btrfs exceeding XFS 
performance on heavily threaded benchmarks - however, I doubt this is 
representative of performance on a multi-disk benchmark?

> It would be nice to see these folks update these
> results with a 3.2.6 kernel, as both BTRFS and XFS have improved
> significantly since 2.6.35.  EXT4 and JFS have seen little performance
> work since.

My understanding is that there was a significant multi-thread 
performance boost for EXT4 in the last year kind of timeframe?  I don't 
have a link to hand, but someone did some work to reduce lock contention 
(??) which I seem to recall made a very large difference on multi-user 
or multi-cpu workloads?  I seem to recall that the summary was that it 
allowed Ext4 to scale up to a good fraction of XFS performance on 
"medium sized" systems? (I believe that XFS still continues to scale far 
better than anything else on large systems)

Point is that I think it's a bit unfair to say that little has changed 
on Ext4? It still seems to be developing faster than "maintenance only"

However, well OT...  The original question was: anyone tried very recent 
BTRFS on a multi-disk system.  Seems like the answer is no.  My proposal 
is that it may be worth watching in the future

Cheers

Ed W

P.S.  I have always been intrigued by the idea that a COW based 
filesystem could potentially implement much faster "RAID" parity, 
because it can avoid reading the whole stripe. The idea is that you 
treat unallocated space as "zero", which means you can compute the 
incremental parity with only a read/write of the checksum value (and 
with a COW filesystem you only ever update by rewriting to new "zero'd" 
space). I had in mind something like a fixed parity disk (RAID4?) and 
allowing the parity disk to be "write behind" cached in ram (ie exposed 
to risk of: power fails AND data disk fails at the same time).  My code 
may not be following along for a while though...