[Dovecot] RAID benchmarks

[Ethan Sommer]

We (a small college with about 3000 active accounts) are currently in 
the process of moving from UW IMAP running on linux to dovecot running 
on a cluster of 3 or 4 new faster Linux machines. (Initially using 
perdition to split the load.)

As we are building and designing the system, I'm attempting to take (or 
find) benchmarks everywhere I can in order to make informed decisions 
and so that when we need something faster we have something to compare 
it to. I've looked all over the Internet and I haven't seen very many 
good benchmarks running on recent hardware for things like this. Even 
simple things like how different raid levels perform with fast CPUs and 
really fast disks.

We recently got the first of the cluster machines (which has the 
following specs: (from penguin computing)
2x 1.6ghz woodcrest (so 4 cores total)
2g ram (we may upgrade to more, depending on what we seem to need)
6x 15k SAS 73G hard drives.

I've always read that RAID 10 is the RAID of choice for mail, because 
the checksums take time and whatnot, but on a system that fast I'm sure 
that's not an issue (which I could see from the cpu usage during 
benchmarks.) We were also wondering whether RAID6 would be a good option 
since it would give more redundancy. (raid6 allows 2 drive failures)

All tests were run on ext3 with all 6 disks in the raid. Everything is 
using Linux software raid.
I used bonnie++ to run the benchmarks, and here are the initial results 
we got:
       Output                   Input
RAID  "Per Chr" "Block" "Per Chr" "Block"
RAID6  34628,    97983,  23316,   39077
RAID5  39559,    32715,  32036,   52231
RAID10 38816,   229062,  40645,  267418

I assume that mail running maildir (as we intend to do) is somewhere 
between writing one char at a time and writing huge blocks to fill 4g 
files. Since the Per Chr values seemed to have maxed out the CPU and 
we'll have 4 cores available, I tried the same thing running 4 copies of 
bonnie at once and adding the results. (I worked backwards and didn't 
bother with raid 6 for obvious reasons)
       Output                   Input
RAID    "Per Chr" "Block" "Per Chr" "Block"
RAID5   37465,    37095,  102828,   174263
RAID10 111239,   128287,   93400,   230320


So, if our system is ever running full out, basically disk bound, it 
should be able to handle about 4X as many writes and at least as many 
reads using RAID10 than raid 5, and since raid 6 was pretty similar to 
raid 5 in the first tests, I assume it will be here too.


I have the full bonnie results if anyone's interested.

I also plan to come up with some basic benchmarks as we start to 
actually have dovecot working as we want it to. (Possibly using the macs 
in our labs and XGrid as a "client".. finally a use for that thing :) )

Do people have suggestions as to what to test for a primarily IMAP (+ a 
little POP) server? I was thinking of doing something similar to 
http://www.courier-mta.org/mbox-vs-maildir/ which selects a mail box, 
deletes and expunges a message, fetches a few messages, and then does 
some searches.

Also, while I have the opportunity, are there any other benchmarks I 
should run? I suppose I could compare dovecot with mbox vs maildir, with 
indexing and without. Dovecot with indexing but the indexes deleted, 
etc. I could compare dovecot to uw imap.

Anything else? I probably can't justify the time to get cyrus up and 
running. (I did that a few months ago when deciding which server to use...)

Ethan Sommer

-- 
--
Ethan Sommer
Systems Administrator
Gustavus Adolphus College
507-933-7042
sommere at gac.edu