The reason is that few if any organizations actually need 28TB (14 2TB Cavier Green drives--popular with idiots today) of mail storage in a single mail store. That's 50 years worth of mail storage for a 50,000 employee company, assuming your employees aren't allowed porn/video attachments, which which most aren't.
WTF? 28TB of mail storage for some is rather small. Good to see your still posting without a clue Stanley. Remember there is a bigger world out there from your tiny SOHO
I'm with you Noel.
We just bought 252TB of raw disk for about 5k users. Given, this is going in to Exchange on Netapp with multi-site database replication, so this cooks down to about 53TB of usable space with room for recovery databases, defragmentation, archives, etc, but still... 28TB is not much anymore.
Of course, Exchange has also gone in a different direction than folks have been indicating. 2010 has some pretty high memory requirements, but the actual IOPS demands are quite low compared to earlier versions. We're using 1TB 7200RPM SATA drives, and at the number of spindles we've got, combined with the cache in the controllers, expect to have quite a good bit of excess IOPS.
Even on the Dovecot side though - if you use the Director to group your users properly, and equip the systems with enough memory, disk should not be a bottleneck if you do anything reasonably intelligent. We support 12k concurrent IMAP users at ~.75 IOPS/user/sec. POP3, SMTP, and shell access on top of that is negligible.
I'm also surprised by the number of people trying to use DRBD to make local disk look like a SAN so they can turn around and put a cluster filesystem on it - with all those complex moving parts, how do you diagnose poor performance? Who is going to be able to support it if you get hit by a bus? Seems like folks would be better off building or buying a sturdy NFS server. Heck, even at larger budgets, if you're probably just going to end up with something that's essentially a clustered NFS server with a SAN behind it.
-Brad