[Dovecot] Maildir over NFS

[Stan Hoeppner]

Noel Butler put forth on 8/6/2010 4:29 PM:

> Actually you will not notice any difference. How do you think all the
> big boys do it now :)  Granted some opted for the SAN approach over NAS,
> but for mail, NAS is better way to go IMHO and plenty of large services,
> ISP, corporations, and universities etc, all use NAS.

The protocol overhead of the NFS stack is such that one way latency is in the
1-50 millisecond range, depending on specific implementations and server load.
 The one way latency of a fibre channel packet is in the sub 100 microsecond
range and is fairly immune to system load.  The performance of fibre channel
is equal to local disk plus approximately one millisecond of additional
effective head seek time due to switch latency, SAN array controller latency,
and latency due to cable length.  A filesystem block served out of SAN array
controller cache returns to the kernel quicker than a block read from local
disk that is not in cache because the former suffers no mechanical latency.
Due to the complexity of the stack, NFS is far slower than either.

Those who would recommend NFS/NAS over fibre channel SAN have no experience
with fibre channel SANs.  I'm no fan of iSCSI SANs due to the reliance on
TCP/IP for transport, and the low performance due to stck processing.
However, using the same ethernet switches for both, iSCSI SAN arrays will also
outperform NFS/NAS boxen by a decent margin.

Regarding the OP's case, given the low cost of new hardware, specifically
locally attached RAID and the massive size and low cost of modern disks, I'd
recommend storing user mail on the new mail host.  It's faster and more cost
effective than both NFS/SAN.  Unless his current backup solution "requires"
user mail dirs to be on that NFS server for nightly backup, local disk is
definitely the way to go.  Four 300GB 15k SAS drives on a good PCIe RAID card
w/256-512MB cache in a RAID 10 configuration would yield ~350-400MB/s of real
filesystem bandwidth, seek throughput equivalent to a 2 disk stripe--about 600
random seeks/s, 600GB of usable space, ability to sustain two simultaneous
disk failures (assuming 1 failure per mirror pair), and cost effectiveness.

-- 
Stan