Hi Stan,
On Sat, Aug 7, 2010 at 10:27 PM, Stan Hoeppner stan@hardwarefreak.comwrote:
Care to elaborate on this point? The NFS server sits in user space. All cluster filesystem operations take place in kernel space.
If you expand your definition of NFS server to include high end systems (NetApp being a common example), the placement of the NFS server isn't necessarily limited to user space. Some vendors like BlueArc, use FPGAs to handle the protocols. ORT in many cases is less than 1 msec on some of these boxes.
econd range.
Dovecot clusters may be simpler to implement using NFS storage servers,
Simpler and more cost effective. The price / performance (per watt if you want to go that far seeing as you don't need 2 fabrics ) generally favor NAS or some other kind of distributed file system based approach. The gains that come from parallelization are worth it at the cost of slightly less performance on an individual node basis, especially if you're dealing with N+2 or greater availabilty schemes.
These Distributed File Systems and specialilzed RPC mechanisms have higher overhead than even NFS, but they make up for it by increasing paralleization and using some very creative optimizations that you can use when you have many multiple machines and some other things that are don't have useful analogs outside of Google.
|In a nutshell, you divide the aggregate application data equally across a |number of nodes with local storage, and each node is responsible for handling |only a specific subset of the total data.
You can go the same thing with NFS nodes, with the added benefit using the automounter (on the low end) to "virtualize" the name space similar to what they do with render farms.
|The cluster host numbers I'm using are merely examples. Google for example |probably has a larger IMAP cluster server count per datacenter than the 256 |nodes in my example--that's only about 6 racks packed with 42 x 1U servers. |Given the number of gmail accounts in the US, and the fact they have less than |2 dozen datacenters here, we're probably looking at thousands of 1U IMAP |servers per datacenter.
The architecture you describe is very similar to webex, but they cap the number of accounts per node at some ridiculously small level, like 10,000 or something and use SAS drives.
~Max