[Dovecot] Maildir over NFS

[Maxwell Reid]

Hi Stan,

On Sat, Aug 7, 2010 at 10:27 PM, Stan Hoeppner <stan at hardwarefreak.com>wrote:

>
>
>
>
> Care to elaborate on this point?  The NFS server sits in user space.  All
> cluster filesystem operations take place in kernel space.
>

If you expand your definition of NFS server to include high end systems
(NetApp being a common example), the placement of the NFS server isn't
necessarily limited to user space.  Some vendors like BlueArc, use FPGAs to
handle the protocols.   ORT in many cases is less than 1 msec on some of
these boxes.




> econd
> range.
>
> Dovecot clusters may be simpler to implement using NFS storage servers,


Simpler and more cost effective.  The price / performance (per watt if you
want to go that far seeing as you don't need 2 fabrics ) generally favor NAS
or some other kind of  distributed file system based approach.  The gains
that come from parallelization are worth it at the cost of slightly less
performance on an individual node basis, especially if you're dealing with
N+2 or greater availabilty schemes.

  These Distributed File Systems and specialilzed RPC mechanisms  have
higher overhead than even NFS, but they make up for it by increasing
paralleization and using some very creative optimizations that you can use
when you have many multiple machines and some other things that are don't
have useful analogs outside of Google.


 |In a nutshell, you divide the aggregate application data equally across a
 |number of nodes with local storage, and each node is responsible for
handling
 |only a specific subset of the total data.


You can go the same thing with NFS nodes, with the added benefit using the
automounter (on the low end) to "virtualize" the name space similar to what
they do with render farms.



|The cluster host numbers I'm using are merely examples.  Google for example
|probably has a larger IMAP cluster server count per datacenter than the 256
|nodes in my example--that's only about 6 racks packed with 42 x 1U servers.
|Given the number of gmail accounts in the US, and the fact they have less
than
|2 dozen datacenters here, we're probably looking at thousands of 1U IMAP
|servers per datacenter.

The architecture you describe is very similar to webex, but they cap the
number of accounts per node at some ridiculously small level, like 10,000 or
something and use SAS drives.

~Max