[Dovecot] GPFS for mail-storage (Was: Re: Compressing existing maildirs)

[Jan-Frode Myklebust]

On Wed, Jan 04, 2012 at 12:09:39AM -0600, list at airstreamcomm.net wrote:
>  Could you remark on GPFS services hosting mail storage over a WAN between two geographically separated data centers?

I haven't tried that, but know the theory quite well. There are  2 or 3 options:

	1 - shared SAN between the data centers. Should work the same as
	     a single data center, but you'd want to use disk quorum or
	     a quorum node on a 3. site to avoid split brain.

	2 - different SANs on the two sites. Disks on SAN1 would belong
	    to failure group 1 and disks on SAN2 would belong to failure 
	    group 2. GPFS will write every block to disks in different
	    failure groups. Nodes on location 1 will use SAN1 directly,
	    and write to SAN2 via tcp/ip to nodes on location 2 (and vica
	    versa). It's configurable if you want to return success when
	    first block is written (asyncronous replication), or if you
	    need both replicas to be written. Ref: mmcrfs -K:

		http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=%2Fcom.ibm.cluster.gpfs.v3r4.gpfs300.doc%2Fbl1adm_mmcrfs.html

	   With asyncronous replication it will try to allocate both
	   replicas, but if it fails you can re-establish the
	   replication level later using "mmrestripefs".

	   Reading will happen from a direct attached disk if possible,
	   and over tcp/ip if there are no local replica of the needed
	   block.

	   Again you'll need a quorum node on a 3. site to avoid split brain.


	3 - GPFS multi-cluster. Separate GPFS clusters on the two
	    locations. Let them mount each others filesystems over IP,
	    and access disks over either SAN or IP network. Each cluster is
	    managed locally, if one site goes down the other site also
	    loses access to the fs.

	    I don't have any experience with this kind of config, but believe
	    it's quite popular to use to share fs between HPC-sites.

		http://www.ibm.com/developerworks/systems/library/es-multiclustergpfs/index.html
		http://www.cisl.ucar.edu/hss/ssg/presentations/storage/NCAR-GPFS_Elahi.pdf


  -jf