[Dovecot] Best Cluster Storage

[Jonathan Tripathy]

On 15/01/11 00:59, Eric Shubert wrote:
> On 01/14/2011 03:58 PM, Jonathan Tripathy wrote:
>>
>> On 14/01/11 19:00, Stan Hoeppner wrote:
>>> Jonathan Tripathy put forth on 1/13/2011 4:17 PM:
>>>
>>>> Regarding the servers, I was thinking of having a 2 node drbd cluster
>>>> (in
>>>> active+standby), which would export a single iSCSI LUN. Then, I would
>>>> have a 2
>>>> node dovecot+postfix cluster (in active-active), where each node
>>>> would mount the
>>>> same LUN (With GFS2 on top). This is 4 servers in total (Well, 4 VMs
>>>> running on
>>>> 4 physically separate servers).
>>> Something you need to consider very carefully:
>>>
>>> drbd is a kernel block storage driver. You run in ON a PHYSICAL
>>> cluster node,
>>> and never inside a virtual machine guest. drbd is RAID 1 over a
>>> network instead
>>> of a SCSI cable. Is is meant to protect against storage and node
>>> failures.
>>> This is how you need to look at drbd. Again, DO NOT run DRBD inside of
>>> a VM
>>> guest. If you have a decent background in hardware and operating
>>> systems, it
>>> won't take you 30 seconds to understand what I'm saying here. If it
>>> takes you
>>> longer, then consider this case:
>>>
>>> You have a consolidated Xen cluster of two 24 core AMD Magny Cours
>>> servers each
>>> with 128GB RAM, an LSI MegaRAID SAS controller with dual SFF8087 ports
>>> backed by
>>> 32 SAS drives in external jbod enclosures setup as a single hardware
>>> RAID 10.
>>> You spread your entire load of 97 virtual machine guests across this
>>> two node
>>> farm. Within this set of 97 guests, 12 of them are clustered network
>>> applications, and two of these 12 are your Dovecot/Postfix guests.
>>>
>>> If you use drbd in the way you currently have in your head, you are
>>> mirroring
>>> virtual disk partitions with drbd _SIX times_ instead of once. Here,
>>> where
>>> you'd want to run drbd is within the Xen hypervisor kernel. drbd works
>>> at the
>>> BLOCK DEVICE level, not the application layer.
>>>
>>> Eric already mentioned this once. Apparently you weren't paying
>>> attention.
>>>
>> I'm sorry I don't follow this. It would be appreciated if you could
>> include a simpler example. The way I see it, a VM disk is just a small
>> chunck "LVM LV in my case" of a real disk.
>>
>
> Perhaps if you were to compare and contrast a virtual disk to a raw 
> disk, that would help. If you wanted to use drbd with a raw disk being 
> accessed via a VM guest, that would probably be all right. Might not 
> be "supported" though.
Thanks Eric, Now I understand where you are coming from: It's not the 
fact that DRBD is running in a VM is the problem, is the fact that DRBD 
should be replicating a raw physical disk, which of course is still 
possible from with a Xen VM

Also thanks to Stan and everyone else for the helpful comments.

I still haven’t decided between GFS2 or OCFS2 yet. I guess I'll have to 
try both and see what works the best.

I really wish NFS didn't have the caching issue, as it's the most simple 
to set up