Stan Hoeppner <stan@hardwarefreak.com> wrote:
Sven, why didn't you chime in? Your setup is similar scale and I think your insights would be valuable here. Or maybe you could repost your last on this topic. Or was that discussion off list? I can't recall.
Rather busy right now with a large scale Identity Management+AD rollout here, so unfortunately not too much time to elaborate my setup in great detail.
But after testing the nothing-shared-6-node-cluster setup with imapc as the backend for shared folders I concluded that this does not scale very well (the imapc-part, that is) and changed my plans to an director-based NFS-backed (Netapp 3240) setup, which is much more common.
I reckoned I'd be nearly the only one on this planet to be so crazy to try to use a backwards-normal-user-as-master-user-for-imapc setup for shared folders and that having anyone other than me understanding that setup, let alone getting support for it, would be to big a hassle.
So I put the mdbox storage on two 15k-SAS-NetApp with 1TB FlashCache, connected with 2x 10GBit to the SAN, using NFS to mount the volumes in my 6 backend-dovecot servers, putting 2 director-dovecots in front, which will sit behind a Linux IPVS loadbalancer. All systems are VMs on ESX.
I recently added two more shelves with SATA drives to the NetApp to use as storage for the alt-storage feature of dovecot to automatically migrate mails older than 180 days to less expensive storage.
As of now, the system is not yet live (see IDM rollout above), I hope to resume my migration in late spring, early summer.
But during initial synthetic benchmarks have show that this setup will be more than sufficient to provide the needed oompf for my 15k users, with enough room to grow.
Interesting datapoint: NetApp Deduplication did only recover about 1% of storage space with mdbox-based mail storage, while on an maildir-based mail storage, the rate was about 15%. (This was tested with a copy of real user data, so is accurate for my workload.)
Grüße, Sven.
-- Sigmentation fault. Core dumped.