dovecot replication (active-active) - server specs

[Martin Schmidt]

Hello


Am 09.10.2014 um 20:41 schrieb Urban Loesch:
> Hi,
>
> Am 09.10.2014 12:35, schrieb Martin Schmidt:
>
>> Our MX server is delivering ca. 30 GB new mails per day.
>> Two IMAP proxy server get the connections from the users. Atm. 
>> without dovecot director.
>> We've got around 700k connections per day (imap 200k / pop3 500k)
>
> Are this the hole connections per day? How many concurrend connections 
> do you have at the same time on each server?

we've got 3 Fileserver with ca. 1200 concurrend IMAP connections and ca. 
50 concurrend POP3 connections on each server.


>
>> So we want to make a new system.
>> We desire the new system to use mdbox format ( bigger files, less I/O)
>> and replication through dovecot replication (active/active) instead 
>> of drbd.
>
> I have no experience with dovecot replication (Still on our roadmap). 
> We are currently using drbd on a 10Gbit dedicated link. Works very 
> well for us.
>
>> Each fileserver should know every mailbox/user and for the time being 
>> 2 dovecot proxies for the user connections (IMAP/POP).
>> (later after the migration from the old system to the new, dovecot 
>> director instead of proxies, for caching reasons).
>
> As Florian said, enable zlib. This also decreases I/O, but needs a bit 
> more of CPU. But not that much.

Yes we have enabled it, estimated space saving is up to 40%
>
>>
>> we've got 2 new fileservers, they have each SSD HDDs for "new-storage"
>> and 7200rpm SATA HDDs on RAID 5 with 10 TB for "alt-storage"
>> 32 GB RAM per Server
>
> You also could move the INDEX files from mdbox to different SSDs. We 
> are doing so with 40k accounts and 2TB user data. Index partition has 
> only 22GB used and is increasing not very fast.
On our testsystem we've got it also on a raid 1 SSD, only alt-storage is 
on raid 5. Looks good.
>
>>
>> Do you have some tips for the system?
>> Do you believe 32 GB RAM are enough for one fileserver each and have 
>> you experience with the I/O Waiting problem with huge amounts of Data 
>> on the alt-storage?
>> Could there be issues with the RAM, if one fileserver has a downtime, 
>> so the second one has to take over all mailboxes for a short amount 
>> of time?
>
> I think memory is not the problem. On IMAP/POP3 servers the main 
> problem is I/O. But with dovecot mdbox and index files on SSD's we 
> have no problem at the moment.
On each of our 3 Fileserver we've got 16 GB RAM, 5-7 GB is used and rest 
is cached. You might be right, the i/o is always the bottleneck.
>>
>> In general are only 2 new fileserver enough or should we think in 
>> bigger dimensions, like 4 fileserver
>> Storage expansion in the new servers should not be a problem (bigger 
>> HDDs and a few slots free, so we can expand the raid 5).
> We are using raid 10 hardware raid controller with cache and sata 
> 7200rpm disks. OK, raid 10 needs more disks, but is much faster than 
> raid 5. Raid 5 is not very fast in my eyes.
We've made tests with raid 10 and raid 5, on 4 sata 7200rpm disks, of 
course raid 10 was faster, but overall not very much. And you can expand 
raid 5 easier.
Can you tell me, if you have a high "Waiting" on your alt-storage?

>
>>
>>
>> thank you
>> kind regards
>>
>> Martin Schmidt
>>
>
> Regards
> Urban

Thank you for your impressions.

kr
Martin Schmidt