Scalability of Dovecot in the Cloud

[Timo Sirainen]

On 04 Jun 2016, at 21:28, KT Walrus <kevin at my.walr.us> wrote:
> 
> Does anyone have any idea of how many IMAP connections a single cloud VM (4 vCores at 2.4GHz, 30GB RAM, local SSD storage - non-RAID) can be expected to handle in production. The mailboxes are fairly small (average 5MB total - 50MB max, as I don’t store attachments in Dovecot expect those saved through IMAP in the Sent/Drafts folders) and each user will probably have an average of 2 devices that have the mail clients configured to access each mailbox.
> 
> Can such a server handle 100,000 mailboxes (200,000 devices/clients)? Or is it more like 10,000? Or, even smaller?
> 
> I can scale the cloud VM up to 32 vCores and 240GB RAM (at 8 times the price) or split the mailboxes onto multiple VMs. The VM will also be running LMTP and other Dovecot services (I don’t plan on supporting POP3 at this time). The mailboxes will be sync’d to a backup VM running Dovecot for high availability so has some load from this background activity. LMTP will not be that high a load, I think, since most messages will be delivered by at night. But, clients will have IMAP connections 24/7.
> 
> Just trying to get an idea of the cost of running a potentially huge/growing mail service in the cloud… I’m going to have to support around a million mailboxes before the site will generate significant revenue to support operations.

Do you mean most of the IMAP clients will be IDLEing waiting for new mails, which mostly won't arrive until the next night? imap-hibernate feature will be very helpful there then.

Bottlenecks are commonly either the disk IO or the memory usage. With SSD you're probably less likely to run bottleneck in disk IO. Memory usage mainly depends on the number of active (non-hibernated) concurrent connections and also the mailbox sizes of the users.

I'd limit a single Dovecot VM to 64 GB of memory. Maybe more would work, but it might run into bottlenecks on the CPU usage side for services that are limited to a single process per instance.

Replication with dsync is going to increase the load and I'm not sure how big of an issue that is.

Anyway, if it's mostly IDLE connections, I'd expect 100k mailboxes/VM to be fine. Generally I'd expect about 10k active (non-hibernated) IMAP connections/VM for 32 GB of memory, but this depends a lot on the mailbox sizes.