Scalability of Dovecot in the Cloud

[Patrick Domack]

This will depend on many more things also.

4 vcores, of unspecified dedication to your vm is not a good thing.

If you use something like gzip/bz2/lzma for compression of the emails,  
will highly affect your cpu usage.
Searching and indexing will affect your cpu usage.



Quoting KT Walrus <kevin at my.walr.us>:

>> Anyway, if it's mostly IDLE connections, I'd expect 100k  
>> mailboxes/VM to be fine. Generally I'd expect about 10k active  
>> (non-hibernated) IMAP connections/VM for 32 GB of memory, but this  
>> depends a lot on the mailbox sizes.
>
> That is great news. 100k mailboxes/VM is a great number. I do expect  
> most IMAP clients will be IDLEing. Do almost all email clients in  
> use today do the IDLE command? Do most email clients open many  
> connections per mailbox? Perhaps IDLEing on multiple  
> namespaces/folders per mailbox? Would this affect your 100k  
> mailboxes/VM estimate?
>
> The cloud VM at 4 vCores, 30GB RAM, and local SSD storage is just  
> $40/month (OVH Public Cloud). I had expected a cost of 10 cents per  
> mailbox per month (with redundancies raising that cost to 25 cents  
> per mailbox per month). But 100k mailboxes/VM would give me a total  
> operating cost of less than 1 cent per month per mailbox, at scale.  
> Maybe even 10 cents per year per mailbox for Public Cloud hosting  
> fees?
>
> Does anyone on this list run a large number of mailboxes per server  
> in production? What is the largest number of Dovecot  
> mailboxes/client connections you supported on a single server before  
> you had to upgrade to multiple Dovecot servers?
>
> Kevin
>
>> On Jun 11, 2016, at 6:37 PM, Timo Sirainen <tss at iki.fi> wrote:
>>
>> On 04 Jun 2016, at 21:28, KT Walrus <kevin at my.walr.us  
>> <mailto:kevin at my.walr.us>> wrote:
>>>
>>> Does anyone have any idea of how many IMAP connections a single  
>>> cloud VM (4 vCores at 2.4GHz, 30GB RAM, local SSD storage -  
>>> non-RAID) can be expected to handle in production. The mailboxes  
>>> are fairly small (average 5MB total - 50MB max, as I don’t store  
>>> attachments in Dovecot expect those saved through IMAP in the  
>>> Sent/Drafts folders) and each user will probably have an average  
>>> of 2 devices that have the mail clients configured to access each  
>>> mailbox.
>>>
>>> Can such a server handle 100,000 mailboxes (200,000  
>>> devices/clients)? Or is it more like 10,000? Or, even smaller?
>>>
>>> I can scale the cloud VM up to 32 vCores and 240GB RAM (at 8 times  
>>> the price) or split the mailboxes onto multiple VMs. The VM will  
>>> also be running LMTP and other Dovecot services (I don’t plan on  
>>> supporting POP3 at this time). The mailboxes will be sync’d to a  
>>> backup VM running Dovecot for high availability so has some load  
>>> from this background activity. LMTP will not be that high a load,  
>>> I think, since most messages will be delivered by at night. But,  
>>> clients will have IMAP connections 24/7.
>>>
>>> Just trying to get an idea of the cost of running a potentially  
>>> huge/growing mail service in the cloud… I’m going to have to  
>>> support around a million mailboxes before the site will generate  
>>> significant revenue to support operations.
>>
>> Do you mean most of the IMAP clients will be IDLEing waiting for  
>> new mails, which mostly won't arrive until the next night?  
>> imap-hibernate feature will be very helpful there then.
>>
>> Bottlenecks are commonly either the disk IO or the memory usage.  
>> With SSD you're probably less likely to run bottleneck in disk IO.  
>> Memory usage mainly depends on the number of active  
>> (non-hibernated) concurrent connections and also the mailbox sizes  
>> of the users.
>>
>> I'd limit a single Dovecot VM to 64 GB of memory. Maybe more would  
>> work, but it might run into bottlenecks on the CPU usage side for  
>> services that are limited to a single process per instance.
>>
>> Replication with dsync is going to increase the load and I'm not  
>> sure how big of an issue that is.
>>
>> Anyway, if it's mostly IDLE connections, I'd expect 100k  
>> mailboxes/VM to be fine. Generally I'd expect about 10k active  
>> (non-hibernated) IMAP connections/VM for 32 GB of memory, but this  
>> depends a lot on the mailbox sizes.