This will depend on many more things also.

4 vcores, of unspecified dedication to your vm is not a good thing.

If you use something like gzip/bz2/lzma for compression of the emails,
will highly affect your cpu usage. Searching and indexing will affect your cpu usage.

Quoting KT Walrus kevin@my.walr.us:

...

Anyway, if it's mostly IDLE connections, I'd expect 100k
mailboxes/VM to be fine. Generally I'd expect about 10k active
(non-hibernated) IMAP connections/VM for 32 GB of memory, but this
depends a lot on the mailbox sizes.

That is great news. 100k mailboxes/VM is a great number. I do expect
most IMAP clients will be IDLEing. Do almost all email clients in
use today do the IDLE command? Do most email clients open many
connections per mailbox? Perhaps IDLEing on multiple
namespaces/folders per mailbox? Would this affect your 100k
mailboxes/VM estimate?

The cloud VM at 4 vCores, 30GB RAM, and local SSD storage is just
$40/month (OVH Public Cloud). I had expected a cost of 10 cents per
mailbox per month (with redundancies raising that cost to 25 cents
per mailbox per month). But 100k mailboxes/VM would give me a total
operating cost of less than 1 cent per month per mailbox, at scale.
Maybe even 10 cents per year per mailbox for Public Cloud hosting
fees?

Does anyone on this list run a large number of mailboxes per server
in production? What is the largest number of Dovecot
mailboxes/client connections you supported on a single server before
you had to upgrade to multiple Dovecot servers?

Kevin

...

On Jun 11, 2016, at 6:37 PM, Timo Sirainen tss@iki.fi wrote:

On 04 Jun 2016, at 21:28, KT Walrus mailto:kevin@my.walr.us> wrote:

...

Does anyone have any idea of how many IMAP connections a single
cloud VM (4 vCores at 2.4GHz, 30GB RAM, local SSD storage -
non-RAID) can be expected to handle in production. The mailboxes
are fairly small (average 5MB total - 50MB max, as I don’t store
attachments in Dovecot expect those saved through IMAP in the
Sent/Drafts folders) and each user will probably have an average
of 2 devices that have the mail clients configured to access each
mailbox.

Can such a server handle 100,000 mailboxes (200,000
devices/clients)? Or is it more like 10,000? Or, even smaller?

I can scale the cloud VM up to 32 vCores and 240GB RAM (at 8 times
the price) or split the mailboxes onto multiple VMs. The VM will
also be running LMTP and other Dovecot services (I don’t plan on
supporting POP3 at this time). The mailboxes will be sync’d to a
backup VM running Dovecot for high availability so has some load
from this background activity. LMTP will not be that high a load,
I think, since most messages will be delivered by at night. But,
clients will have IMAP connections 24/7.

Just trying to get an idea of the cost of running a potentially
huge/growing mail service in the cloud… I’m going to have to
support around a million mailboxes before the site will generate
significant revenue to support operations.

Do you mean most of the IMAP clients will be IDLEing waiting for
new mails, which mostly won't arrive until the next night?
imap-hibernate feature will be very helpful there then.

Bottlenecks are commonly either the disk IO or the memory usage.
With SSD you're probably less likely to run bottleneck in disk IO.
Memory usage mainly depends on the number of active
(non-hibernated) concurrent connections and also the mailbox sizes
of the users.

I'd limit a single Dovecot VM to 64 GB of memory. Maybe more would
work, but it might run into bottlenecks on the CPU usage side for
services that are limited to a single process per instance.

Replication with dsync is going to increase the load and I'm not
sure how big of an issue that is.

Anyway, if it's mostly IDLE connections, I'd expect 100k
mailboxes/VM to be fine. Generally I'd expect about 10k active
(non-hibernated) IMAP connections/VM for 32 GB of memory, but this
depends a lot on the mailbox sizes.