Scaling to 10 Million IMAP sessions on a single server

[Timo Sirainen]

On 23 Feb 2017, at 23.00, Timo Sirainen <tss at iki.fi> wrote:
> 
> I mainly see such external databases as additional reasons for things to break. And even if not, additional extra layers of latency.

Oh, just thought that I should clarify this and I guess other things I said. I think there are two separate things we're possibly talking about in here:

1) Temporary state: This is what I was mainly talking about. State related to a specific IMAP session. This doesn't take much space and can be stored in the proxy's memory since it's specific to the TCP session anyway.

2) Permanent state: This is mainly about the storage. A lot of people use Dovecot with NFS. So one possibility for storing the permanent state is NFS. Another possibility with Dovecot Pro is to store it to object storage as blobs and keep a local cache of the state. A 3rd possibility might be to use some kind of a database for storing the permanent state. I'm fine with the first two, but with 3rd I see a lot of problems and not a whole lot of benefit. But if you think of the databases (or even NFS) as blob storage, you can think of them the same as any object storage and use the same obox format with them. What I'm mainly against is attempting to create some kind of a database that has structured format like (imap_uid, flags, ...) - I'm sure that can be useful for various purposes but performance or scalability isn't one of them.