[Dovecot] Design: Asynchronous I/O for single/multi-dbox

[Timo Sirainen]

On 14.3.2010, at 5.27, Frank Cusack wrote:

> On 3/14/10 4:48 AM +0200 Timo Sirainen wrote:
>> On 14.3.2010, at 4.40, Frank Cusack wrote:
>> 
>>> Just playing devil's advocate since you haven't presented the advantage
>>> of async I/O.  I don't want to guess at the reasoning, but e.g. I hope
>>> you're not planning to return success results before I/O actually
>>> completes.
>> 
>> The idea was that a single process could handle tons of connections.
> 
> And what's the advantage of that?  process-per-client already scales
> really well, at the expense of more memory.  In both Linux and Solaris,
> the cost of a process context switch is nearly the same as a thread
> context switch.  I don't know about *BSD but I imagine it's similar.

Well, you just mentioned the benefits :) Less memory usage, less context switches (of any kind). (You aren't assuming I'd do something like one thread per connection, right? That's not going to happen.)

>> Maybe in the end the number of IMAP processes you'd have would be about
>> 1-2 x the number of CPU cores.
> 
> I'd think many more than that (regardless of client model - multithread,
> AIO or PPC), since most time is spent with the user idle and the IMAP
> connection doing nothing.  I'd guess (a) you should be able to support at
> least 100 simultaneous clients per CPU, if not 500, and (b) that simul
> client support is I/O limited, not CPU limited.

That's kind of the point. You could have just a few IMAP processes where each can handle hundreds of connections. Currently that's not a very good idea, because a single connection that waits on disk I/O blocks all the other connections in the same process from doing anything.

>> And not just that. Also parallelism. Dovecot could issue a lot of I/O
>> requests in parallel and OS can reorder those so that it gives the best
>> performance by reading them from disk in the right order. And the higher
>> the latency to disks is, the higher benefits there comes from parallelism
>> (NFS, NoSQL).
> 
> The kernel already does that regardless of server implementation.

Kernel can only parallelize requests from different processes. But AIO allows for example telling kernel that:

 - open files 1..10
 - read contents of files 1..10

rather than:

 - open file 1
 - read contents of file 1
 - open file 2
 - read contents of file 2
 - ..etc..

All of this when processing a single client connection's command(s). And while waiting for I/O replies Dovecot can do other work. So user sees results earlier.