[Dovecot] Design: Asynchronous I/O for single/multi-dbox

[Frank Cusack]

On 3/14/10 1:59 AM +0200 Timo Sirainen wrote:
> The long term plan is to get all of Dovecot disk I/O asynchronous. The
> first step to that direction would be to make dbox/mdbox I/O
> asynchronous. This might also allow mbox/maildir to become partially
> asynchronous.

Does it really need to be?  Sure, for example you might be able to return
to a client that (say) a move didn't complete but is that really useful?
A single blocked I/O is probably indicative of a systemic problem and
the next operation is going to fail as well.  A mail server is part of
a system and in this case, wouldn't it be easier to let the client handle
timeouts?

Just playing devil's advocate since you haven't presented the advantage
of async I/O.  I don't want to guess at the reasoning, but e.g. I hope
you're not planning to return success results before I/O actually
completes.

> So now that all of the APIs have been designed, all that's needed to do
> is to write a simple FS API implementation using kernel's async IO API,
> right? Wrong. There is no usable async IO API in Linux, and probably
> nowhere else either:
...
> So for now the only practical way is to implement it with threads. There

Threads are simpler anyway.

> are several libraries that could make this easier.. But all of them
> enable (and require) full thread safety for libc calls. I don't really
> like that. Dovecot isn't using threads, I shouldn't pay the penalty of
> using extra locking when it's really not necessary.

You're pre-optimizing.  The overhead of an uncontested lock is essentially
zero.

> So I was thinking about doing the async IO in two ways:
> 2) Fallback version that uses pthreads with mutexes.

I would do this version only.

-frank