On Tue, 2010-03-16 at 13:19 -0400, Frank Cusack wrote:
On 3/16/10 6:44 PM +0200 Timo Sirainen wrote:
Threads are useful when you need to do CPU intensive work in parallel, and threads need access to shared data structures.
And when you don't want to block on I/O. Threads are almost always easier than AIO, and especially easy (ie, no scary complexity issues eg deadlock) if you aren't sharing data structures.
But there are some shared data structures, so I'd need to add locking for them. And threads won't help with the biggest benefit of the whole AIO change: "fast" access to high-latency storage. To access such storage without constant waiting on IO, multiple read requests must be done in parallel during the execution of a single command. So even if I added support for threads, I'd need to process a single command in parallel, which eventually leads to more or less the same kind of AIO processing I'm thinking about now anyway, only with tons of extra locking.
Also Dovecot's file locking rules are simple and short-lived. In v2.0 it's already possible to have a single process serve multiple connections. And Apple's Dovecot does that already with v1.1. I haven't seen/heard of any deadlocking problems with either. And actually my long term plan is to make single-dbox (including index files) completely lockless.
You mentioned locking overhead earlier, which is a red herring, but even on Linux if you are running an SMP kernel you have the locking "overhead" even if your process is single threaded.
Yes, but once threads are enabled there are even more locking overhead from libc. (Yeah, probably small overhead.)