On Sun, 2007-06-03 at 18:05 +0300, Timo Sirainen wrote:

...

My laptop runs a local IMAP server so that I'm immune from breaking my mail client, and when playing around with powertop I discovered that Dovecot (in particular imap-login) wakes up every 1000ms and then every 50ms even when no clients are connected. A bit of stracing shows this:

I did once try to reduce these unnecessary wakeups, but then I thought it's probably not worth the trouble. No-one's going to run an IMAP server in their laptop.. :)

This fixes it for imap/pop3-login: http://hg.dovecot.org/dovecot/rev/0021765627f3

Fixing dovecot-auth and dovecot processes is a bit more difficult. Patches welcome. :)

Maybe the optimal solution would be to create a new abstraction layer. Most of the timeout handlers are just checking for idle timeouts.

struct idle_timeout *idle_timeout_new(unsigned int secs, timeout_callback_t *callback, void *context); void idle_timeout_free(struct idle_timeout *idle); void idle_timeout_reset(struct idle_timeout *idle);

The code would internally keep just one timeout handler and whenever it's called, calculate the new time when it should be called.

On Sun, 2007-06-03 at 18:27 +0300, Timo Sirainen wrote:

..Or maybe just fix the basic timeout_*() API. Add a new timeout_reset() call == timeout_remove() + timeout_add(original values) and then make the implementation be fast with hundreds of timeouts. The timeouts are currently kept in linked list, so that could be changed to red-black tree I guess (sorted by next execution time). Or is there a better data structure for this?

Continuing my monolog.. :)

There are only a few different idle timout values, so maybe the idle_timeout_*() API is a good idea anyway. It allows the implementation to be really fast and simple:

struct idle_timeout { struct idle_timeout *prev, *next; time_t next_run; void *context; };

So a doubly linked list for each different timeout value. Adding a new timeout or reseting existing one moves the struct to last in the list. When timeout handler runs, it needs to check only the first ones in the list and move them to last.

On Sun, 2007-06-03 at 18:13 +0300, Timo Sirainen wrote:

...

I did once try to reduce these unnecessary wakeups, but then I thought it's probably not worth the trouble. No-one's going to run an IMAP server in their laptop.. :)

:)

My poor laptop is running a IMAP server, a HTTP server and a SMTP server.

...

This fixes it for imap/pop3-login: http://hg.dovecot.org/dovecot/rev/0021765627f3

Sweet, although this still wakes up when there are clients connected right?

...

Fixing dovecot-auth and dovecot processes is a bit more difficult. Patches welcome. :)

Maybe the optimal solution would be to create a new abstraction layer. Most of the timeout handlers are just checking for idle timeouts.

struct idle_timeout *idle_timeout_new(unsigned int secs, timeout_callback_t *callback, void *context); void idle_timeout_free(struct idle_timeout *idle); void idle_timeout_reset(struct idle_timeout *idle);

The code would internally keep just one timeout handler and whenever it's called, calculate the new time when it should be called.

Sounds like what GLib does. Glib also (in 2.13 onwards) lets you create timeouts with second-resolution, so that N timers that go off at roughly the same time do actually go off at the same time, meaning the application wakes up once instead of N times.

I presume porting Dovecot to use the glib main loop abstraction (which is nice and lean, the object system is a separate library) is out of the question?

Ross

Ross Burton mail: ross@burtonini.com jabber: ross@burtonini.com www: http://www.burtonini.com./ PGP Fingerprint: 1A21 F5B0 D8D0 CFE3 81D4 E25A 2D09 E447 D0B4 33DF

On Sun, 2007-06-03 at 18:48 +0300, Timo Sirainen wrote:

I've used GLib before. The biggest problem I see with it is that it doesn't support memory pools. That's why I duplicated most of its useful functionality originally instead of just using it directly. So I think it's much better to fix Dovecot's main loop code to work better instead of adding a dependency to GLib and using only a tiny part of it.

Are they against adding memory pools?

Richard

On Sun, 2007-06-03 at 19:13 +0100, Ross Burton wrote:

By pool do you mean memory regions which can be split into children, and freeing the parent region also frees the children? That isn't in Glib.

That's the idea in dovecot, yes. Actually, no parent region/child region but just basically like a slice that you can free at once.

johannes

On Jun 3, 2007, at 11:29 AM, Ross Burton wrote:

I presume porting Dovecot to use the glib main loop abstraction (which is nice and lean, the object system is a separate library) is out
of the question?

Please don't. One reason Dovecot is so easy to get up and running
quickly is that it has minimal dependencies. For those of us who
aren't running Linux on PCs that can become a headache very quickly,
as the authors of many of the depended-upon libraries often get
"creative" with nonportable GCCisms and other such nonsense. And
even when one *is* running Linux on a PC, sometimes we don't want to
build fifteen packages in order to build one.

That's not to say that simply adding one dependency on glib would
cause a huge problem...but it indicates the adoption of a mindset,
and it's a slippery slope.

      -Dave

-- Dave McGuire Port Charlotte, FL

On Jun 3, 2007, at 3:45 PM, Richard Laager wrote:

...

That's not to say that simply adding one dependency on glib would cause a huge problem...but it indicates the adoption of a mindset, and it's a slippery slope.

The same applies to duplicating code in the interest of avoiding dependencies.

I really don't think that applies. Including a huge library (and
modifying the package to conform to its requirements) just to get one
piece of its functionality isn't an improvement (IMO) over
duplication of one piece of code.

It's especially not an improvement if it results in fewer people
being able to actually use the package.

     -Dave

-- Dave McGuire Port Charlotte, FL

On Sunday June 03, 2007 at 01:48:33 (PM) Dave McGuire wrote:

On Jun 3, 2007, at 11:29 AM, Ross Burton wrote:

...

I presume porting Dovecot to use the glib main loop abstraction (which is nice and lean, the object system is a separate library) is out
of the question?

Please don't. One reason Dovecot is so easy to get up and running
quickly is that it has minimal dependencies. For those of us who
aren't running Linux on PCs that can become a headache very quickly,
as the authors of many of the depended-upon libraries often get
"creative" with nonportable GCCisms and other such nonsense. And
even when one *is* running Linux on a PC, sometimes we don't want to
build fifteen packages in order to build one.

That's not to say that simply adding one dependency on glib would
cause a huge problem...but it indicates the adoption of a mindset,
and it's a slippery slope.

I am using FreeBSD-6.2 presently. Using 'portmanager' as my updating application, I have never experienced the problems or conceived problems as they may be on my system. Perhaps that is just a localized phenomena though.

-- Gerard