On Jun 11 14:51, Timo Sirainen wrote:
On 11.6.2012, at 11.09, Jesper Dahl Nyerup wrote:
In short, as far as we can tell, all the processes in D state appear to be waiting to close the file handle they got from their inotify_init(), and eventually all these close()s go through almost simultaneously.
Yeah. Looks like a kernel bug. You could try if it goes away by disabling inotify in Dovecot. Either recompile with "configure --with-notify=none" or maybe you can disable inotify globally with:
echo 0 > /proc/sys/fs/inotify/max_user_watches echo 0 > /proc/sys/fs/inotify/max_user_instances
I can confirm that this removes the symptoms, and that it doesn't affect the service. Obviously IDLEing users are now only notified upon polling of the file system, but the I/O overhead of doing this seems minimal.
It may be important to note, that even though load on our servers surpass 2000, both Dovecot and the server as a whole is responsive and servicing requests, up until the point where Dovecot reaches its configured maximal number of child processes.
We're still chasing the root cause in the kernel or the VServer patch set. We'll of course make sure to post our findings here, and I'd very much appreciate to hear about other people's progress.
Jesper.