[Dovecot] dovecot-auth file descriptor usage
In dovecot-auth (rc15) we experienced problems when the file descriptors reached 256 (due to a Solaris fdopen() call described previously). The issue was that there was one file descriptor being used for every imap-login and pop3-login process (in our environment of SSL-only that means one for every active IMAP and POP connection).
Mods were made (and appear in rc16) to close out the descriptor in question after using it during authentication (as I understand it). However, I am still seeing plenty of file descriptors left open in rc16 (one per session). For example, on a Solaris 9 server with 9 IMAP connections open here is an lsof excerpt of dovecot-auth:
dovecot-a 26071 root 7u unix 105,136 0t79 3190 /devices/pseudo/tl@0:ticots->/var/opt/dovecot/run/dovecot/login/default (0x3000d084250) dovecot-a 26071 root 8u unix 105,140 0t80 3190 /devices/pseudo/tl@0:ticots->/var/opt/dovecot/run/dovecot/login/default (0x3c9c4227b68) dovecot-a 26071 root 9u unix 105,149 0t79 3190 /devices/pseudo/tl@0:ticots->/var/opt/dovecot/run/dovecot/login/default (0x3cbaa7680b8) dovecot-a 26071 root 10u unix 105,266 0t79 3190 /devices/pseudo/tl@0:ticots->/var/opt/dovecot/run/dovecot/login/default (0x3cbaa765d48) dovecot-a 26071 root 11u unix 105,160 0t79 3190 /devices/pseudo/tl@0:ticots->/var/opt/dovecot/run/dovecot/login/default (0x3cbaa7667e0) dovecot-a 26071 root 12u unix 105,255 0t79 3190 /devices/pseudo/tl@0:ticots->/var/opt/dovecot/run/dovecot/login/default (0x3cbaa769288) dovecot-a 26071 root 13r DOOR 265,0 0t0 50 /var/run (swap) (door to nscd[7752]) dovecot-a 26071 root 14u unix 105,258 0t79 3190 /devices/pseudo/tl@0:ticots->/var/opt/dovecot/run/dovecot/login/default (0x30002dc4008) dovecot-a 26071 root 15u unix 105,282 0t80 3190 /devices/pseudo/tl@0:ticots->/var/opt/dovecot/run/dovecot/login/default (0x30003185738) dovecot-a 26071 root 16u unix 105,141 0t80 3190 /devices/pseudo/tl@0:ticots->/var/opt/dovecot/run/dovecot/login/default (0x30003185ac8)
With the modification made, shouldn't these no longer accumulate?
--
Steven F. Siirila Office: Lind Hall, Room 130B Internet Services E-mail: sfs@umn.edu Office of Information Technology Voice: (612) 626-0244 University of Minnesota Fax: (612) 626-7593
On Fri, 2007-01-05 at 13:55 -0600, Steven F Siirila wrote:
Mods were made (and appear in rc16) to close out the descriptor in question after using it during authentication (as I understand it). However, I am still seeing plenty of file descriptors left open in rc16 (one per session). For example, on a Solaris 9 server with 9 IMAP connections open here is an lsof excerpt of dovecot-auth:
How about if you check those imap-login processes that are doing only SSL proxying, do they also have the socket open?
I just tried myself, and dovecot-auth keeps only as many auth sockets open as there are imap-login processes accepting new connections..
On Wed, Jan 17, 2007 at 12:07:43AM +0200, Timo Sirainen wrote:
On Fri, 2007-01-05 at 13:55 -0600, Steven F Siirila wrote:
Mods were made (and appear in rc16) to close out the descriptor in question after using it during authentication (as I understand it). However, I am still seeing plenty of file descriptors left open in rc16 (one per session). For example, on a Solaris 9 server with 9 IMAP connections open here is an lsof excerpt of dovecot-auth:
How about if you check those imap-login processes that are doing only SSL proxying, do they also have the socket open?
In our environment, we do not allow non-SSL connections. Given that, should I continue to expect to see this behavior? If so, I will probably pursue compiling Dovecot in 64-bit mode on Solaris 10 to get around this dang limit.
I just wanted to know 1) if this latest mod did what was intended, and 2) do 'n' imap-login sessions (SSL) always imply 'n' file descriptors used within dovecot-auth?
I just tried myself, and dovecot-auth keeps only as many auth sockets open as there are imap-login processes accepting new connections..
I assume you mean non-SSL. ?
--
Steven F. Siirila Office: Lind Hall, Room 130B Internet Services E-mail: sfs@umn.edu Office of Information Technology Voice: (612) 626-0244 University of Minnesota Fax: (612) 626-7593
On Tue, 2007-01-16 at 16:14 -0600, Steven F Siirila wrote:
On Wed, Jan 17, 2007 at 12:07:43AM +0200, Timo Sirainen wrote:
On Fri, 2007-01-05 at 13:55 -0600, Steven F Siirila wrote:
Mods were made (and appear in rc16) to close out the descriptor in question after using it during authentication (as I understand it). However, I am still seeing plenty of file descriptors left open in rc16 (one per session). For example, on a Solaris 9 server with 9 IMAP connections open here is an lsof excerpt of dovecot-auth:
How about if you check those imap-login processes that are doing only SSL proxying, do they also have the socket open?
In our environment, we do not allow non-SSL connections. Given that, should I continue to expect to see this behavior? If so, I will probably pursue compiling Dovecot in 64-bit mode on Solaris 10 to get around this dang limit.
I just wanted to know 1) if this latest mod did what was intended, and 2) do 'n' imap-login sessions (SSL) always imply 'n' file descriptors used within dovecot-auth?
I just tried myself, and dovecot-auth keeps only as many auth sockets open as there are imap-login processes accepting new connections..
I assume you mean non-SSL. ?
No, non-SSL exits the imap-login processes after login, so that has never reserved fds. So the fix was for SSL connections, and I tested SSL connections..
I'm now just wondering if it's only dovecot-auth that doesn't notice the socket closing for some reason, or if the imap-login processes proxying SSL didn't really close the socket after all.
Le 16.01.2007 23:26, Timo Sirainen a écrit :
No, non-SSL exits the imap-login processes after login, so that has never reserved fds. So the fix was for SSL connections, and I tested SSL connections..
I'm now just wondering if it's only dovecot-auth that doesn't notice the socket closing for some reason, or if the imap-login processes proxying SSL didn't really close the socket after all.
I am also experiencing an increased file descriptors usage on a server with several hundreds of users (running debian sarge amd64) since I'm using rc15. rc17 did not fix the problem. The server was formerly running fine with beta8. The problem is that when the file descriptors limit is reached, dovecot stops responding to new connections or responds strangely (thunderbird says that the server is not an imapV4 server) but nothing is logged anywhere. The only fix is a restart.
-- Nico Les plus grands ne sont pas ceux qui ne tombent jamais, mais ceux qui se relèvent toujours.
On Tue, Jan 16, 2007 at 11:37:00PM +0100, Nicolas STRANSKY wrote:
Le 16.01.2007 23:26, Timo Sirainen a �crit :
No, non-SSL exits the imap-login processes after login, so that has never reserved fds. So the fix was for SSL connections, and I tested SSL connections..
I'm now just wondering if it's only dovecot-auth that doesn't notice the socket closing for some reason, or if the imap-login processes proxying SSL didn't really close the socket after all.
I am also experiencing an increased file descriptors usage on a server with several hundreds of users (running debian sarge amd64) since I'm using rc15. rc17 did not fix the problem. The server was formerly running fine with beta8. The problem is that when the file descriptors limit is reached, dovecot stops responding to new connections or responds strangely (thunderbird says that the server is not an imapV4 server) but nothing is logged anywhere. The only fix is a restart.
This may be a different, more basic issue. Try using "ulimit -n unlimited" or whatever equivalent command you have on Debian to remove the file descriptor limitation. From my experience, both dovecot master and the dovecot-auth process require well over 256 file descriptors in normal operation. The master process needs one for every child process in order to do logging for that child process. The auth process seems to have one open for every imap-login "listening" or "SSL Proxying" process you have. (Timo, correct me if I'm wrong on this.)
-- Nico Les plus grands ne sont pas ceux qui ne tombent jamais, mais ceux qui se rel�vent toujours.
--
Steven F. Siirila Office: Lind Hall, Room 130B Internet Services E-mail: sfs@umn.edu Office of Information Technology Voice: (612) 626-0244 University of Minnesota Fax: (612) 626-7593
participants (3)
-
Nicolas STRANSKY
-
Steven F Siirila
-
Timo Sirainen