[Dovecot] failed: Too many open files
Dear list,
I have questions about content of the following error messages.
------ dovecot-auth: PAM unable todlopen(/lib/security/pam_nologin.so) dovecot-auth: PAM [error:/lib/security/pam_nologin.so: cannot open shared object file: Too many open files] dovecot-auth: PAM adding faulty module:/lib/security/pam_nologin.so dovecot-auth: PAM _pam_load_conf_file: unable to open /etc/pam.d/system-auth dovecot-auth: PAM unable to dlopen(<*unknown module path*>) dovecot-auth: PAM [error: *unknown module path*>: cannot open shared object file: Too many open files] dovecot-auth: PAM adding faulty module: <*unknown module path*> dovecot-auth: PAM _pam_load_conf_file: unable to open /etc/pam.d/system-auth dovecot-auth: PAM _pam_load_conf_file: unable to open /etc/pam.d/system-auth dovecot-auth: PAM unable to dlopen(/lib/security/pam_deny.so) dovecot-auth: PAM [error:/lib/security/pam_deny.so: cannot open shared object file: Too many open files] dovecot-auth: PAM adding faulty module:/lib/security/pam_deny.so
------
dovecot: auth(default): pam(xxx@xxx.com,127.0.0.1): pipe() failed: Too many open files dovecot: auth(default): pam(xxx@xxx.com,127.0.0.1): pipe() failed: Too many open files
It seems that this is due to that there are files existed to open beyond its limitation.
Other information that could be relevant are as follows.
# ulimit -n
1024
# cat /proc/cat /var/run/dovecot/master.pid/limits
cat: /proc/7273/limits: No such file or directory
# lsof -p cat /var/run/dovecot/master.pid | wc -l
29
# ls /proc/pidof dovecot-auth/fd | wc -l
1024
From the above result, dovecot-auth proccess opened 10243 files, and that seems to be the limit. So I expect that adding following lines into /etc/init.d/dovecot could be fix this issue.
# vi /etc/init.d/dovecot
. /etc/init.d/functions
ulimit -n 4098 <-add test -x /usr/sbin/dovecot || exit 0 <-add
if [ -f /etc/sysconfig/dovecot ]; then . /etc/sysconfig/dovecot fi
Is my understanding right? Even if so, how could I verify what files are opened by dovecot-auth?
I would very much appreciate it if someone in this list could provide comments/suggestions.
Sincerely,
On 26.3.2010, at 8.27, Masaharu Kawada wrote:
From the above result, dovecot-auth proccess opened 10243 files, and that seems to be the limit. So I expect that adding following lines into /etc/init.d/dovecot could be fix this issue. ..
dovecot: auth(default): pam(xxx@xxx.com,127.0.0.1): pipe() failed: Too many open files .. ulimit -n 4098 <-add
How many dovecot-auth processes were there running? Unless there were 1024 dovecot-auth processes, it sounds more like a file descriptor leak and adding more fds will only delay the error.
Anyway, the problem is only with v1.0 and I've already rewritten the PAM code, so it's fixed in newer versions at least. Too much trouble trying to debug v1.0 bugs anymore..
Hi Timo-san,
Thank you very much for your response!
How many dovecot-auth processes were there running? Unless there were 1024 dovecot-auth processes, it sounds more like a file descriptor leak and adding more fds will only delay the error.
According to the result of a command "ps aux", only one dovecot-auth process running on the system. Here is the output of the "ps aux".
root 7273 0.0 0.0 1860 612 ? Ss 2009 9:10
/usr/sbin/dovecot
root 7346 0.0 0.7 141336 132880 ? S 2009 158:18
dovecot-auth
.
.
dovecot 8867 0.dovecot 8867 0.0 0.0 4804 1756 ? S
15:54 0:00 pop3-login
dovecot 8886 0.0 0.0 4808 1456 ? S 15:54 0:00 pop3-login
dovecot 8887 0.0 0.0 4804 1456 ? S 15:54 0:00 pop3-login
0 0.0 4804 1756 ? S 15:54 0:00 pop3-login
dovecot 8886 0.0 0.0 4808 1456 ? S 15:54 0:00 pop3-login
dovecot 8887 0.0 0.0 4804 1456 ? S 15:54 0:00 pop3-login
Did you mean that this phenomenon could be a bug? In case it is impossible to update the dovecot package, adding a line in /etc/init.d/dovecot that I wrote in my first email is the only workaround to avoid the errors?
Sincerely,
Timo Sirainen wrote:
On 26.3.2010, at 8.27, Masaharu Kawada wrote:
dovecot: auth(default): pam(xxx@xxx.com,127.0.0.1): pipe() failed: Too many open files
..
From the above result, dovecot-auth proccess opened 10243
files, and that seems to be the limit. So I expect that adding following lines into /etc/init.d/dovecot could be fix this issue.
..
ulimit -n 4098 <-add
How many dovecot-auth processes were there running? Unless there were 1024 dovecot-auth processes, it sounds more like a file descriptor leak and adding more fds will only delay the error.
Anyway, the problem is only with v1.0 and I've already rewritten the PAM code, so it's fixed in newer versions at least. Too much trouble trying to debug v1.0 bugs anymore..
On 26.3.2010, at 10.47, Masaharu Kawada wrote:
How many dovecot-auth processes were there running? Unless there were 1024 dovecot-auth processes, it sounds more like a file descriptor leak and adding more fds will only delay the error.
According to the result of a command "ps aux", only one dovecot-auth process running on the system. .. Did you mean that this phenomenon could be a bug?
Yes. There should be no reason for dovecot-auth to have 1024 fds open.
In case it is impossible to update the dovecot package, adding a line in /etc/init.d/dovecot that I wrote in my first email is the only workaround to avoid the errors?
My guess it that it takes 4 times as long for the error to show up, but it doesn't prevent it.
On 3/26/2010 4:53 AM, Timo Sirainen wrote:
On 26.3.2010, at 10.47, Masaharu Kawada wrote:
How many dovecot-auth processes were there running? Unless there were 1024 dovecot-auth processes, it sounds more like a file descriptor leak and adding more fds will only delay the error.
According to the result of a command "ps aux", only one dovecot-auth process running on the system.
..
Did you mean that this phenomenon could be a bug?
Yes. There should be no reason for dovecot-auth to have 1024 fds open.
In case it is impossible to update the dovecot package, adding a line in /etc/init.d/dovecot that I wrote in my first email is the only workaround to avoid the errors?
My guess it that it takes 4 times as long for the error to show up, but it doesn't prevent it.
Run "lsof" or something similar to see what open filedescriptors there are. I ran into this same issue...and it had nothing to do with Dovecot...Dovecot was just unlucky enough to be requesting to open a file when the limit was already reached.
On Fri, 2010-03-26 at 11:15 -0400, Tony Rutherford wrote:
Run "lsof" or something similar to see what open filedescriptors there are. I ran into this same issue...and it had nothing to do with Dovecot...Dovecot was just unlucky enough to be requesting to open a file when the limit was already reached.
The error message for that situation is actually different:
Too many open files in system
vs.
Too many open files
Hi Timo-san, Tony-san,
Thank you very much for your comments. In the output of a 'lsof', I see a lot of the following messages.
dovecot-a 7346 root 10u sock
0,5 181268 can't identify protocol dovecot-a 7346 root 11u sock
0,5 277537 can't identify protocol
I found a quite similar issue posted at: http://www.mail-archive.com/dovecot@dovecot.org/msg07229.html
I would like to make sure if there is no way to avoid this issue without updating the dovecot package.
Sincerely,
Timo Sirainen wrote:
On Fri, 2010-03-26 at 11:15 -0400, Tony Rutherford wrote:
Run "lsof" or something similar to see what open filedescriptors there are. I ran into this same issue...and it had nothing to do with Dovecot...Dovecot was just unlucky enough to be requesting to open a file when the limit was already reached.
The error message for that situation is actually different:
Too many open files in system
vs.
Too many open files
participants (3)
-
Masaharu Kawada
-
Timo Sirainen
-
Tony Rutherford