[Dovecot] dovecot: pipe() failed: Too many open files
In the past few months we've been having increasing problems with dovecot timing out pop3 logins.
As best I can tell it gets to the point where it won't accept any new pop3 logins.
The error in the mail log is:
dovecot: pipe() failed: Too many open files
I've researched this problem a number of times, usually I see suggestions to increase ulimit.
I've done this a number of times, all the way up to:
ulimit -n 8192
I've put that in the init script right above the daemon start line.
So far it hasn't helped.
In addition, when looking at lsof, it doesn't seem like we're anywhere near the actual limit. for example, last time dovecot only had 1032 files open.
Does anyone have any idea what could be causing the error? Or what else I can try to figure this out?
Thank you,
JW--
System Administrator - Cedar Creek Software http://www.cedarcreeksoftware.com
On Dec 18, 2009, at 4:47 PM, JW wrote:
I've researched this problem a number of times, usually I see suggestions to increase ulimit.
I've done this a number of times, all the way up to:
ulimit -n 8192
I've put that in the init script right above the daemon start line.
What exactly is the "daemon start line"? Many Linux distros override ulimit in their internal scripts. If you're using Linux, you could check if dovecot process really has 8192 limit:
cat /proc/pidof dovecot/limits | grep 'Max open'
On Friday 18 December 2009 16:00:48 Timo Sirainen wrote:
On Dec 18, 2009, at 4:47 PM, JW wrote:
I've researched this problem a number of times, usually I see suggestions to increase ulimit.
I've done this a number of times, all the way up to:
ulimit -n 8192
I've put that in the init script right above the daemon start line.
What exactly is the "daemon start line"
I'm sorry, I forgot to specify that I was using Debian Linux:
case "$1" in
start)
if grep protocols /etc/dovecot/dovecot.conf | sed 's/#.*$//' | tr -d '"' |
egrep -q '[^#]*(\bpop3s?\b|\bimaps?\b)';
then
if [ -x /usr/lib/dovecot/imap-login -a -x /usr/lib/dovecot/imap ]
|| [ -x /usr/lib/dovecot/pop3-login -a -x /usr/lib/dovecot/pop3 ];
then
echo -n "Starting $DESC: $NAME"
ulimit -n 8192
start-stop-daemon --start --quiet --oknodo --exec $DAEMON
echo "."
fi
fi
;;
stop)
echo -n "Stopping $DESC: $NAME "
start-stop-daemon --stop --quiet --oknodo --exec $DAEMON
echo "."
;;
? Many Linux distros override ulimit in their internal scripts. If you're using Linux, you could check if dovecot process really has 8192 limit:
cat /proc/
pidof dovecot/limits | grep 'Max open'
I've seen this suggestion on the web a few times, but there's no such
directory, /proc/pidof dovecot/limits or /proc/*/limits for that matter
pidof dovecotshows the pid for:
/usr/sbin/dovecotI've tried "grep -i max" and grep -i open in the /proc/$pid/* directory for every pid of every dovecot process on the system, none of the files in there contain either word.
mail:/var/log# ps auxw | grep dove
root 23299 0.0 0.0 1948 648 ? Ss 00:41
0:07 /usr/sbin/dovecot
root 23300 0.0 0.2 8752 2700 ? S 00:41 0:05 dovecot-auth
mail:/var/log# ls /proc/23299
attr cmdline cwd exe maps mounts oom_adj root stat status
wchan
auxv cpuset environ fd mem mountstats oom_score smaps statm task
Is there a different place I should put the ulimit?
JW--
System Administrator - Cedar Creek Software http://www.cedarcreeksoftware.com
On Dec 18, 2009, at 5:46 PM, JW wrote:
I'm sorry, I forgot to specify that I was using Debian Linux:
echo -n "Starting $DESC: $NAME" ulimit -n 8192 start-stop-daemon --start --quiet --oknodo --exec $DAEMON
Well, this seemed to work in Debian unstable.. So maybe the problem is that something just causes Dovecot to eat all available file descriptors. When it happens again you should get the lsof output of dovecot and save it.
Also you never mentioned your Dovecot version. It's possible it's already been fixed in recent versions.
? Many Linux distros override ulimit in their internal scripts. If you're using Linux, you could check if dovecot process really has 8192 limit:
cat /proc/
pidof dovecot/limits | grep 'Max open'I've seen this suggestion on the web a few times, but there's no such directory, /proc/
pidof dovecot/limits or /proc/*/limits for that matter
I guess you've old enough kernel that it doesn't support it.
JW wrote:
On Friday 18 December 2009 16:00:48 Timo Sirainen wrote:
On Dec 18, 2009, at 4:47 PM, JW wrote:
I've researched this problem a number of times, usually I see suggestions to increase ulimit.
I've done this a number of times, all the way up to:
ulimit -n 8192
I've put that in the init script right above the daemon start line.
What exactly is the "daemon start line"
I'm sorry, I forgot to specify that I was using Debian Linux:
case "$1" in start) if grep protocols /etc/dovecot/dovecot.conf | sed 's/#.*$//' | tr -d '"' |
egrep -q '[^#]*(\bpop3s?\b|\bimaps?\b)'; then if [ -x /usr/lib/dovecot/imap-login -a -x /usr/lib/dovecot/imap ]
|| [ -x /usr/lib/dovecot/pop3-login -a -x /usr/lib/dovecot/pop3 ]; then echo -n "Starting $DESC: $NAME" ulimit -n 8192 start-stop-daemon --start --quiet --oknodo --exec $DAEMON echo "." fi fi ;; stop) echo -n "Stopping $DESC: $NAME " start-stop-daemon --stop --quiet --oknodo --exec $DAEMON echo "." ;;? Many Linux distros override ulimit in their internal scripts. If you're using Linux, you could check if dovecot process really has 8192 limit:
cat /proc/
pidof dovecot/limits | grep 'Max open'I've seen this suggestion on the web a few times, but there's no such directory, /proc/
pidof dovecot/limits or /proc/*/limits for that matterpidof dovecot
shows the pid for:
/usr/sbin/dovecot
I've tried "grep -i max" and grep -i open in the /proc/$pid/* directory for every pid of every dovecot process on the system, none of the files in there contain either word.
mail:/var/log# ps auxw | grep dove root 23299 0.0 0.0 1948 648 ? Ss 00:41
0:07 /usr/sbin/dovecot root 23300 0.0 0.2 8752 2700 ? S 00:41 0:05 dovecot-authmail:/var/log# ls /proc/23299 attr cmdline cwd exe maps mounts oom_adj root stat status
wchan auxv cpuset environ fd mem mountstats oom_score smaps statm taskIs there a different place I should put the ulimit?
JW
You could try: /etc/security/limits.conf
your_dovecot_user soft nofile 524288 your_dovecot_user hard nofile 524288
Not sure if instead of your_dovecot_user, you should increase the limits for the root user .... ??
Regards, Mario Antonio
On Dec 18, 2009, at 6:12 PM, Mario Antonio wrote:
JW wrote:You could try: /etc/security/limits.conf
your_dovecot_user soft nofile 524288 your_dovecot_user hard nofile 524288
I think these are read by PAM when user logs in. Dovecot never uses them. I'm not sure if even setting root's limits here will affect software started by init.d scripts at system startup.
On 12/18/2009, JW (jw@mailsw.com) wrote:
I'm sorry, I forgot to specify that I was using Debian Linux:
You also forgot the bare neceesities, like dovecot version, dovecot -n output and log entries exhibiting the problem.
I'm actually amazed at how many people post on support lists and say 'Help! Its not working!! What did I do wrong?!' - do they really expect meaningful help?
Even more amazing, on this list Timo can often divine the answer anyway...
On Fre, 2009-12-18 at 18:32 -0500, Charles Marcus wrote:
On 12/18/2009, JW (jw@mailsw.com) wrote:
I'm sorry, I forgot to specify that I was using Debian Linux:
You also forgot the bare neceesities, like dovecot version, dovecot -n output and log entries exhibiting the problem. And for the current case looking into /proc/$(pidof dovecot)/fd should show all open fds (and $(ls -1 /proc/$(pidof dovecot)/fd | wc -l) should be somewhat near the limit).
I'm actually amazed at how many people post on support lists and say 'Help! Its not working!! What did I do wrong?!' - do they really expect meaningful help?
Even more amazing, on this list Timo can often divine the answer anyway... He has probably am extremely good crystal ball;-)
Bernd-- mobil: +43 664 4416156 http://bernd.petrovitsch.priv.at/ Linux Software Entwicklung, Beratung und Dienstleistungen
participants (5)
-
Bernd Petrovitsch
-
Charles Marcus
-
JW
-
Mario Antonio
-
Timo Sirainen