[Dovecot] Apple Mail and too many open files?
Hello,
For two days in a row, I've noticed that my dovecot server (hosted on
a PowerMac running OS X) has stopped working, with a bunch of error
messages in /var/log/mail.log like:
May 24 07:31:10 top dovecot: pipe() failed: Too many open files
To solve this, I have to kill -9 dovecot and the (many) imap
processes running.
The only client accessing the server is a machine running Apple Mail.
I tried to increase the limit in the number of files (I did a
"launchctl limit maxfiles 1024 unlimited", but the problem still
happened.
Is this a known problem? Should I allow still more files?
Thanks,
Alan
-- Alan Schmitt http://alan.petitepomme.net/
The hacker: someone who figured things out and made something cool
happen.
.O.
..O
OOO
Alan,
On Wed, May 24, 2006 at 07:42:34AM +0200, Alan Schmitt wrote:
Hello,
For two days in a row, I've noticed that my dovecot server (hosted on
a PowerMac running OS X) has stopped working, with a bunch of error
messages in /var/log/mail.log like: May 24 07:31:10 top dovecot: pipe() failed: Too many open files
How many is a bunch?
Dovecot once died on me after an upgrade, because the logfile had filled up the 7Gb partition with millions of these errors:
dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files
In my case, the logs just preceding these were:
dovecot: Feb 27 03:17:56 Fatal: execv(/usr/local/libexec/dovecot/ssl-build-param) failed: No such file or directory dovecot: Feb 27 03:17:56 Error: child 4422 (ssl-build-param) returned error 84 (exec() failed) dovecot: Feb 27 03:27:56 Fatal: execv(/usr/local/libexec/dovecot/ssl-build-param) failed: No such file or directory dovecot: Feb 27 03:27:57 Error: child 14606 (ssl-build-param) returned error 84 (exec() failed) dovecot: Feb 27 03:37:57 Fatal: execv(/usr/local/libexec/dovecot/ssl-build-param) failed: No such file or directory dovecot: Feb 27 03:37:57 Error: child 24932 (ssl-build-param) returned error 84 (exec() failed) dovecot: Feb 27 03:47:58 Fatal: execv(/usr/local/libexec/dovecot/ssl-build-param) failed: No such file or directory dovecot: Feb 27 03:47:58 Error: child 2728 (ssl-build-param) returned error 84 (exec() failed) dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files ...
It turned out that I'd configured dovecot with './configure --program-suffix=1.0b3'. The compile therefore built a ssl-build-param-1.0b3 binary, but as you can see in the logs, still expected a 'ssl-build-params' binary. Symlinking one to the other solved this.
HTH
--Jeff
To solve this, I have to kill -9 dovecot and the (many) imap
processes running.The only client accessing the server is a machine running Apple Mail.
I tried to increase the limit in the number of files (I did a
"launchctl limit maxfiles 1024 unlimited", but the problem still
happened.Is this a known problem? Should I allow still more files?
Thanks,
Alan
-- Alan Schmitt http://alan.petitepomme.net/
The hacker: someone who figured things out and made something cool
happen. .O. ..O OOO
On 24 mai 06, at 10:10, Jeff Turner wrote:
Alan,
On Wed, May 24, 2006 at 07:42:34AM +0200, Alan Schmitt wrote:
Hello,
For two days in a row, I've noticed that my dovecot server (hosted on a PowerMac running OS X) has stopped working, with a bunch of error messages in /var/log/mail.log like: May 24 07:31:10 top dovecot: pipe() failed: Too many open files
How many is a bunch?
Dovecot once died on me after an upgrade, because the logfile had
filled up the 7Gb partition with millions of these errors:dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files
I got about 600 MB of them, but it was clearly running out of files.
I've fetched the latest beta and recompiled, so it might work better.
I did not get the error you had.
Thanks for the suggestion,
Alan
-- Alan Schmitt http://alan.petitepomme.net/
The hacker: someone who figured things out and made something cool
happen.
.O.
..O
OOO
On Wed, 2006-05-24 at 10:43 +0200, Alan Schmitt wrote:
Dovecot once died on me after an upgrade, because the logfile had
filled up the 7Gb partition with millions of these errors:dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files dovecot: Feb 27 03:55:46 Error: pipe() failed: Too many open files
I got about 600 MB of them, but it was clearly running out of files.
I've fetched the latest beta and recompiled, so it might work better.
I did not get the error you had.
The latest beta attempts to handle the "too many files" case and doesn't anymore fill all your hard disk with those errors in logs.
Alan Schmitt wrote:
Hello,
For two days in a row, I've noticed that my dovecot server (hosted on a PowerMac running OS X) has stopped working, with a bunch of error messages in /var/log/mail.log like: May 24 07:31:10 top dovecot: pipe() failed: Too many open files
To solve this, I have to kill -9 dovecot and the (many) imap processes running.
The only client accessing the server is a machine running Apple Mail. I tried to increase the limit in the number of files (I did a "launchctl limit maxfiles 1024 unlimited", but the problem still happened.
Is this a known problem? Should I allow still more files?
You need to look at the parameters: sysctl -a kern.maxfiles = 12288 kern.maxfilesperproc = 10240 and fix them by creating a file /etc/sysctl.conf
OS X is configured by default with these numbers way too low. OS X has some really dumb processes like AFP that will chew through all of your open files and not cope cleanly with running out of allowable/available filehandles.
OS X has a DoS vulnerability in the way ssh processes are spawned and the ssh interaction with their PAM modules (it exhibits with the symptoms you have described). Have you really got port 22 blocked from the outside world?? Have you tested that? Consider running ssh on an alternate port if running OS X server (as Apple's GUI config tools for the firewall don't always allow you to block port 22).
Do consider running your mail services off a machine that is not a Mac OS X server. OS X server is merely OS X client/workstation with a pretty management utility for some 'nix services. It is not stable under high load -- and it is not even stable under moderate load without numerous performance tweaks (it doesn't cope at all well if the disk queue goes up a touch or loadavg is at all interesting - ie it degrades poorly under load).
Rob.
On 24 mai 06, at 11:55, Rob Middleton wrote:
You need to look at the parameters: sysctl -a kern.maxfiles = 12288 kern.maxfilesperproc = 10240 and fix them by creating a file /etc/sysctl.conf
Ah, thanks, this is good to know.
OS X is configured by default with these numbers way too low. OS X
has some really dumb processes like AFP that will chew through all
of your open files and not cope cleanly with running out of
allowable/available filehandles.OS X has a DoS vulnerability in the way ssh processes are spawned
and the ssh interaction with their PAM modules (it exhibits with
the symptoms you have described). Have you really got port 22
blocked from the outside world?? Have you tested that? Consider
running ssh on an alternate port if running OS X server (as Apple's
GUI config tools for the firewall don't always allow you to block
port 22).
I do not have port 22 blocked, and I unfortunately need to use it.
Do consider running your mail services off a machine that is not a
Mac OS X server. OS X server is merely OS X client/workstation with
a pretty management utility for some 'nix services. It is not
stable under high load -- and it is not even stable under moderate
load without numerous performance tweaks (it doesn't cope at all
well if the disk queue goes up a touch or loadavg is at all
interesting - ie it degrades poorly under load).
I'll keep this in mind.
Thanks again,
Alan
-- Alan Schmitt http://alan.petitepomme.net/
The hacker: someone who figured things out and made something cool
happen.
.O.
..O
OOO
participants (4)
-
Alan Schmitt
-
Jeff Turner
-
Rob Middleton
-
Timo Sirainen