[Dovecot] Too many open files
dovecot: Nov 08 04:04:23 Error: IMAP(user): open() failed with file mailstore/user/Maildir/.Alerts/dovecot.index.log.2: Too many open files dovecot: Nov 08 04:04:23 Error: IMAP(user): open(mailstore/user/Maildir/.Alerts/tmp/1162976663.P4853Q1817.server) failed: Too many open files dovecot: Nov 08 04:04:23 Error: IMAP(user): open() failed with file mailstore/user/Maildir/dovecot.index.tmp: Too many open files
Seeing this with both 1.0beta9 and 1.0rc12
This occured when a user was not using the client, however the client(Thunderbird) filtering mechanism was active as their client is left on 24x7.
Do we really need to increase individual user maxfiles to >1024 ? Dovecot is set to 31768 so it is definately the user process here.
Anything I can tweak in dovecot.conf to prevent this?
Thanks!
On Wed, 2006-11-08 at 10:31 -0500, bofh list wrote:
dovecot: Nov 08 04:04:23 Error: IMAP(user): open() failed with file mailstore/user/Maildir/.Alerts/dovecot.index.log.2: Too many open files dovecot: Nov 08 04:04:23 Error: IMAP(user): open(mailstore/user/Maildir/.Alerts/tmp/1162976663.P4853Q1817.server) failed: Too many open files dovecot: Nov 08 04:04:23 Error: IMAP(user): open() failed with file mailstore/user/Maildir/dovecot.index.tmp: Too many open files
Seeing this with both 1.0beta9 and 1.0rc12
This occured when a user was not using the client, however the client(Thunderbird) filtering mechanism was active as their client is left on 24x7.
Do we really need to increase individual user maxfiles to >1024 ?
No. There's a leak somewhere. Could you check what file descriptors are open for such process once it has been running for some hours? So lsof -p <pid> or look into /proc/pid/fd/ directly. If it's leaking it should show at least tens of opened files. Once I know what files it's not closing it's easier for me to fix this.
Timo Sirainen wrote:
No. There's a leak somewhere. Could you check what file descriptors are open for such process once it has been running for some hours? So lsof -p <pid> or look into /proc/pid/fd/ directly. If it's leaking it should show at least tens of opened files. Once I know what files it's not closing it's easier for me to fix this.
I'm still getting "Too many open files" with 1.0rc14 on NetBSD with kqueue; when this happens, lsof says there are over 1000 pipes open. Is this addressed in 1.0rc15? If not, what can I do to help it be addressed before 1.0?
Thanks,
- Amitai
On 19.11.2006, at 6.57, Amitai Schlair wrote:
Timo Sirainen wrote:
No. There's a leak somewhere. Could you check what file
descriptors are open for such process once it has been running for some hours? So
lsof -p <pid> or look into /proc/pid/fd/ directly. If it's leaking it
should show at least tens of opened files. Once I know what files it's not closing it's easier for me to fix this.I'm still getting "Too many open files" with 1.0rc14 on NetBSD with kqueue; when this happens, lsof says there are over 1000 pipes
open. Is this addressed in 1.0rc15? If not, what can I do to help it be
addressed before 1.0?
I guess it's "dovecot" process that's leaking those fds? Does the
number of used fds grow every time you login+logout?
I tried debugging this with one FreeBSD and one NetBSD, but I
couldn't reproduce it. What NetBSD version are you using?
On Nov 19, 2006, at 5:10 AM, Timo Sirainen wrote:
On 19.11.2006, at 6.57, Amitai Schlair wrote:
Timo Sirainen wrote:
No. There's a leak somewhere. Could you check what file
descriptors are open for such process once it has been running for some hours? So
lsof -p <pid> or look into /proc/pid/fd/ directly. If it's leaking it
should show at least tens of opened files. Once I know what files it's not closing it's easier for me to fix this.I'm still getting "Too many open files" with 1.0rc14 on NetBSD with kqueue; when this happens, lsof says there are over 1000 pipes
open. Is this addressed in 1.0rc15? If not, what can I do to help it be
addressed before 1.0?I guess it's "dovecot" process that's leaking those fds? Does the
number of used fds grow every time you login+logout?
Yes, it's the "dovecot" process. With a very quick-and-dumb test
(quitting and reopening Mail.app) it seems to have more open pipes
each time.
I tried debugging this with one FreeBSD and one NetBSD, but I
couldn't reproduce it. What NetBSD version are you using?
This is on NetBSD 2.0.2. (If I could upgrade, I would, but I can't,
so I won't. :-)
Thanks,
- Amitai
On Thu, 2006-11-30 at 19:19 -0500, Amitai Schlair wrote:
I tried debugging this with one FreeBSD and one NetBSD, but I
couldn't reproduce it. What NetBSD version are you using?This is on NetBSD 2.0.2. (If I could upgrade, I would, but I can't,
so I won't. :-)
OK, since I've heard it only happening with NetBSD 2.0, I think it's a bug in NetBSD itself. So don't use kqueue with NetBSD 2.0..
On Sat, Nov 18, 2006 at 11:57:07PM -0500, Amitai Schlair wrote:
Timo Sirainen wrote:
No. There's a leak somewhere. Could you check what file descriptors are open for such process once it has been running for some hours? So lsof -p <pid> or look into /proc/pid/fd/ directly. If it's leaking it should show at least tens of opened files. Once I know what files it's not closing it's easier for me to fix this.
I'm still getting "Too many open files" with 1.0rc14 on NetBSD with kqueue; when this happens, lsof says there are over 1000 pipes open. Is this addressed in 1.0rc15? If not, what can I do to help it be addressed before 1.0?
I've long stopped using kqueue on my prod servers, and while I've hardly looked at the issue recently, I noticed that it was much harder to reproduce on my build machine, which happens to run a 3.99.x Xen kernel, whereas the prod server runs 2.0. I _think_ I managed to make dovecot leak one fd at some point on the build machine, but that might just be memory playing tricks on me.
-- Quentin Garnier - cube@cubidou.net - cube@NetBSD.org "You could have made it, spitting out benchmarks Owe it to yourself not to fail" Amplifico, Spitting Out Benchmarks, Hometakes Vol. 2, 2005.
participants (5)
-
Amitai Schlair
-
bofh list
-
Geert Hendrickx
-
Quentin Garnier
-
Timo Sirainen