[Dovecot] rc7 crashing under heavy load
Help!
Our students are back and they are pounding my dovecot server to death (ie "Login process died too early - shutting down" or "too many open files" type crash). I've had my third crash today.
My setup: Solaris 10, imap and imaps, 3K users coming in via horde webmail. My current connection settings are:
login_max_processes_count = 2048 login_max_connections = 4096
Q: Does raising these limits actually make my problem worse or better? Are there any other variables that I should change to prevent the "too many open files"? Suggestions?
Jeff Earickson Colby College
Too many open files - try increasing the number of file handles for the Dovecot master process; I use "plimit -n 4096 <Dovecot master pid>"
Login process died too early - are you using NIS? It's too slow for Dovecot, I think. We create a Dovecot "passwd-file" from NIS overnight and then "HUP" the Dovecot master process to make it re-read it (we use it only for UIDs, as we use pam-ldap to Active Directory for authentication, but it could have the passwords in as well). Dovecot caches the passwd-file so it's very quick.
Hope this helps, Chris
Jeff A. Earickson wrote:
Help!
Our students are back and they are pounding my dovecot server to death (ie "Login process died too early - shutting down" or "too many open files" type crash). I've had my third crash today.
My setup: Solaris 10, imap and imaps, 3K users coming in via horde webmail. My current connection settings are:
login_max_processes_count = 2048 login_max_connections = 4096
Q: Does raising these limits actually make my problem worse or better? Are there any other variables that I should change to prevent the "too many open files"? Suggestions?
Jeff Earickson Colby College
-- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
----- Original Message ----- From: "Chris Wakelin" c.d.wakelin@reading.ac.uk
Too many open files - try increasing the number of file handles for the Dovecot master process; I use "plimit -n 4096 <Dovecot master pid>"
Login process died too early - are you using NIS? It's too slow for Dovecot, I think. We create a Dovecot "passwd-file" from NIS overnight and then "HUP" the Dovecot master process to make it re-read it (we use it only for UIDs, as we use pam-ldap to Active Directory for authentication, but it could have the passwords in as well). Dovecot caches the passwd-file so it's very quick.
Chris, I am new to Dovecot and this list, and struggled this weekend to get Dovegot to authenticate the username against a "passwd-file" and using pam to authenticate the passwords. I was wondering if you could tell me what the format of the "passwd-file" should look like. I want to the username to be user@domain, and tried stripping the domain part for username/password authentication against pam, but that hasn't worked for me, so I thought I would use a "passwd-file" like your doing. Also, what does your dovecot.conf section look like for using a "passwd-file" and pam?
Thanks for any help you can provide!
Bill
See http://wiki.dovecot.org/AuthDatabase/PasswdFile
We've just got:
<username>:x:<uid>:<gid>::<homedir>
and the standard:
passdb pam { }
userdb passwd-file { args = /somewhere/etc/userdb }
It might be worth looking at http://wiki.dovecot.org/VirtualUsers as well.
Best Wishes, Chris
billl@inetmsg.com wrote:
Chris, I am new to Dovecot and this list, and struggled this weekend to get Dovegot to authenticate the username against a "passwd-file" and using pam to authenticate the passwords. I was wondering if you could tell me what the format of the "passwd-file" should look like. I want to the username to be user@domain, and tried stripping the domain part for username/password authentication against pam, but that hasn't worked for me, so I thought I would use a "passwd-file" like your doing. Also, what does your dovecot.conf section look like for using a "passwd-file" and pam?
Thanks for any help you can provide!
Bill
-- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
----- Original Message ----- From: "Chris Wakelin" c.d.wakelin@reading.ac.uk
See http://wiki.dovecot.org/AuthDatabase/PasswdFile
We've just got:
<username>:x:<uid>:<gid>::<homedir>
and the standard:
passdb pam { }
userdb passwd-file { args = /somewhere/etc/userdb }
It might be worth looking at http://wiki.dovecot.org/VirtualUsers as well.
Thanks Chris, found what I was looking for in the auth.txt file included in the tar file. I'm using Dovecot with Postfix on Fedora Core 3, with TLS/SSL and SASL implemented on SMTPD and IMAPS. Works very nicely indeed. My complements to the development team!
Best regards,
Bill
Hey! Don't hijack somebody else's thread, think up a new subject line.
On Tue, 5 Sep 2006, billl@inetmsg.com wrote:
Date: Tue, 5 Sep 2006 18:13:33 -0700 From: billl@inetmsg.com To: dovecot@dovecot.org Subject: Re: [Dovecot] rc7 crashing under heavy load
----- Original Message ----- From: "Chris Wakelin" c.d.wakelin@reading.ac.uk
Too many open files - try increasing the number of file handles for the Dovecot master process; I use "plimit -n 4096 <Dovecot master pid>"
Login process died too early - are you using NIS? It's too slow for Dovecot, I think. We create a Dovecot "passwd-file" from NIS overnight and then "HUP" the Dovecot master process to make it re-read it (we use it only for UIDs, as we use pam-ldap to Active Directory for authentication, but it could have the passwords in as well). Dovecot caches the passwd-file so it's very quick.
Chris, I am new to Dovecot and this list, and struggled this weekend to get Dovegot to authenticate the username against a "passwd-file" and using pam to authenticate the passwords. I was wondering if you could tell me what the format of the "passwd-file" should look like. I want to the username to be user@domain, and tried stripping the domain part for username/password authentication against pam, but that hasn't worked for me, so I thought I would use a "passwd-file" like your doing. Also, what does your dovecot.conf section look like for using a "passwd-file" and pam?
Thanks for any help you can provide!
Bill
----- Original Message ----- From: "Jeff A. Earickson" jaearick@colby.edu
Hey! Don't hijack somebody else's thread, think up a new subject line.
Hey! Don't top-post!!! My question was directed to a comment Chris made in this thread. Consider getting a life...
Bill
Gang,
I may have stumbled on a solution to the "too many open files" issue, but I am wondering about the security consequences. I changed login_process_per_connection from "yes" to "no". This makes a HUGE reduction in the number of imap-login processes, from ~200 down to the login_processes_count (currently the default of 3). It also made my "too many open files" syslog complaints vanish. Yippee!
But is there any serious security risk of login_process_per_connection=no?
Jeff Earickson Colby College
On Wed, 6 Sep 2006, Chris Wakelin wrote:
Date: Wed, 06 Sep 2006 01:12:34 +0100 From: Chris Wakelin c.d.wakelin@reading.ac.uk To: Jeff A. Earickson jaearick@colby.edu Cc: dovecot@dovecot.org Subject: Re: [Dovecot] rc7 crashing under heavy load
Too many open files - try increasing the number of file handles for the Dovecot master process; I use "plimit -n 4096 <Dovecot master pid>"
A plimit on my dovecot master process shows:
plimit 25773 25773: /opt/dovecot/sbin/dovecot resource current maximum time(seconds) unlimited unlimited file(blocks) unlimited unlimited data(kbytes) unlimited unlimited stack(kbytes) 8192 unlimited coredump(blocks) unlimited unlimited nofiles(descriptors) 65536 131072 vmemory(kbytes) unlimited unlimited
The number of file descriptors comes from my kernel tweaks in /etc/system.
Login process died too early - are you using NIS? It's too slow for Dovecot, I think. We create a Dovecot "passwd-file" from NIS overnight and then "HUP" the Dovecot master process to make it re-read it (we use it only for UIDs, as we use pam-ldap to Active Directory for authentication, but it could have the passwords in as well). Dovecot caches the passwd-file so it's very quick.
No, but I was using automounter and LOFS (loopback file systems) in a big way on my imap server to access user homedirs. It was elegant from a sysadmin point of view, but a latency issue for dovecot with all of the waiting for homedirs to automount. I scrapped that last night and it helped.
On Wed, 2006-09-06 at 11:39 -0400, Jeff A. Earickson wrote:
Gang,
I may have stumbled on a solution to the "too many open files" issue, but I am wondering about the security consequences. I changed login_process_per_connection from "yes" to "no". This makes a HUGE reduction in the number of imap-login processes, from ~200 down to the login_processes_count (currently the default of 3). It also made my "too many open files" syslog complaints vanish. Yippee!
But is there any serious security risk of login_process_per_connection=no?
Theoretically it's more risky, but as long as there are no security holes found from Dovecot (or OpenSSL if you're using that) it doesn't really make a difference.
As for "too many open files" you probably could have increased the max. file count for dovecot process. Don't know how it's done in Solaris (ulimit -n 10000 before running dovecot?)
participants (6)
-
Bill Landry
-
billl@inetmsg.com
-
Chris Wakelin
-
Jeff A. Earickson
-
Jos Chrispijn
-
Timo Sirainen