[Dovecot] Postfix+Dovecot+LDAP
This message does not contain a bug report or any issues. It is a summary of my experiences with Postfix, Dovecot, and Ldap.
Do not use Dovecot 0.99, even if it's what your vendor has packaged. 1a. ... When you upgrade past 0.99, you'll get some double mail because of UIDL's.
Do not use PAM and Ldap and then let dovecot talk to pam. It is bad. Don't do it. 2a. Unless you like users digging around in random mailboxes. 2b. NSCD is broken and should not be trusted.
Dovecot+Ldap works, and works well. 3a. If you don't do "Auth Binds", it cuts the work on the ldap server in half. 3b. User Prefetch cuts that in half again.
Postfix+Ldap works, and works well. 4a. If your server is underspec'd, you may get the occassional "Temporary lookup failure" 400 error during periods of heavy load, specifically when backups are running.
OpenLdap works.
5a. When you run as ldap, the config files need to be owned by ldap. 5b. ... And the database 5c. ... And the schema 5d. Adding slapd_db_recover to the init script isn't a bad idea.Don't be dumb. 6a. ... Like me. 6b. If you are using ldap to lookup the attribute mail, index the mail attribute. 6c. If you add an index to Openldap, you have to reindex the database with slapindex 6d. ... Or you bounce a lot of mail before you figure it out. 6e. Make sure you can login to the console and/or ssh if the ldap server dies. 6f. ... Be Really sure. 6g. The convert plugin can switch users between maildirs and mboxes.
Ejay Hire IT Manager, Confidential Business Resources 615-665-5555 (office) 615-456-9813 (mobile)
Quoting Ejay Hire ehire@globaloptions.com:
- Do not use PAM and Ldap and then let dovecot talk to pam. It is bad. Don't do it. 2a. Unless you like users digging around in random mailboxes. 2b. NSCD is broken and should not be trusted.
Erm... since I'm running precisely this kind of setup, could you
please add some detail? I'd be very interested.
Feel free to reply directly to me, but I think it should be useful
also for ML archives ;)
TIA, Fabrizio
hermooz@hermooz.net wrote:
Quoting Ejay Hire ehire@globaloptions.com:
- Do not use PAM and Ldap and then let dovecot talk to pam. It is bad. Don't do it. 2a. Unless you like users digging around in random mailboxes. 2b. NSCD is broken and should not be trusted.
Erm... since I'm running precisely this kind of setup, could you please add some detail? I'd be very interested. Feel free to reply directly to me, but I think it should be useful also for ML archives ;)
Hello,
Me too.
Btw the combination is working here perfectly with over 2 500 users.
-arto
- Do not use PAM and Ldap and then let dovecot talk to pam. It is bad. Don't do it. 2a. Unless you like users digging around in random mailboxes. 2b. NSCD is broken and should not be trusted.
Erm... since I'm running precisely this kind of setup, could you please add some detail? I'd be very interested. Feel free to reply directly to me, but I think it should be useful also for ML archives ;)
For Me, When I was using Dovecot Pam->nss_ldap on 0.99, and 1.0rc1 and rc2, it caused heavy system load. When I turned on nscd, the system load went down (Yeah!) but 8 of my 170ish users managed to POP mail out of the wrong mboxes. (I checked all the obvious things)
When I reconfigured Dovecot to talk directly to Ldap (Without Pam), the universe returned to the normal level of entropy.
I posted to the mailing list http://dovecot.org/list/dovecot/2006-October/016720.html
and got a link to another person with the same behavior. http://dovecot.org/list/dovecot/2006-September/016454.html
The general consensus was that nss_ldap has some emotional problems and dovecot seems to annoy it.... Kind of like poking an angry snapping turtle with a stick.
On Wed, 2007-03-14 at 12:20 -0500, Ejay Hire wrote:
The general consensus was that nss_ldap has some emotional problems and dovecot seems to annoy it.... Kind of like poking an angry snapping turtle with a stick.
The problem has been found since and it can be worked around: http://wiki.dovecot.org/AuthDatabase/Passwd
Timo Sirainen wrote:
On Wed, 2007-03-14 at 12:20 -0500, Ejay Hire wrote:
The general consensus was that nss_ldap has some emotional problems and dovecot seems to annoy it.... Kind of like poking an angry snapping turtle with a stick.
The problem has been found since and it can be worked around: http://wiki.dovecot.org/AuthDatabase/Passwd
Should this paragraph be updated in the dovecot-example.conf?
== # /etc/passwd or similar, using getpwnam() # In many systems nowadays this uses Name Service Switch, which is # configured in /etc/nsswitch.conf. WARNING: nss_ldap is known to be broken # with Dovecot. Don't use it, or users might log in as each others! # http://wiki.dovecot.org/AuthDatabase/Passwd userdb passwd { }
Maybe that should reflect the "set blocking=yes" statements from the wiki? Or doesn't this apply to userdb {} blocks? I'm a little confused if I should be setting blocking=yes for passdb, userdb, or both...
The example conf lists blocking=yes in the passdb {} blocks, but the Wiki specifically gives an example for a userdb{} block. Is blocking=yes applicable to both types of blocks? Should I be setting it both places?
Thanks, -te
-- Troy Engel | Systems Engineer Fluid, Inc | http://www.fluid.com
On Wed, 2007-03-14 at 11:02 -0700, Troy Engel wrote:
The example conf lists blocking=yes in the passdb {} blocks, but the Wiki specifically gives an example for a userdb{} block. Is blocking=yes applicable to both types of blocks? Should I be setting it both places?
I accidentally added it to passdb instead of to userdb. It doesn't at least currently work in passdb passwd (although it could, but does anyone use it anyway?)
Changed:
# /etc/passwd or similar, using getpwnam(). In many systems nowadays this # uses Name Service Switch, which is configured in /etc/nsswitch.conf. # http://wiki.dovecot.org/AuthDatabase/Passwd userdb passwd { # [blocking=yes] - By default the lookups are done in the main dovecot-auth # process. This setting causes the lookups to be done in auth worker # proceses. Useful with remote NSS lookups that may block. # NOTE: Be sure to use this setting with nss_ldap or users might get # logged in as each others! #args = }
Timo Sirainen wrote:
I accidentally added it to passdb instead of to userdb. It doesn't at least currently work in passdb passwd (although it could, but does anyone use it anyway?)
OK thanks for that, but I'm still a bit confused -- I'm after the second part of the wiki entry in relation to nss_ldap; I was thinking to set blocking=yes to just avoid any possible problems in the future with PAM/nss_ldap.
A) Does blocking=yes still apply to passdb pam {}?
B) Do I still add blocking=yes to userdb passwd {} if I'm using passdb pam {} as above? Is there no such block as 'userdb pam {}' since I don't see one in dovecot-example.conf?
Thanks for helping, it just doesn't seem clear to me yet what the right combination of settings are. I'll happily update the wiki to be more clear once y'all can get me on track. :) All my user and pass auth is done via PAM/nss_ldap to an external server.
Thanks, -te
-- Troy Engel | Systems Engineer Fluid, Inc | http://www.fluid.com
On Wed, 2007-03-14 at 11:38 -0700, Troy Engel wrote:
Timo Sirainen wrote:
I accidentally added it to passdb instead of to userdb. It doesn't at least currently work in passdb passwd (although it could, but does anyone use it anyway?)
OK thanks for that, but I'm still a bit confused -- I'm after the second part of the wiki entry in relation to nss_ldap; I was thinking to set blocking=yes to just avoid any possible problems in the future with PAM/nss_ldap.
A) Does blocking=yes still apply to passdb pam {}?
Yes, http://wiki.dovecot.org/PasswordDatabase/PAM explains what it does and why it might not be a good idea. Although no-one has yet reported any success/failure stories, those are all my own guesses..
B) Do I still add blocking=yes to userdb passwd {} if I'm using passdb pam {} as above?
You probably don't have to, but doing so might give you better performance.
Is there no such block as 'userdb pam {}' since I don't see one in dovecot-example.conf?
No. PAM can't be asked any userdb info (home dirs etc.)
Timo Sirainen wrote:
Yes, http://wiki.dovecot.org/PasswordDatabase/PAM explains what it does and why it might not be a good idea. Although no-one has yet reported any success/failure stories, those are all my own guesses..
I gotcha, makes sense having read it. Now I'm back on the fence, as I'll want to research the status of these PAM/nss_ldap memory leaks that might occur if we set it to blocking=yes.
It seems like the choice is to be asynchronous and possibly run into a user collision on the file descriptor (percentage chance, anyone? 2%? 5%?) or to use a synchronous/blocking pipeline at the expense of never releasing the PAM code (sic) and possibly leaking memory.
Here comes my "I am not a real programmer" thought -- there really is no way for dovecot-auth's forked child processes to set up their own unique descriptor? Maybe an alternative way of doing the fork? Just pondering...
-te
-- Troy Engel | Systems Engineer Fluid, Inc | http://www.fluid.com
On Thu, 2007-03-15 at 10:58 -0700, Troy Engel wrote:
Timo Sirainen wrote:
Yes, http://wiki.dovecot.org/PasswordDatabase/PAM explains what it does and why it might not be a good idea. Although no-one has yet reported any success/failure stories, those are all my own guesses..
I gotcha, makes sense having read it. Now I'm back on the fence, as I'll want to research the status of these PAM/nss_ldap memory leaks that might occur if we set it to blocking=yes.
blocking=yes doesn't break anything with nss_ldap, since without blocking=yes it'll run in one process anyway. PAM works differently.
It seems like the choice is to be asynchronous and possibly run into a user collision on the file descriptor (percentage chance, anyone? 2%? 5%?) or to use a synchronous/blocking pipeline at the expense of never releasing the PAM code (sic) and possibly leaking memory.
You're somehow mixing up these things. :) Probably because of the "blocking" naming, which actually does the opposite of what it's named.. Hmm. I guess it's too late to rename it, but I could update the wiki. My thought process went like "blocking=yes tells Dovecot that the passdb/userdb does blocking lookups, and it should do something about it so that the calls don't block everything else".
Here are the possible combinations and how they work:
blocking=yes pam, blocking=yes nss_ldap: Potential memory leaks. Fixes nss_ldap problems. Dovecot creates multiple worker processes to execute both the NSS and the PAM lookups. It increases the worker process count if all of them are busy, so it's not really synchronous/blocking.
blocking=no pam, blocking=yes nss_ldap: No memory leaks leaks. Fixes nss_ldap problems. Each PAM lookup is done in a forked process. NSS lookups are done in auth worker processes, as described above. So again no lookup blocks others.
blocking=yes pam, blocking=no nss_ldap: Potential memory leaks. Should also fix nss_ldap problems. Each NSS lookup is synchronous, so they are blocking everything else.
blocking=no pam, blocking=no nss_ldap: NSS lookups are blocking, and nss_ldap is broken.
Here comes my "I am not a real programmer" thought -- there really is no way for dovecot-auth's forked child processes to set up their own unique descriptor? Maybe an alternative way of doing the fork? Just pondering...
In the blocking=no case, there's no way to release the existing LDAP connection without exec()ing another binary which gets rid of the internal state. That's a bug in nss_ldap I think, it should close the LDAP connection when forking (since it's not closing it in endpwent()).
Timo Sirainen wrote:
blocking=yes doesn't break anything with nss_ldap, since without blocking=yes it'll run in one process anyway. PAM works differently.
Thanks for clarifying that.
You're somehow mixing up these things. :) Probably because of the "blocking" naming, which actually does the opposite of what it's named..
Yes, thanks for explaining further -- I was completely reading blocking=yes backwards from what you had designed as you pointed out. I was reading it as an instruction to Dovecot what to do, not an explanation to Dovecot what the machine is already doing.
blocking=no pam, blocking=yes nss_ldap: No memory leaks leaks. Fixes nss_ldap problems. Each PAM lookup is done in a forked process. NSS lookups are done in auth worker processes, as described above. So again no lookup blocks others.
OK this seems like the perfect solution; in dovecot.conf terms for a setup such as mine (nothing in /etc/passwd, 100% LDAP lookups for homedir, password, /etc/nsswitch.conf "passwd: files ldap", etc.) this would then be:
passdb pam { args = cache_key=%u dovecot }
userdb passwd { args = blocking=yes }
This would not block/stall in the pipelines, not cause memory leaks (since underlying code is released each cycle), avoid/fix nss_ldap issues with file descriptor reuse.
Do I finally have a good understanding now? (thanks for taking the time to work it out)
-te
-- Troy Engel | Systems Engineer Fluid, Inc | http://www.fluid.com
On Thu, 2007-03-15 at 11:33 -0700, Troy Engel wrote:
This would not block/stall in the pipelines, not cause memory leaks (since underlying code is released each cycle), avoid/fix nss_ldap issues with file descriptor reuse.
Do I finally have a good understanding now? (thanks for taking the time to work it out)
Yea, now it's correct.
Thanks Timo!
-----Original Message----- From: Timo Sirainen [mailto:tss@iki.fi] Sent: Wednesday, March 14, 2007 12:39 PM To: Ejay Hire Cc: 'Arto Saraniva'; dovecot@dovecot.org Subject: Re: [Dovecot] Postfix+Dovecot+LDAP
On Wed, 2007-03-14 at 12:20 -0500, Ejay Hire wrote:
The general consensus was that nss_ldap has some emotional problems and dovecot seems to annoy it.... Kind of like poking an angry snapping turtle with a stick.
The problem has been found since and it can be worked around: http://wiki.dovecot.org/AuthDatabase/Passwd
participants (5)
-
Arto Saraniva
-
Ejay Hire
-
hermooz＠hermooz.net
-
Timo Sirainen
-
Troy Engel