Here is my dovecot -n:

# 1.1.3: /etc/dovecot.conf log_path: /var/log/dovecot info_log_path: /var/log/dovecot-info login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_greeting: MECnet Mail System, Authorized Use Only, Please Log In. login_process_per_connection: no login_process_size: 1024 login_max_processes_count: 1024 login_max_connections: 1024 max_mail_processes: 50000 verbose_proctitle: yes first_valid_uid: 50 mail_uid: 93 mail_gid: 12 mail_location: maildir:/var/spool/maildirs/%d/%n/Maildir mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_process_size: 1024 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 namespace: type: private prefix: INBOX. inbox: yes list: yes subscriptions: yes auth default: verbose: yes passdb: driver: ldap args: /etc/dovecot-ldap.conf plugin: quota: maildir

My buddy and I found

#define DB_LDAP_MAX_QUEUE_SIZE 1024

in the db-ldap.h file in the source.

We believe we hitting this threshold for some reason. So, we are
looking to increase this to 8192. However, when trying to build the
RPM source from atrpms, I get this:

# rpmbuild -ba dovecot.spec error: line 1: Unknown tag: %bcond_without inotify

Any help?

Dave

Quoting Jurvis LaSalle lasalle@idi.harvard.edu:

On Oct 8, 2008, at 8:01 AM, David Cunningham wrote:

...

After a few hours of running, I get tons of the following errors in
my logs:

dovecot: Oct 08 07:41:50 Error: auth(default):
ldap(user@domain,x.x.x.x): Request queue is full

I removed the username and IP, obviously.

Any idea how to stop this?

I have about 5 Thousand users using horde that login ever 1-5
minutes to refresh their page. I assume it is a setting, but I am
confused as to why it doesn't happen almost right away. It seems
to take some time to build up.

Please help! This is taking my webmail system down hourly.

dovecot -n?

Hunch is login_max_processes_count is too low. http://wiki.dovecot.org/LoginProcess

hth, JL

David Cunningham schrieb:

...

...

I have about 5 Thousand users using horde that login ever 1-5 minutes to refresh their page.

Imapproxy can be configured to cache connections only for a defined amount of time, eg. close connections not used for more than a few minutes. Setting is cache_expiration_time in imapproxy.conf.

Remember: Horde creates a new imap connection for every page click, imapproxy heavily reduces this login/logout noise.

If your 5k users are all using webmail *simultaneously* I don't know how the proxy will perform.

Thank you, I will try the caching.

Dave

Quoting Timo Sirainen tss@iki.fi:

On Wed, 2008-10-08 at 10:48 -0400, David Cunningham wrote:

...

I agree. In fact, I may have found a DNS issue that may have been causing login sessions to hang and thus reach max too quickly. The last few hours have been stable. So, I am keeping my fingers crossed.

I have also recompiled dovecot and changed the setting in db-ldap.h that reads:

#define DB_LDAP_MAX_QUEUE_SIZE 1024

to

#define DB_LDAP_MAX_QUEUE_SIZE 8192

If you're getting more than 1024 requests queued, something's wrong or you have hundreds or logins per second. Which one is it? (5000 users logging in once per minute is still only 83/sec)

The queue keeps increasing if the LDAP server isn't replying to old requests. So have you looked at the LDAP server side if it's running too slow?

Anyway two things you could do here:

1. Enable auth cache with large enough size so Dovecot doesn't consult LDAP server nearly as much.

2. Increase the number of auth processes (auth { .. count=5 }), so that you'll use more connections and hopefully the LDAP server likes that better than one connection sending lots of requests. Or maybe not.

On Wed, 2008-10-08 at 16:13 -0400, David Cunningham wrote:

Simply changing auth_cache_size to a non-zero number enables caching, correct?

How big is too big?

If it uses up too much of your memory. Although you'll probably also need to change auth_process_size then, because by default it kills the process if it grows larger than 256 MB. You could also disable it by setting it to 0.

Where does it cache it?

Here is what I set:

auth_cache_size = 1048576

I was hoping for 1GB worth of cache.

I think you could do with a lot less :) I think a single user takes maybe 50-100 bytes normally. Even with 200 bytes your 5000 users would fit into 1 MB. Of course negative hits are also cached, so the rest of the memory would go to them. I'd decrease it to 10 MB or so :)

Also you might want to increase auth_cache_ttl from the default hour.

Well, most of my issues are gone with adding auth cache. However, I
am having an issue. Sometimes, even though cache incorrect passwords
is disabled, new passwords do not work. It would seem that once a
user logs in with one password successfully the cache does not
automatically retry if the user tries a different passwords. I would
think that the auth cache should check to see if the password changed
on the ldap server if something other than the cached password is
entered.

Is this something wrong with my configuraiton, or the auth code itself?

Thanks,

Dave

Quoting Timo Sirainen tss@iki.fi:

On Wed, 2008-10-08 at 08:01 -0400, David Cunningham wrote:

...

After a few hours of running, I get tons of the following errors in my logs:

dovecot: Oct 08 07:41:50 Error: auth(default): ldap(user@domain,x.x.x.x): Request queue is full

BTW. I improved this error message slightly to also tell how many seconds old data is in the queue. http://hg.dovecot.org/dovecot-1.1/rev/0329dc4df5ed

I guess you're using auth binds? If you weren't, I think it wouldn't be possible to fill the queue.

Yes, i telnet to port 143 and enter everything manually.

Dave

Quoting Charles Marcus CMarcus@Media-Brokers.com:

On 11/19/2008 10:17 PM, David Cunningham wrote:

...

Well, most of my issues are gone with adding auth cache. However, I am having an issue. Sometimes, even though cache incorrect passwords is disabled, new passwords do not work. It would seem that once a user logs in with one password successfully the cache does not automatically retry if the user tries a different passwords. I would think that the auth cache should check to see if the password changed on the ldap server if something other than the cached password is entered.

Is this something wrong with my configuraiton, or the auth code itself?

Maybe it is the mail client doing the caching... have you tested this on the command line?

--

Best regards,

Charles

On Wed, 2008-11-19 at 22:17 -0500, David Cunningham wrote:

Well, most of my issues are gone with adding auth cache. However, I
am having an issue. Sometimes, even though cache incorrect passwords
is disabled,

Do you mean auth_cache_negative_ttl=0 by this? It only affects "user not found" caching.

new passwords do not work. It would seem that once a
user logs in with one password successfully the cache does not
automatically retry if the user tries a different passwords. I would
think that the auth cache should check to see if the password changed
on the ldap server if something other than the cached password is
entered.

Is this something wrong with my configuraiton, or the auth code itself?

The way it should work is that:

1. User logs in with password X which succeeds.
2. Password is changed to Y.
3. User logs in with password Y. Dovecot sees that X != Y, but it sees that the previous auth succeeded, so it'll do an auth lookup, sees that the password was changed and caches it.

But this can also happen:

1. User logs in with password X which succeeds.
2. Password is changed to Y.
3. User logs in with password X, which succeeds.

Or:

1. User logs in with password X which succeeds.
2. User logs in with password Y. Dovecot sees that X != Y, but it sees that the previous auth succeeded, so it'll do an auth lookup and sees that the password wasn't changed.
3. Password is changed to Y.
4. User logs in with password Y. Dovecot sees that X != Y, but it sees that the previous auth failed, so it doesn't bother doing another lookup.

Can you consistently make Dovecot behave differently as described above?

I think the last thing you say is exactly what is happening to me. I
think the user is updating the password, but a slight delay in my LDAP
replication is causing them to try the new password before it is
actually the new password.

Yes, I was refering to auth_cache_negative_ttl=0. I didn't realize
that was user not found only. Is there any way to force the cache to
check the password for anything that was not previously cached as
being the correct password?

Dave

Quoting Timo Sirainen tss@iki.fi:

On Wed, 2008-11-19 at 22:17 -0500, David Cunningham wrote:

...

Well, most of my issues are gone with adding auth cache. However, I am having an issue. Sometimes, even though cache incorrect passwords is disabled,

Do you mean auth_cache_negative_ttl=0 by this? It only affects "user not found" caching.

...

new passwords do not work. It would seem that once a user logs in with one password successfully the cache does not automatically retry if the user tries a different passwords. I would think that the auth cache should check to see if the password changed on the ldap server if something other than the cached password is entered.

Is this something wrong with my configuraiton, or the auth code itself?

The way it should work is that:

1. User logs in with password X which succeeds.
2. Password is changed to Y.
3. User logs in with password Y. Dovecot sees that X != Y, but it sees that the previous auth succeeded, so it'll do an auth lookup, sees that the password was changed and caches it.

But this can also happen:

1. User logs in with password X which succeeds.
2. Password is changed to Y.
3. User logs in with password X, which succeeds.

Or:

1. User logs in with password X which succeeds.
2. User logs in with password Y. Dovecot sees that X != Y, but it sees that the previous auth succeeded, so it'll do an auth lookup and sees that the password wasn't changed.
3. Password is changed to Y.
4. User logs in with password Y. Dovecot sees that X != Y, but it sees that the previous auth failed, so it doesn't bother doing another lookup.

Can you consistently make Dovecot behave differently as described above?

Wowa, that's easy enough. I will do that the next time that I upgrade
in a few weeks.

Dave

Quoting Timo Sirainen tss@iki.fi:

On Fri, 2008-11-21 at 14:50 -0500, David Cunningham wrote:

...

Is there any way to force the cache to check the password for anything that was not previously cached as being the correct password?

Nope. Hmm. Perhaps there should be a different TTL for that. I don't really like adding new settings though.

For now you can at least do it by modifying sources: src/auth/passdb-cache.c:

  if (ret == 0 && node->last_success) {

Change it to:

  if (ret == 0) {