[Dovecot] 45second delays on log in
Dear List Members,
We have been running Dovecot as an IMAP server for web based access
to email for our customers. At the moment, when the server is under
load, some users are finding that it takes 45 seconds to log in. I
was wondering if any of you guys would have any tips. I would love
any information that would help solve or debug this issue.
Background information:
The server is Debian with kernel 2.4.31. We are running Dovecot
1.0rc6. I have reviewed the change logs and not found any changes
pertinent to our situation from rc6 to rc14 (But maybe I am wrong).
We have our own auth mechanism. This auth mechanism is the same used
across our other products and we haven't had the same delays with the
other products. From the information in the various log files the
auth mechanism responds almost instantaneously but it still takes a
long time for
1 OK Logged in.
message to arrive on the screen when using telnet localhost 143 to
login.
All the mailboxes are stored over NFS. I have taken the step of
moving the INDEXES from NFS to a local hard drive using
default_mail_env = maildir:%h/mail:INDEX=/spare/%h
in the config file. (/spare is a locally attached hard drive). We
are using Maildir for our mailboxes so I am told that locking should
not be a problem. A mailbox would never be accessed by two machines
simultaneously due to the architecture of the system.
In order to gain performance increases we also implemented an IMAP
proxy between the webmail client and the imap server. This removes
delays in all but the first attempt to access webmail.
When logging into a user for which login has been verified to take
about 45seconds deletion of the large numbers of messages seemed to
speed up login for that particular mailbox indicating to me that it
was other things that dovecot does after login and before the 1 OK
Logged in. message is displayed that takes up all the time.
Unfortunately this theory failed when I tried it with other users.
I have also analysed the log files. of the 100,000 lines in the log
file
49 were messages:
Child 12040 (login) returned error 89
54 were messages:
imap-login: Master sent reply with unknown tag 18
The actual number at the end of the message changes lots.
The frequency of the extended logins would make it extremely unlikely
that these messages are the problem.
Any help is very much appreciated.
Thanks,
Ben Marsh
hi,
can you at least describe:
roughly how you authenticate
how do you combine dovecot and your auth system?
darix
-- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org
On 14/11/2006, at 10:07 PM, Marcus Rueckert wrote:
hi,
can you at least describe:
- roughly how you authenticate
Dovecot communicate with a daemon over a UNIX domain socket.
- how do you combine dovecot and your auth system?
A passdb and userdb part was written for the auth-worker worker process.
Thanks,
Ben Marsh
Hi,
maybe a stupid idea, but it looks like a double DNS timeout. Can the server resolve the incoming IP addresses to names ?
Charles
Ben Marsh wrote:
Dear List Members,
We have been running Dovecot as an IMAP server for web based access to email for our customers. At the moment, when the server is under load, some users are finding that it takes 45 seconds to log in. I was wondering if any of you guys would have any tips. I would love any information that would help solve or debug this issue.
Background information:
The server is Debian with kernel 2.4.31. We are running Dovecot 1.0rc6. I have reviewed the change logs and not found any changes pertinent to our situation from rc6 to rc14 (But maybe I am wrong).
We have our own auth mechanism. This auth mechanism is the same used across our other products and we haven't had the same delays with the other products. From the information in the various log files the auth mechanism responds almost instantaneously but it still takes a long time for
1 OK Logged in.
message to arrive on the screen when using telnet localhost 143 to login.
All the mailboxes are stored over NFS. I have taken the step of moving the INDEXES from NFS to a local hard drive using
default_mail_env = maildir:%h/mail:INDEX=/spare/%h
in the config file. (/spare is a locally attached hard drive). We are using Maildir for our mailboxes so I am told that locking should not be a problem. A mailbox would never be accessed by two machines simultaneously due to the architecture of the system.
In order to gain performance increases we also implemented an IMAP proxy between the webmail client and the imap server. This removes delays in all but the first attempt to access webmail.
When logging into a user for which login has been verified to take about 45seconds deletion of the large numbers of messages seemed to speed up login for that particular mailbox indicating to me that it was other things that dovecot does after login and before the 1 OK Logged in. message is displayed that takes up all the time. Unfortunately this theory failed when I tried it with other users.
I have also analysed the log files. of the 100,000 lines in the log file
49 were messages:
Child 12040 (login) returned error 89
54 were messages:
imap-login: Master sent reply with unknown tag 18
The actual number at the end of the message changes lots.
The frequency of the extended logins would make it extremely unlikely that these messages are the problem.
Any help is very much appreciated.
Thanks,
Ben Marsh
-- Charles Bueche charles@bueche.ch sand, snow, wave, wind and net -surfer A-Cat SUI 192
On Tue, 2006-11-14 at 17:19 +1100, Ben Marsh wrote:
We have our own auth mechanism. This auth mechanism is the same used
across our other products and we haven't had the same delays with the
other products. From the information in the various log files the
auth mechanism responds almost instantaneously but it still takes a
long time for1 OK Logged in.
Internally it works like this:
- Login process sends auth request to auth process and waits for a reply. Apparently this works fast for you.
- The client's fd is sent to master process, which again asks auth process for the userdb information.
- A new IMAP process is forked, and the fd is moved there.
- IMAP process writes the "OK Logged in." after it has initialized its configuration.
Best way to debug this would be to add some debugging lines to show at which point the delay is.
All the mailboxes are stored over NFS. I have taken the step of
moving the INDEXES from NFS to a local hard drive using
I don't think any of this should matter, since the "OK Logged in" is sent before any mailbox is opened.
imap-login: Master sent reply with unknown tag 18
I fixed this in rc14. It only happened if the client was disconnected while it was waiting for a "OK" reply from master. So I guess the long delay is related to master process not getting a userdb reply fast enough, or master process getting stuck somewhere for a long time.
participants (4)
-
Ben Marsh
-
Charles Bueche
-
Marcus Rueckert
-
Timo Sirainen