[Dovecot] Dovecot 1.0-stable mbox performance and disconnections
I've got some of our busiest users now using Dovecot instead of UW-IMAP with mbox-format mailboxes and predominantly Outlook Express as the client.
Doing some perfomance monitoring (with Solaris 8 process accounting), it looks like Dovecot needing to read only 30-50% of the disk blocks needed by UW-IMAP, but I was hoping for better!
What seems to happen, is that most of the connections use only a small number of blocks (due to Dovecot indexes, presumably) but occassionally a process will read lots of blocks, sometimes even more than a UW-IMAP session.
Is this expected, or is something going wrong with the index updates? I have the default sync settings, I think (mbox_dirty_syncs = yes, mbox_very_dirty_syncs = no, mbox_lazy_writes = yes)
My INBOX is an extreme example with ~10,000 messages and most processes reading a few hundred blocks but a couple using 20,000 or so!
Also, Dovecot seems to be dropping connections, sometimes, for no good reason (putting some extra i_info lines suggests for "inactivity"). Thunderbird tends to reconnect automatically, though. It seems it may be happening at the end of an IDLE command, something like:-
4 IDLE
- idling
- 1005 EXISTS DONE
- 1005 EXPUNGE
but then the expected
4 OK Idle completed
never comes.
Has anybody else seen this?
Best Wishes, Chris
-- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
On Tue, 2005-07-12 at 15:37 +0100, Chris Wakelin wrote:
Doing some perfomance monitoring (with Solaris 8 process accounting), it looks like Dovecot needing to read only 30-50% of the disk blocks needed by UW-IMAP, but I was hoping for better!
What seems to happen, is that most of the connections use only a small number of blocks (due to Dovecot indexes, presumably) but occassionally a process will read lots of blocks, sometimes even more than a UW-IMAP session.
Most of this probably happens when new mails are appended to mailbox using some external tool, and after that the mailbox is selected again in Dovecot (if new mails arrive to selected mailbox, mbox_dirty_syncs=yes prevents the mbox from being fully re-read).
Another reason why Dovecot might be reading more than UW-IMAP is that when setting message flags, it reads the mail headers for those mails, and later when closing the mailbox it writes the flags. Those extra reads are completely unnecessary and I should fix the code not to do that..
Is this expected, or is something going wrong with the index updates? I have the default sync settings, I think (mbox_dirty_syncs = yes, mbox_very_dirty_syncs = no, mbox_lazy_writes = yes)
Setting mbox_very_dirty_syncs = yes makes Dovecot not re-read the mbox entirely when selecting externally modified mbox, unless it notices that mails aren't where they should be.
Also, Dovecot seems to be dropping connections, sometimes, for no good reason (putting some extra i_info lines suggests for "inactivity"). Thunderbird tends to reconnect automatically, though. It seems it may be happening at the end of an IDLE command, something like:-
Hmm.. Dovecot closes the connection because of inactivity? How long has Thunderbird been idling then? It should be disconnected if it doesn't say anything for 30 minutes, but not before that.
Timo Sirainen wrote:
Setting mbox_very_dirty_syncs = yes makes Dovecot not re-read the mbox entirely when selecting externally modified mbox, unless it notices that mails aren't where they should be.
OK, I've set up another test Dovecot service with "mbox_very_dirty_syncs = yes", just for me ;)
I don't think I'd like to risk it with the other users as they stand a chance of using UW-IMAP at the same time as Dovecot (e.g. use Webmail from home while leaving Outlook Express running on a machine on a Dovecot-ed subnet at work).
<snip>
Hmm.. Dovecot closes the connection because of inactivity? How long has Thunderbird been idling then? It should be disconnected if it doesn't say anything for 30 minutes, but not before that.
I've just looked at my packet traces again, and it seems it may be Thunderbird closing the connection (it's tricky to see as Thunderbird will open up to five concurrent connections by default). My colleague who runs PC-Pine 4 has had problems, though. I haven't managed to reproduce the problem with my "debug" client (telnet). I'll keep digging...
Best Wishes, Chris
-- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
participants (2)
-
Chris Wakelin
-
Timo Sirainen