Anthony Wendell Kay wrote:

Hi,

We are thinking of converting to Dovecot to lighten our I/O load. I have done some performace tests comparing UW-IMAP to Dovecot when using mbox format if anyone is interested. They primarily examine the I/O improvements that can be gained from using Dovecot over UW when staying with mbox format:

Perhaps I just missed something - but what was used to benchmark? IMHO? I'm curious as I wouldn't mind trying to benchmark (in general) my Maildir setup. Granted, I don't have an mbox comoparison anymore for the same machine, but it would be an interesting one none the less.

Anyway, we are stuck with the mbox format for the forseeable future, and I noticed how hard hitting Dovecot is on initial access to a large mailbox, so we need a way to pre-create some of those indexes. I saw in the docs that such an indexer has been planned, but does anyone have a hack for doing it now?

IMHO, it's not a difficult transition as long as you have a good sized window to make the swap (depending on number of users and speed of the hardware), and your MDA is capable of switching easily (Procmail is). I found/wrote a couple of scripts to facilitate the transition, which were recently sent to this list. Perhaps there are other (political?) reasons for sticking w/ mbox, however.

Myth dispelling time: For those who are concerned about inode usage with Maildir, my 110GB available RAID is 21% used on disk space with mail, but only 4% used for inodes. It seems to me I'll run out of space long before I run out of inodes with my user's present usage. This is for 150 users, approx 60% of which use IMAP and 1/2 of them again store significant amounts of mail on the server. Some of my largest users have upwards of 30,000 messages (packrat managers).

-Rick

Rick Johnson, RHCE #807302311706007 - rjohnson@medata.com Linux/Network Administrator - Medata, Inc. (from home) PGP Public Key: https://mail.medata.com/pgp/rjohnson.asc

On Tue, 2004-02-17 at 02:47, Anthony Wendell Kay wrote:

We are thinking of converting to Dovecot to lighten our I/O load. I have done some performace tests comparing UW-IMAP to Dovecot when using mbox format if anyone is interested. They primarily examine the I/O improvements that can be gained from using Dovecot over UW when staying with mbox format:

http://whizzo.uoregon.edu/public/src/mailperf/results.html

Good testing. I was going to do something similiar myself, but only for the next release which will have rewritten index and mbox code. I know the current mbox code is really slow and buggy in some situations.

I've however heard of getting much larger performance benefits when switching from UW-IMAP. I think they had mostly Outlook/OE clients. Drops from average 20MB/s disk I/O to few kB/s.

Anyway, we are stuck with the mbox format for the forseeable future, and I noticed how hard hitting Dovecot is on initial access to a large mailbox, so we need a way to pre-create some of those indexes. I saw in the docs that such an indexer has been planned, but does anyone have a hack for doing it now?

This, and POP access will work better in next version too. Currently Dovecot goes and stores all kinds of cached data to index files even if client never uses them. This will change so that data is stored only when client actually asks for it which I think makes opening large mailboxes almost as fast as UW-IMAP (small hit for writing 200-500kB indexes per 10000 mails).

I'll also create Dovecot LDA which is able to update the indexes directly while storing the mail, that should help some too.

Anyway, back to coding. I hope to get it usable soon and committed to CVS. Before that I'm pretty unresponsive..