[Dovecot] Dovecot performance under high load (vs. Courier)

[Timo Sirainen]

On Thu, 2012-06-21 at 13:05 -0700, email builder wrote:
> Thank you very much for the fast reply.
> 
> >>  We are building a new system that will support a large number of users 
> 
> >> (high volume, high concurrent usage, etc).  We have played with Dovecot, but in 
> >> most serious applications we have traditionally used Courier IMAP.  It's my 
> >> (lay) understanding that with indexing and perhaps other things in Dovecot, it 
> >> might perform better than Courier in larger environments like this.  Am I 
> >> correct or is it less clear-cut?
> > 
> > If you disable index index files in Dovecot, its performance should be slightly 
> > better than Courier. With index files the performance is typically much better 
> > in Dovecot, especially if you use a (non-caching) webmail.
> 
> Interesting.  What would be the motivations for disabling indexing?
> Indexing is by default enabled?

Yes, enabled by default. There aren't many good reasons for disabling
indexing.

> Do you know what webmails are caching vs. non-caching?  

Nearly all of them are non-caching. (I don't know of any caching ones.)

> Am I correct that what you're pointing out is that with non-caching
> webmails you will notice IMAP performance differences more readily
> but that a caching webmail application might be better no matter
> which IMAP server because it reduces the need for webmail to make
> IMAP connections?

It's not about the IMAP connections themselves, but how often they fetch
message (meta)data. http://www.imapwiki.org/Benchmarking should explain
this better. Dovecot's indexing can lower the disk I/O usage perhaps by
10x compared to Courier.

> >>  Any tips on making the migration (not migrating an existing system, I mean 
> >> migrating our paradigm - things to consider, things to watch out for)?
> > 
> > If you don't migrate any existing users, I guess this doesn't differ 
> > much from any other optimized Dovecot installation. Usually large installations 
> > (>1M users) use NetApp NFS + Dovecot director. You might also want to enable 
> > full text searches. http://wiki2.dovecot.org/PerformanceTuning lists some other 
> > things.
> 
> Ah, I didn't know about Director.  That looks very nice.  I had in mind that
> we would have to use Perdition, but an integrated solution might be good.
> 
> Anyone have any thoughts or opinions considering Perdition vs. Director?

Dovecot proxy has several Dovecot-specific features that make it work
better than perdition (forwards client IP address to backend, handle
CAPABILITY stuff better, maybe other things).

> Full text searches don't hurt performance too bad?

They should improve the performance, at least from the user's point of
view when doing a search on webmail. But yes, the indexing itself does
cost CPU cycles, disk I/O and disk usage (perhaps 30% more disk space).