[Dovecot] rule of thumb for indexing overhead

[Christian Balzer]

Hello,

On Wed, 29 Aug 2007 18:10:00 -0700 Tom Bombadil <grlists at gmail.com> wrote:
> 
> > 
> >> of indexing during delivery are negligible for us in general and
> >> potentially negative in some scenarios (mass mail to many/all users).
> > 
> > Is this negative hypothetical or have you actually seen load spikes in
> > situations like this?
> 
> I actually did see that.
> In the case of exim, for each piece of mail going to the mailbox, one
> exim process is spawned, and this exim delivery process spawns the
> dovecot's LDA. The load pretty much doubles.
> 
There is that little detail of the additional processes spawned, but
the overhead for that alone is not so much of an issue here (linux,
plenty of CPU and memory to spare). But when you have 80000 emails
rushing towards your mailstorage (and yes, of course we have sensible
limits on number of process, maximum load before defer, etc) your
I/O will be a bottleneck and adding the additional strain of indexing
on top of that is not a good idea. It is only at times of mass mails 
like that I see our mailstorge boxes break into a light sweat and 
avoiding any additional load then is just the sensible thing to do. 
The indexing for these mails will be spread out over hours to
days (frequency of access by the clients) instead of being lumped 
on top of the existing peak and I/O saturation. 

It all boils down to your hardware and usage patterns.
Our systems are plenty fast and any delay by having to (re)build the
indexes is hardly noticeable.
But if you have a system with less reserves, users that access mail
very infrequently but get plenty of mail (so indexing at access time
will be an involved process) and no mass mail spikes, dovecot LDA
starts to look a lot more promising. ^_^

Regards,

Christian
-- 
Christian Balzer        Network/Systems Engineer                NOC
chibi at gol.com   	Global OnLine Japan/Fusion Network Services
http://www.gol.com/