[Dovecot] Question about "slow" storage but fast cpus, plenty of ram and dovecot

[Eric Rostetter]

Quoting Stan Hoeppner <stan at hardwarefreak.com>:

> <snipped good bare metal recommendations>
>
> Eric you missed up above that he's running Dovecot on an ESX cluster, so
> SSDs or any hardware dedicated to Dovecot isn't possible for the OP.

Well, it is true I know nothing about vmware/ESX.  I know in my virtual
machine setups, I _can_ give the virtual instances access to devices which
are not used by other virtual instances.  This is what I would do.  Yes,
it is still virtualized, but it is dedicated, and should still perform
pretty well -- faster than shared storage, and in the case of SSD faster
than normal disk or iscsi.

> Javier, email is an I/O intensive application, whether an MTA spool, an
> IMAP server, or POP server.  The more concurrent users you have the
> greater the file I/O.  Thus, the only way to decrease packets across
> your iSCSI SAN is to increase memory so more disk blocks are cached.

He was already asking about throwing memory at the problem, and I think
he implied he had a lot of memory. As such, the caching is there already.
Your statement is true, but it is also a "zero config" option if he really
does have lots of memory in the machine.

> But keep in mind, at one point or another, everything has to be written
> to disk, or deleted from disk.  So, while you can decrease disk *reads*
> by adding memory to the VM, you will never be able to decrease writes,
> you can only delay them with things like write cache, or in the case of
> XFS, the delaylog mount option.  These comments refer to mail file I/O.

And in ext3, the flush rate.  Good point, that I forgot about.  It is set
to a very small value by default (2-3 seconds maybe), and can be increased
without too much danger (to say 10-30 seconds).

> IMAP is a very file I/O intensive application.  As Eric mentioned, you
> could put your user *index* files in a RAM disk or make them memory
> resident via Dovecot directive.  This would definitely decrease disk
> reads and writes quite a bit.  Also as Eric mentioned, if you reboot you
> lose the indexes, and along with them Dovecot's key performance enabler.
>  User response times will be poor until the indexes get rebuilt.

Assuming normal downtime stats, this would still be a huge win.  Since the
machine rarely goes down, it would rarely need to rebuild indexes, and hence
would only run poorly a very small percentage of the time.   Of course, it
could run _very_ poorly right after a reboot for a while, but then will be
back to normal soon enough.

One way to help mitigate this if using a RAM disk is have your shutdown script
flush the RAM disk to physical disk (after stoping dovecot) and the reload
it to RAM disk at startup (before starting dovecot).  This isn't possible if
you use the dovecot index memory settings though.

> If this is a POP server, then you really have no way around the disk I/O
> issue.

I agree.  POP is very inefficient...

> So, either:
>
> 1.  Increase memory and/or
> 2.  Move indexes to memory
>
> #1 will be less effective at decreasing I/O.  #2 will be very effective,
> but at the cost of lost indexes upon reboot or crash.

Still some room for filesystem tuning, of course, but the above two options
are of course the ones that will make the largest performance improvement
IMHO.

> --
> Stan

-- 
Eric Rostetter
The Department of Physics
The University of Texas at Austin

Go Longhorns!