[Dovecot] Question about "slow" storage but fast cpus, plenty of ram and dovecot

[Stan Hoeppner]

Eric Rostetter put forth on 12/10/2010 10:11 PM:
> Quoting javierdemiguel at us.es:
> 
>> in our vmware esx cluster. We want to minimize disk I/O, what config
>> options should we use. We can "exchange" CPU & RAM to minimize disk i/o.
> 
> Depends on what you are doing -- pop3, imap, both, deliver or some other
> LDA?  Do you care if the indexes are lost on reboot or not?
> 
> You might try putting the indexes in, memory (either via dovecot settings
> or a RAM DISK) or on SSD.

<snipped good bare metal recommendations>

Eric you missed up above that he's running Dovecot on an ESX cluster, so
SSDs or any hardware dedicated to Dovecot isn't possible for the OP.

Javier, email is an I/O intensive application, whether an MTA spool, an
IMAP server, or POP server.  The more concurrent users you have the
greater the file I/O.  Thus, the only way to decrease packets across
your iSCSI SAN is to increase memory so more disk blocks are cached.
But keep in mind, at one point or another, everything has to be written
to disk, or deleted from disk.  So, while you can decrease disk *reads*
by adding memory to the VM, you will never be able to decrease writes,
you can only delay them with things like write cache, or in the case of
XFS, the delaylog mount option.  These comments refer to mail file I/O.

IMAP is a very file I/O intensive application.  As Eric mentioned, you
could put your user *index* files in a RAM disk or make them memory
resident via Dovecot directive.  This would definitely decrease disk
reads and writes quite a bit.  Also as Eric mentioned, if you reboot you
lose the indexes, and along with them Dovecot's key performance enabler.
 User response times will be poor until the indexes get rebuilt.

If this is a POP server, then you really have no way around the disk I/O
issue.  Due to the nature of POP, there is very little opportunity to do
effective disk block caching, unless the bulk of your users configure
their clients to check mail every 60 seconds, 24 hours a day.  In this
scenario you have good opportunity for block caching.  However, if the
bulk of the users only pop the server every half hour or more, there is
no opportunity for file caching.  The files will be read from disk every
time a user pops the server.  However, in this scenario, you'd have
relatively low disk I/O load to begin with, and you'd not be inquiring
here.  Thus, I can only assume your Dovecot server is configured for IMAP.

So, either:

1.  Increase memory and/or
2.  Move indexes to memory

#1 will be less effective at decreasing I/O.  #2 will be very effective,
but at the cost of lost indexes upon reboot or crash.

-- 
Stan