High I/O wait with Dovecot POP3

[Frido Otten]

Op 25-10-14 om 02:21 schreef Timo Sirainen:
> On 22 Oct 2014, at 06:15, Frido Otten <fridi at dds.nl> wrote:
>
>> In the past we've had 4 loadbalanced servers configured with courier
>> pop3 and dovecot handling respectively POP3 and IMAP connections. These
>> 4 servers have a shared NFS storage on a Netapp FAS3220 controller with
>> SAS disks. This setup is handling 50 POP3 sessions per second without
>> any problems.
>>
>> Now we're installing 6 new loadbalanced servers to do the same job on
>> the same storage, but using Dovecot to handle the POP3 sessions instead
>> of Courier. The IMAP sessions are already handled by those 6 new
>> servers, but we're encountering high iowait issues when using POP3 on
>> those servers.
>>
>> I've applied the hints from http://wiki2.dovecot.org/NFS but this had no
>> improvement.
> See also http://wiki2.dovecot.org/POP3Server - especially pop3_fast_size_lookups may be useful.
I've applied the hints on this page and also set pop3_fast_size_lookups
to yes. The same for pop3_no_flag_updates.
>> mail_nfs_index = yes
>> mail_nfs_storage = yes
> These slow things down a bit, ideally get rid of these and use Dovecot directors instead.
Can these make such a big difference? I've commented these out. When
switching to the Dovecot as POP3 server the iowait immediately
skyrockets to 80-90% as well as the overall load of the machine.
>> protocol pop3 {
>>  pop3_uidl_format = %v.%u
>> }
> I don't think your Courier installation was using this UIDL format? If you switched to Dovecot the UIDLs would likely differ and clients would redownload mails. Try to preserve the UIDLs as described in http://wiki2.dovecot.org/Migration/Courier - and even then I'd recommend setting pop3_uidl_format to something else, for example %f

This was indeed an issue. I've changed this to %f and checked with our
current running courier and now the output of UIDL are the same on
Courier and Dovecot. This change unfortunately had no effect on the high
iowait.