[Dovecot] POP3 vs. IMAP Load/Memory usage in Dovecot 1.0.15

[lists at truthisfreedom.org.uk]

>> * All the servers are made by the same manufacturer (Dell)
>> * They are all the same model (R410)
>> * The have the same engine (24 cores, 24G RAM, SAS Drives)
>
> The R410 is a two socket Xeon box with max 2 x 6 core CPUs.  The 24 CPUs
> you see is the result of HyperThreading being enabled.  I'd disable HT
> if I were you, or those boxen mine.

OK, I'll take a look at this, thanks.

>> * The motorway is exactly the same for all servers (NFS to a NetApp 6080
>> and a RAMSAN)
>> * The weather is almost exactly the same (Same Datacentre, different
>> rooms/racks)
>> * The Driver is exactly the same (Dovecot 1.0.15)
>
> What operating system?  Linux or *BSD?  If Linux, what kernel version?
> Given that you're running Dovecot 1.0.15 I'm guessing you're using
> CentOS or RHEL 5.x and thus have kernel 2.6.18-xxx.  2.6.18 is 5 years
> old now and not inappropriate for a modern 2 socket, 6 core
> HyperThreading box.  You need a much newer kernel, preferably in the
> 2.6.3x series.  2.6.18 could be reporting incorrect load numbers on
> these machines.

Linux, Centos 5.6 and (yup, you've guessed it...) 2.6.18 again, I'll  
take a look at this, thanks.

>> 1) Load Average
>
> On Linux, load average strictly shows total system CPU usage in
> intervals, nothing else.  Neither memory, disk, nor network or anything
> else affects load average.  Thus, with a 12 core system, until you see a
> load average above 12 you have absolutely nothing to worry about.  With
> HT enabled load averages pretty much go out the window as half the
> "CPUs" are merely glorified duplicate register file phantoms.
>
> Given that all mail apps are 100% IO bound, never CPU or memory bound,
> I'd guess you'll never see a load average over 4.00 on any of these
> machines with less than 1000 concurrent connections.  This assuming you
> run a newer kernel and with HT disabled.  In other words, no more than 4
> cores worth of CPU time will ever be eaten by your workload.  What
> number do your Munin graphs show for load average for each set of boxes?
>  Do they even come close to 4?

They're showing as between 20 and 24 for the POP3 servers and 1.4 for  
the IMAP servers.

> Also note that TCP stack processing on the pop nodes will be greater
> than that of the imap boxes, eating more CPU cycles.  More data sent
> over the wire means more packets, more packets means more CPU time in
> both code/data processing and interrupts.  If you're running iptables
> rules on each host that bumps up network processing cycles a bit more yet.

OK, I'll take a look at that as well

>> 2) RAM Usage (particularly in regard to cache)
>
>> In both cases, the value for each area is higher on the three nodes
>> running POP3 than the nodes running IMAP.
>
> Almost all the memory consumption on both systems is buffer cache.  Thus
> you don't have a memory issue on either host.  The kernel will free and
> immediately reassign pages from cache to application processes as
> needed.  I don't see evidence of the pop machine using more memory, in
> fact the imap processes are using more.  Both boxes are just under 24GB
> total usage and both using right at 20GB of cache.  Looks like a default
> config Linux kernel based on the ultra aggressive caching and eating up
> nearly all memory.

So a kernel update is more than sensible...

> It may have been.  I'll know when you post your load numbers from those
> top secret graphs. ;)

LOL, see above.

Thanks again,

Matt