[Dovecot] distributed mdbox

[list at airstreamcomm.net]

On Fri, 23 Mar 2012 23:03:01 +0200, Timo Sirainen <tss at iki.fi> wrote:
> On 23.3.2012, at 19.43, <list at airstreamcomm.net>
<list at airstreamcomm.net>
> wrote:
> 
>>> Have you tried stress testing it with imaptest? Run in parallel for
both
>>> servers:
>> I did stress test it, but we have developed a "mail bot net" tool for
the
>> purpose.  I should mention this was tested using dovecot 1.2, as this
is
>> our current production version (hopefully will be upgrading soon).  Its
>> comprised of a control server that starts a bot network of client
>> machines
>> that creates pop/imap connections (smtp as well) on our test cluster of
>> dovecot (and postfix) servers.  In my test I distributed the load
across
>> a
>> two node dovecot (/postfix) cluster back ended by glusterfs, which has
>> SAN
>> storage attached to it.  I actually didn't change my configuration from
>> when I had a test NFS server connected to the test servers (mmap
>> disabled,
>> fcntl locking, etc), because glusterfs was an afterthought when we were
>> stress testing our new netapp system using NFS.  We have everything in
>> VMware, including the glusterfs servers.  Using five bot servers and
>> connecting 7 times a second from each server (35 connections per
second)
>> for both pop and imap (70 total connections per second) split between
two
>> dovecot servers I was not seeing any big issues.  The load average was
>> low,
>> and there were no errors to speak of in dovecot (or postfix).  I was
>> mounting the storage with the glusterfs native client, not using NFS
>> (which
>> I have not tested).  I would like to do a more thorough test of
glusterfs
>> using Dovecot 2.0 on some dedicated hardware and see how much further I
>> can
>> push the system.
> 
> What did the bots do? Add messages and delete messages as fast as they
> could? I guess that's mostly enough to see if things work. imaptest
anyway
> hammers the server as fast as it can with all kinds of commands.

We created two python scripts on the bots that listed all the messages in
the inbox then deleted all the messages in the inbox, one script doing pop
and the other doing imap.  The bots were also sending messages to the
server simultaneously to repopulate inboxes.  I didn't know about imaptest,
thanks!