[Dovecot] dbmail benchmarking

[Timo Sirainen]

I thought I'd try benchmarking with dbmail (v2.2.4) to see how much  
slower a SQL backend could actually be. Skip to bottom for the  
conclusions.

Originally I ran the tests with the databases being in XFS  
filesystem. MySQL's performance was horrible. It went 3-7x faster  
with ext3.

MySQL 5.0.30 backend (innodb):

./imaptest clients=1 - append=100 seed=1 secs=30 msgs=1000000 logout=0
Logi Sele Appe
100% 100% 100%
             5%
    1  291  303

So that's 10 messages/sec saved. Now how about with 5 concurrent  
clients?

Logi Sele Appe
100% 100% 100%
             5%
    5 1259 1332

Pretty well. Then something more generic:

./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0
Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100% 100%  50% 100% 100% 100%
                           30%                  5%
    1   37   36   75   73  110   34   24   73   78

Without Fetc (header/metadata fetching):
Logi List Stat Sele Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100%  50% 100% 100% 100%
                      30%                  5%
    1   94   94  199  283  102   85  198  210

PostgreSQL 8.1.5 backend:

./imaptest clients=1 - append=100 seed=1 secs=30 msgs=1000000 logout=0
Logi Sele Appe
100% 100% 100%
             5%
    1  267  277

./imaptest clients=5 - append=100 seed=1 secs=30 msgs=1000000 logout=0
Logi Sele Appe
100% 100% 100%
             5%
    5 1094 1144

./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0
Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100% 100%  50% 100% 100% 100%
                           30%                  5%
    9   29   40   74   72   99   22   12   64   71

./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0 fetch=0
Logi List Stat Sele Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100%  50% 100% 100% 100%
                      30%                  5%
   35  105   95  200  277   54   70  165  175

The last two tests gave "Unexpected tagged reply:" errors that I  
didn't get with MySQL. So apparently there's some PostgreSQL-specific  
bug in dbmail.

The same values with Dovecot 1.0 + maildir:

./imaptest clients=1 - append=100 seed=1 secs=30 msgs=1000000 logout=0
Logi Sele Appe
100% 100% 100%
             5%
    1  346  364

./imaptest clients=5 - append=100 seed=1 secs=30 msgs=1000000 logout=0
Logi Sele Appe
100% 100% 100%
             5%
    5 1408 1470

./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0
Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100% 100%  50% 100% 100% 100%
                           30%                  5%
    1  130  133  259  258  382  123  127  257  271

./imaptest clients=1 seed=1 secs=30 msgs=1000000 logout=0 fetch=0
Logi List Stat Sele Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100%  50% 100% 100% 100%
                      30%                  5%
    1  155  163  339  478  169  175  338  354

So, what are the conclusions?

  - In raw append speed dbmail is almost as fast as maildir.
  - In raw read/write speed maildir is about 1,6 times faster
  - When adding metadata fetches Dovecot is 4 times faster than  
dbmail. This is most likely because dbmail doesn't have a cache  
equivalent to dovecot.index.cache so it has to do the fetches the  
slow way.

I would have liked to also run the generic tests with more than 1  
client, but then I start hitting dbmail bugs and the test stops  
(reported already to their bugtracker).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20070512/ca097f37/attachment.pgp