[Dovecot] Benchmarking CVS HEAD vs. v1.0

[Timo Sirainen]

Some new features in CVS HEAD are:

 - v1.0 saves most of the data to dovecot.index.cache only when the
client is FETCHing the messages. With mboxes the message headers are
saved while parsing the mbox, which is almost the same thing.

With CVS HEAD the dovecot.index.cache file is updated already when the
new messages are being saved (with deliver or IMAP APPEND). This avoids
reading the message second time (although if client fetches the new
message bodies immediately it doesn't probably help that much).

 - Mailbox list index. LIST command can be handled by reading the
mailboxes from the index. The index also contains the mailbox's message
counters, so STATUS command can be handled by only looking into the
index file.

 - Multiple commands can be handled simultaneously. For example you can
start a long running SEARCH command and then at the same time continue
FETCHing new messages. Or if you're doing a long FETCH (eg. fetching all
message bodies) you can do a smaller FETCH while the long one is still
running, and get the result in the middle of the long running FETCH
reply.

There are also indexing support for SORT and THREAD, and full text
search indexes for SEARCH BODY/TEXT, but I didn't benchmark these now.

It looks like CVS HEAD is quite a lot faster than v1.0. I think that's
mostly because of the APPEND optimization.

Here are the results:

100MB ext3 filesystem inside a file in XFS.

C: CVS HEAD, fsync_disable=yes, mmap_disable=yes
S: CVS branch_1_0 (upcoming 1.0.rc28), fsync_disable=yes, mmap_disable=yes

./imaptest seed=1 secs=60
   Logi  List Stat Sele  Fetc  Fet2  Stor Dele Expu  Appe Logo 
C: 11275 5647 5597 11275 11264 16179 5571 4126 11256 6074 22531 
C: 11287 5740 5606 11287 11275 16193 5698 4154 11267 6064 22553 
S:  9201 4660 4575 9197   9190 13107 4614 2843  9184 3601 18382 
S:  9271 4701 4657 9267   9260 13189 4580 2799  9256 3540 18522 
   82%   82%                                         59%

./imaptest seed=1 secs=60 no_pipelining
   Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 
C: 5756 2856 2795 5756 5747 8087 2796 1942 5746 5081 11489 
C: 5779 2865 2872 5777 5768 8273 2929 1965 5768 5138 11532 
S: 5016 2564 2467 5016 5008 7090 2512 1756 5006 4116 10012 
S: 5009 2529 2481 5008 4999 7142 2503 1734 4997 4165 9998 
   87%  89%                                     81%

----------

Trying to see how much mailbox list indexing code affects the results by
disabling LIST and STATUS (which means that CVS HEAD is doing more I/O
by updating the list indexes which are never used). Looks like the
indexes do help some, since CVS HEAD is ~5% less faster now.

./imaptest seed=1 secs=60 list=0 status=0
   Logi  Sele  Fetc  Fet2  Stor Dele Expu  Appe Logo 
C: 11435 11432 11422 16341 5639 4122 11415 5941 22848 
S: 9984   9984  9976 14347 5013 2987  9967 3840 19947 
   87%                                     65%

./imaptest seed=1 secs=60 list=0 status=0 no_pipelining
   Logi Sele Fetc Fet2 Stor Dele Expu Appe Logo 
C: 5778 5778 5768 8105 2922 2021 5767 5107 11533 
S: 5240 5239 5230 7384 2525 1823 5229 4343 10460 
   90%                                85%

---------------

Any difference with just one client instead of 10 simultaneously
running? Looks like with pipelining CVS HEAD is now even faster, but
without pipelining it's a bit slower. I'm not sure why.

I tried with different max. command queue lengths with CVS HEAD. The C1,
C4 and C8 mean queue lengths of 1, 4 and 8 commands. 1 practically
disables it, so it runs like v1.0. Doesn't look like the queue length
makes much of a different.

./imaptest seed=1 secs=60 list=0 status=0 clients=1
    Logi Sele Fetc Fet2  Stor Dele Expu Appe Logo 
C4: 9968 9968 9967 14244 5020 2664 9967 2990 19934
S : 6510 6510 6508  9340 3216 1729 6508 1942 13017
    65%                                 63%

C1: 9956 9956 9954 14314 4959 2642 9954 2987 19907
C4: 9968 9968 9967 14244 5020 2664 9967 2990 19934
C8: 9978 9978 9976 14329 4848 2618 9976 2962 19953

./imaptest seed=1 secs=60 list=0 status=0 clients=1 no_pipelining
   Logi Sele Fetc Fet2 Stor Dele Expu Appe Logo 
C: 1942 1942 1941 2711  962  508 1941  605 3882 
S: 1868 1868 1867 2669  965  520 1866  618 3734 
   96%                                102%

---------------

mmap_disable=no seems to work more slowly after all even with etx3..:

   Logi List Stat Sele Fetc Fet2  Stor Dele Expu Appe Logo 
C: 8254 4150 4139 8254 8244 11813 4126 2791 8234 3658 16487 
S: 7833 3940 3910 7829 7821 11243 3846 2345 7816 2966 15646 
   95%                                           81%

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20070314/a02fce48/attachment.pgp