[Dovecot] dbox benchmarks
http://hg.dovecot.org/dovecot-dbox-redesign/
Looks like multi-dbox scales pretty nicely. Even after 100k messages the peak saved msgs/sec is the same as the initial saved msgs/sec, even if the average slows down somewhat.
I tested this by first deleting mailbox, then running "imaptest" for a second to get saving to start writing several fields to dovecot.index.cache file. Then ran imaptest for 60 seconds that did nothing but save messages using IMAP APPEND commands. The summary is that multi-dbox is almost 10x as fast as maildir when fsync is disabled and 2,5 faster with fsyncing:
fsync_disable=yes:
multi-dbox : ~1700 -> ~1500 msgs/sec, 90842 msgs/60 sec single-dbox: ~1700 -> ~1400 msgs/sec, 86149 msgs/60 sec maildir : ~330 -> ~110 msgs/sec, 9789 msgs/60 sec
fsync_disable=no:
multi-dbox : ~220 -> ~170 msgs/sec, 10113 msgs/60 sec maildir : ~100 -> ~70 msgs/sec, 4123 msgs/60 sec
Also I just noticed that saving messages with two clients gets a peak of 2200 msgs/sec, most likely because two CPU cores can be used then. Hmm. I guess if there were no locking it should have reached 3400 msgs/sec, but it's better than no performance increase.
And the raw data:
multi-dbox, fsync_disable=yes
imaptest clients=1 secs=1;imaptest clients=1 msgs=1000000 - logout=0 append=100,0 secs=60 Logi Sele Appe 100% 100% 100% 1 1 1759 1/ 1 0 0 1424 1/ 1 0 0 1734 1/ 1 ... 0 0 1701 1/ 1 0 0 1640 1/ 1 0 0 778 1/ 1 0 0 1305 1/ 1 0 0 1598 1/ 1
Totals: Logi Sele Appe 100% 100% 100% 1 1 90842
multi-dbox, fsync_disable=no
imaptest clients=1 secs=1;imaptest clients=1 msgs=1000000 - logout=0 append=100,0 secs=60 Logi Sele Appe 100% 100% 100% 1 1 209 1/ 1 0 0 227 1/ 1 0 0 208 1/ 1 ... 0 0 203 1/ 1 0 0 145 1/ 1 0 0 156 1/ 1 0 0 180 1/ 1 0 0 162 1/ 1
Totals: Logi Sele Appe 100% 100% 100% 1 1 10113
maildir, fsync_disable=yes
Logi Sele Appe 100% 100% 100% 1 1 331 1/ 1 0 0 311 1/ 1 0 0 288 1/ 1 ... 0 0 96 1/ 1 0 0 119 1/ 1 0 0 111 1/ 1
Totals: Logi Sele Appe 100% 100% 100% 1 1 9789
maildir, fsync_disable=no
Logi Sele Appe 100% 100% 100% 1 1 77 1/ 1 0 0 101 1/ 1 0 0 45 1/ 1 ... 0 0 52 1/ 1 0 0 67 1/ 1 0 0 89 1/ 1
Totals: Logi Sele Appe 100% 100% 100% 1 1 4123
single-dbox, fsync_disable=yes
imaptest clients=1 secs=1;imaptest clients=1 msgs=1000000 - logout=0 append=100,0 secs=60 Logi Sele Appe 100% 100% 100% 1 1 1765 1/ 1 0 0 1715 1/ 1 0 0 1764 1/ 1 ... 0 0 1722 1/ 1 0 0 1679 1/ 1 0 0 1678 1/ 1 0 0 828 1/ 1 0 0 1467 1/ 1 0 0 1469 1/ 1 0 0 1149 1/ 1 0 0 1135 1/ 1
Totals: Logi Sele Appe 100% 100% 100% 1 1 86149
I can see where large mailservers would benefit from significant write performance increases - but unless the server is being actively limited by the local delivery agent, what other performance benefits does this offer?
In particular, is there any increase in READ performance by using dbox?
Does it reduce memory consumption at all?
-- Daniel
On Apr 2, 2009, at 10:08 PM, Daniel L. Miller wrote:
I can see where large mailservers would benefit from significant
write performance increases - but unless the server is being
actively limited by the local delivery agent, what other performance
benefits does this offer?In particular, is there any increase in READ performance by using
dbox?
Should be, since the files are larger there is most likely less
fragmentation. But it's more difficult to benchmark read performance
in a realistic way.
Does it reduce memory consumption at all?
Maybe a bit, but that's probably irrelevant.
Timo Sirainen wrote:
On Apr 2, 2009, at 10:08 PM, Daniel L. Miller wrote:
I can see where large mailservers would benefit from significant write performance increases - but unless the server is being actively limited by the local delivery agent, what other performance benefits does this offer?
In particular, is there any increase in READ performance by using dbox?
Should be, since the files are larger there is most likely less fragmentation. But it's more difficult to benchmark read performance in a realistic way.
Does it reduce memory consumption at all?
Maybe a bit, but that's probably irrelevant.
what's most relevant for us about dbox: Backups. We have lots of customers with huge maildirs, and Backups are taking forever (just having to fstat >100k entries takes a while, then all the individual stuff for all the new messages...). Go dbox! :)
Kind regards,
Felix
-- Felix Schüren Head of NOC
Host Europe GmbH - http://www.hosteurope.de Welserstraße 14 - D-51149 Köln - Germany Telefon: (0800) 4 67 83 87 - Telefax: (01805) 66 32 33 HRB 28495 Amtsgericht Köln - UST ID DE187370678 Geschäftsführer: Uwe Braun - Alex Collins - Mark Joseph - Patrick Pulvermüller
participants (3)
-
Daniel L. Miller
-
Felix Schueren
-
Timo Sirainen