http://hg.dovecot.org/dovecot-dbox-redesign/
Looks like multi-dbox scales pretty nicely. Even after 100k messages the peak saved msgs/sec is the same as the initial saved msgs/sec, even if the average slows down somewhat.
I tested this by first deleting mailbox, then running "imaptest" for a second to get saving to start writing several fields to dovecot.index.cache file. Then ran imaptest for 60 seconds that did nothing but save messages using IMAP APPEND commands. The summary is that multi-dbox is almost 10x as fast as maildir when fsync is disabled and 2,5 faster with fsyncing:
fsync_disable=yes:
multi-dbox : ~1700 -> ~1500 msgs/sec, 90842 msgs/60 sec single-dbox: ~1700 -> ~1400 msgs/sec, 86149 msgs/60 sec maildir : ~330 -> ~110 msgs/sec, 9789 msgs/60 sec
fsync_disable=no:
multi-dbox : ~220 -> ~170 msgs/sec, 10113 msgs/60 sec maildir : ~100 -> ~70 msgs/sec, 4123 msgs/60 sec
Also I just noticed that saving messages with two clients gets a peak of 2200 msgs/sec, most likely because two CPU cores can be used then. Hmm. I guess if there were no locking it should have reached 3400 msgs/sec, but it's better than no performance increase.
And the raw data:
multi-dbox, fsync_disable=yes
imaptest clients=1 secs=1;imaptest clients=1 msgs=1000000 - logout=0 append=100,0 secs=60 Logi Sele Appe 100% 100% 100% 1 1 1759 1/ 1 0 0 1424 1/ 1 0 0 1734 1/ 1 ... 0 0 1701 1/ 1 0 0 1640 1/ 1 0 0 778 1/ 1 0 0 1305 1/ 1 0 0 1598 1/ 1
Totals: Logi Sele Appe 100% 100% 100% 1 1 90842
multi-dbox, fsync_disable=no
imaptest clients=1 secs=1;imaptest clients=1 msgs=1000000 - logout=0 append=100,0 secs=60 Logi Sele Appe 100% 100% 100% 1 1 209 1/ 1 0 0 227 1/ 1 0 0 208 1/ 1 ... 0 0 203 1/ 1 0 0 145 1/ 1 0 0 156 1/ 1 0 0 180 1/ 1 0 0 162 1/ 1
Totals: Logi Sele Appe 100% 100% 100% 1 1 10113
maildir, fsync_disable=yes
Logi Sele Appe 100% 100% 100% 1 1 331 1/ 1 0 0 311 1/ 1 0 0 288 1/ 1 ... 0 0 96 1/ 1 0 0 119 1/ 1 0 0 111 1/ 1
Totals: Logi Sele Appe 100% 100% 100% 1 1 9789
maildir, fsync_disable=no
Logi Sele Appe 100% 100% 100% 1 1 77 1/ 1 0 0 101 1/ 1 0 0 45 1/ 1 ... 0 0 52 1/ 1 0 0 67 1/ 1 0 0 89 1/ 1
Totals: Logi Sele Appe 100% 100% 100% 1 1 4123
single-dbox, fsync_disable=yes
imaptest clients=1 secs=1;imaptest clients=1 msgs=1000000 - logout=0 append=100,0 secs=60 Logi Sele Appe 100% 100% 100% 1 1 1765 1/ 1 0 0 1715 1/ 1 0 0 1764 1/ 1 ... 0 0 1722 1/ 1 0 0 1679 1/ 1 0 0 1678 1/ 1 0 0 828 1/ 1 0 0 1467 1/ 1 0 0 1469 1/ 1 0 0 1149 1/ 1 0 0 1135 1/ 1
Totals: Logi Sele Appe 100% 100% 100% 1 1 86149