I cleaned up the lib-storage code a bit so that it's easier to write support for new mailbox formats. It could still use a bit of cleaning up.
Anyway, I thought I'd try how easy it would be to implement Cyrus-like mail store which would consist of only Dovecot's index files and "<uid>." message files. Flags and everything else is kept only in Dovecot's index files (so it might not be the best idea to use it in production just yet). Several hours later I ended up having it working. The code contains:
~/cvs/m/dovecot-head/src/lib-storage/index/uiddir% wc *[ch] 144 329 3552 uiddir-mail.c 240 565 6206 uiddir-save.c 532 1367 13628 uiddir-storage.c 55 126 1712 uiddir-storage.h 169 404 4057 uiddir-sync.c 21 48 573 uiddir-sync.h 103 258 2762 uiddir-transaction.c 1264 3097 32490 total
uiddir-storage.c is still a bit too bloaty with lots of copy&pasted code. The same with maildir and mbox:
maildir: 6022 18232 165605 total mbox: 7859 23283 213686 total
Benchmarks show that with small mailboxes it's a lot faster than maildir, and with larger mailboxes it's a bit faster.
I also tested it against Cyrus. I found out that fetching ENVELOPE vs. specific headers gives quite a huge performance difference. Cyrus is optimized for ENVELOPE while Dovecot is optimized for specific headers. With larger mailboxes + ENVELOPE Cyrus is faster than Dovecot, while with specific headers Dovecot is always faster. Maybe I'll some day make Dovecot optimize for ENVELOPE also if it sees that client only wants to use it.
BTW. Does anyone have better naming ideas than "uiddir"? I'll commit it to CVS HEAD once I'm sure about the name. :)
u: Dovecot CVS HEAD + uiddir m: Dovecot CVS HEAD + Maildir c: Cyrus IMAP 2.2.13
./imaptest list=0 seed=1 secs=60 clients=10
Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 23486 11822 23481 23476 33426 11659 8038 23472 11922 46952 m: 19367 9734 19367 19356 27453 9687 6287 19347 7983 38709 c: 5124 2553 5121 5105 7080 2559 1877 5104 3271 10228
./imaptest list=0 dele=20 expu=20 seed=1 secs=60 msgs=10000 clients=10
Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 5291 2704 5289 5278 7448 2706 1048 1061 5283 10563 m: 4162 2103 4162 4153 5977 2124 762 853 4153 8304 c: 4302 2157 4301 4289 6021 2072 806 879 4292 8584
./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=10
Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 3211 1637 3206 3195 4639 1537 287 302 3202 6402 m: 2957 1508 2955 2944 4173 1440 279 306 2948 5894 c: 3933 1929 3933 3920 5604 2020 380 379 3923 7846
./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=1 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 1529 745 1529 1528 2184 819 136 162 1528 3056 c: 1579 813 1579 1578 2257 811 171 148 1578 3156
./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=50 Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 3098 1552 3093 3085 4432 1520 293 334 3089 6173 c: 3664 1765 3659 3596 5106 1802 365 364 3614 7228
======
FETCH BODY.PEEK[HEADER.FIELDS(from to cc subject)] instead of ENVELOPE:
./imaptest list=0 dele=20 expu=20 seed=1 secs=60 msgs=10000 clients=10
Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 5777 2933 5777 5767 8215 2901 1133 1191 5767 11531 c: 3636 1794 3635 3623 5196 1843 682 722 3627 7254
./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=10
Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 3869 1959 3868 3861 5452 1943 364 394 3859 7717 c: 3191 1620 3189 3184 4501 1598 315 327 3181 6362
./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=1
Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 1912 966 1912 1910 2783 940 184 197 1911 3820 c: 1342 696 1342 1341 1888 682 109 126 1341 2682
./imaptest list=0 dele=10 expu=10 seed=1 secs=60 msgs=100000 clients=50
Logi Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo u: 4618 2320 4610 4604 6558 2266 452 455 4605 9205 c: 2960 1472 2956 2941 4134 1425 281 312 2910 5820