On 22.12.2006, at 9.14, Steffen Kaiser wrote:
One optimization left for it would be to not store flags and
keywords to the dbox files at all, but keep everything in index files. Once I get that implemented, I'll start benchmarking it. Of course the
problem with that is that it relies on index files completely then.IMHO this is not good for disaster recovery and makes life
needlessly hard in such case. Esp. when both files become out-of- sync, you need a cunning tool to re-construct any good state,
rather than a simple "REINDEX". This should apply to each kind of
"index" file. Also, consider upgrades to newer index file formats
etc.pp.
I was thinking that before doing this, the index file handling should
be hardened more. So that if it detects a corruption, it tries really
hard to preserve the flags and write a new valid index file, instead
of just rebuilding it like it does now.
Are there really that many writes of message attributes? I mean,
there is only an improvement because you have just one write into
the indexes rather than two, one to the indexes and one into the
message.
And with some updates to index file code, it would eventually be
possible to just write to dovecot.index.log file and update
dovecot.index only once in a while. This way the disk writes would be
really minimal, since it only needs to writes to a single file.
Anyway, this would be optional, so that you could decide if the
performance increase is worth the potential problems. Also it would
be possible to still update the flags once in a while, so that if the
indexes get trashed you'd lose only the last day's (or so) flag changes.
I don't know how much of a difference it would make in real world,
but anything that reduces disk I/O is good :)
You've wrote that you reserve space for the flags in mbox, in case
they get updated, so you reduce the probability to rewrite the
whole mbox for each flag. Doing so should apply to dbox as well?
Yes, there's space reserved for flags and keywords in dbox files. It
doesn't even support moving data around like mbox does.