[Dovecot] CONDSTORE + QRESYNC extensions

[Timo Sirainen]

I wrote a mostly-working CONDSTORE support for Dovecot today. You can
try it with:

hg clone http://hg.dovecot.org/dovecot-condstore/

There are a few problems left:

 - STORE FLAGS.SILENT should be sending FETCH MODSEQ updates

 - When SELECTing a mailbox while CONDSTORE hasn't yet been enabled in
the session, Dovecot returns a higher HIGHESTMODSEQ than what's used
after CONDSTORE is enabled. This is because modseqs aren't tracked until
client has told us that it wants them. So Dovecot doesn't waste time
trying to figure out the correct HIGHESTMODSEQ.. Something probably
should be done about this.

 - STORE (UNCHANGEDSINCE xx) doesn't work correctly. The full fix would
require adding a new transaction log record type containing the modseq,
but currently it doesn't even bother refreshing the mailbox state before
checking what the modseqs are.

The modseq numbers are calculated as:

modseq = ((indexid + log_file_seq) << 32) | log_file_offset

Where:
 - indexid = A growing unique index file ID, usually the same as the
UNIX timestamp of index file creation.
 - log_file_seq = dovecot.index.log file sequence. When the .log is
rotated to .log.2 the new .log file's sequence is increased (so indexid
and log_file_seq uniquely identify a log file)
 - log_file_offset = offset in dovecot.index.log where the flag change
record was written.

The numbers are a bit large, but this way if index files are deleted the
modseqs always grow (so nothing breaks), because the newly created index
file will have a larger indexid.

The modseqs are also stored to dovecot.index file in "modseq" extension
records. The highest modseq is stored in header extension.

QRESYNC would require keeping track of expunges. This could be
implemented by just going through records in log files to find out what
messages have been expunged since the given modseq (= log seq+offset).
Although I'm not sure if it would be better to have a separate expunge
log. To allow rarely (e.g. once a week?) syncing clients the .log files
may have to be rotated less often, so they would take more space. Also
reading through them to find expunges would be slower than reading a
separate log having only expunges. But writing a separate expunge log
would add extra writes and there's no guarantee that it's ever even
read.

Other than that QRESYNC should be easy to implement.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20080313/5617beb4/attachment.bin