On Mon, 2009-09-28 at 18:35 +0100, Ed W wrote:
I would have thought that your API will prefer to request message parts where it can (eg header, body, mime part), and just issue a read_bytes, where that's what the client is asking for otherwise. This would allow the storage engine to optimise where it can and sadly for the dumb client we just stream bytes since that's all they asked for...
In my mind this is more about what lib-storage API was supposed to abstract out, whereas my filesystem API would be used simply for binary data storage. The same FS API could be used to store both dbox files and index files.
Perhaps the API should also request specific headers from the storage engine where possible and ask for all headers only where it's necessary? This would allow an sql database to be heavily normalised (I'm sure performance is iffy, but we have to pre-suppose some reason why this design is useful for other reasons)
This is really going towards what lib-storage API is supposed to do already.. It's not even horribly difficult to write a new backend for it. For example in v2.0 the fully functional Cydir backend code looks like:
% wc *[ch] 152 357 3740 cydir-mail.c 319 783 8420 cydir-save.c 402 1087 10806 cydir-storage.c 35 82 1085 cydir-storage.h 187 465 4798 cydir-sync.c 24 54 615 cydir-sync.h 1119 2828 29464 total
There is still a bit of code duplication between backends that could reduce the line count by maybe 100-200 lines. Anyway I think the only good way to implement support for normalized SQL database in Dovecot would be to implement a new lib-storage backend, and it shouldn't be a hugely difficult job.