[Dovecot] Scalability plans: Abstract out filesystem and make it someone else's problem

[Ed W]

>> Oh, have you considered some "optional" api calls in the storage API?
>> The logic might be to assume that someone wanted to do something
>> clever and split the message up in some way, eg store headers
>> separately to bodies or bodies carved up into mime parts.  The
>> motivation would be if there was a certain access pattern to optimise.
>> Eg for an SQL database it may well be sensible to split headers and
>> the message body in order to optimise searching - the current API may
>> not take advantage of that?  
>>     
>
> Well, files have paths. I think the storage backend can determine from
> that what type the data is. So if you're writing to mails/foo/bar/123 it
> means you're storing a message with ID 123 to mailbox "foo/bar". It
> could then internally parse the message and store its header/body/mime
> separately.
>   


I actually thought your idea of having a bunch of cut down IMAP type 
servers as the backend storage talking to a bunch of beefier frontend 
servers was quite an interesting idea!

Certainly though a simplification of the on-disk API would encourage new 
storage engines, so perhaps a three tier infrastructure is worth 
considering? (Frontend, intelligent backend, storage)

Ed W