[Dovecot] key -> object mailstore

[Damien Churchill]

On 14 September 2012 15:59, Timo Sirainen <tss at iki.fi> wrote:
> On 14.9.2012, at 17.46, Damien Churchill wrote:
>
>> I was wondering what would be entailed in modifying dovecot 2.2 to
>> support storing mail in an object store. I've seen a few mails dotted
>> around in the ML history about supporting such a thing and seen it's
>> basically dependant upon some changes in lib-storage to support
>> writing messages without locking. Is this still the case?
>
> I've a whole new design for it and I was planning on implementing it for v2.2. Do you want to help coding it? :) Which storage would you want to use?
>

That's good to hear :) I've been evaluating RADOS as an object store,
which is similar to S3. Although any distributed storage would be
great. I'd be more than happy to help code it!

> The generic idea is:
>  - only one server accesses one user simultaneously
>  - index files are copied from object storage to local filesystem and accessed there, once in a while uploaded back to object storage
>  - if user is accessed from two servers because of some bug/split brain/something, the changes are merged using dsync
>  - support high latency: asynchronous reads/writes. prefetch mail bodies.
>

I'm assuming that the director would be used in order to distribute
connections to the same server, so it's only within a local instance
of dovecot you'd need to be aware of what currently has a connection
open for that user?

How are you planning on handling the situation where say node X dies
and hasn't uploaded the latest index file? Would that result in
missing messages from the mailbox when accessed by another node, or is
the local index intended to be more of a write-through cache?