Storing Messages in the cloud

[Michael Slusarz]

> On July 10, 2018 at 12:37 PM "M. Balridge" <dovecot at r.paypc.com> wrote:

[snip]

> I also imagine the hard-working crew at Timo, Inc. have such things for sale
> to facilitate this with security and good performance, but they won't be
> inexpensive.
> 
> See: http://www.dovecot.fi/products/

More up-to-date link is:
https://www.open-xchange.com/portfolio/dovecot-pro/

It's fairly easy to write support for storing message blobs in object storage.  In fact, we are working on open sourcing the generic Dovecot object storage interface in the near future as part of core that would allow this kind of message storage.

But:

1) if you have to store indexes in a separate block storage system, that kind of defeats the purpose of having a single storage layer and, alternately
2) using object storage in a block-storage like mode for indexes does not scale at all - block storage access patterns do not work well at object storage latencies and I/O capacity.

There are several other components on top of generic object storage access that are used by Dovecot Pro to address these concerns.  This allows Dovecot Pro's object storage support to store all data (messages, indexes, FTS) in a single object storage bucket while remaining performant and scalable.

These additional components are expensive to develop and maintain, and mainly useful for very large installations, so they will remain part of the commercial Dovecot Pro package when the basic object storage support is released.

michael