Storing Messages in the cloud
A colleague asked me if it was possible for Dovecot to store messages in the cloud. I have not heard of it being done and therefore assume it is not possible, but I thought I would ask regardless. I tried checking on the “One Drive” forum; however, I have not received any response back yet.
Thanks!
-- Jerry
On Jul 9, 2018, at 9:33 AM, Jerry jerry@seibercom.net wrote:
A colleague asked me if it was possible for Dovecot to store messages in the cloud.
Does he have a more specific description of what he wants than “in the cloud”, or does he just like using buzzwords? From a user perspective, I would say that Dovecot, or any other IMAP server, already stores messages “in the cloud”. They are on a remote server, accessible from any location by any device that has a functional IMAP client.
On 07/10/2018 09:23 AM, dclist@list.jmatt.net wrote:
A colleague asked me if it was possible for Dovecot to store messages in the cloud.
Does he have a more specific description of what he wants than “in the cloud”, or does he just like using buzzwords? From a user perspective, I would say that Dovecot, or any other IMAP server, already stores messages “in the cloud”. They are on a remote server, accessible from any location by any device that has a functional IMAP client.
I'm glad someone else said it. ;) When I read the OP's message I just sat there shaking my head. As for his colleague, the local McDonald's is hiring.
-Dave-- Dave McGuire, AK4HZ New Kensington, PA
Quoting Dave McGuire mcguire@neurotica.com:
A colleague asked me if it was possible for Dovecot to store messages in the cloud.
Does he have a more specific description of what he wants than "in the cloud", or does he just like using buzzwords? - From a user
On 07/10/2018 09:23 AM, dclist@list.jmatt.net wrote: perspective,
I would say that Dovecot, or any other IMAP server, already stores messages "in the cloud". They are on a remote server, accessible from any location by any device that has a functional IMAP client.
I'm glad someone else said it. ;) When I read the OP's message I just sat there shaking my head. As for his colleague, the local McDonald's is hiring.
While I chuckled as well, I imagine what Jerry might be asking is how to use a remote file storage protocol or even a key-value storage backend, or something else that simply provides for remote back-end storage.
I suppose this is for those shops who do not want to spill any potentially confidential data - it's easy to store all of the data remotely with symmetric encryption so it's opaque to the remote provider - but do not or cannot deal with tasks and maintenance of backing up and maintaining the data and storage equipment to support its operations.
I also imagine the hard-working crew at Timo, Inc. have such things for sale to facilitate this with security and good performance, but they won't be inexpensive.
See: http://www.dovecot.fi/products/
"Dovecot Object Storage" is probably what you're asking about, but you know the drill with software that doesn't have displayed pricing: you probably cannot afford it.
=M=
Hi,
If you consider ceph as "the cloud", this could also apply:
https://github.com/ceph-dovecot/dovecot-ceph-plugin
MJ
On Wed, 11 Jul 2018, mj wrote:
If you consider ceph as "the cloud", this could also apply: https://github.com/ceph-dovecot/dovecot-ceph-plugin
Maybe the OP will find the following article interesting
http://www.admin-magazine.com/Archive/2015/26/Scalable-mail-storage-with-Dovecot-and-Amazon-S3Joseph Tam jtam.home@gmail.com
On July 10, 2018 at 12:37 PM "M. Balridge" dovecot@r.paypc.com wrote:
[snip]
I also imagine the hard-working crew at Timo, Inc. have such things for sale to facilitate this with security and good performance, but they won't be inexpensive.
More up-to-date link is: https://www.open-xchange.com/portfolio/dovecot-pro/
It's fairly easy to write support for storing message blobs in object storage. In fact, we are working on open sourcing the generic Dovecot object storage interface in the near future as part of core that would allow this kind of message storage.
But:
- if you have to store indexes in a separate block storage system, that kind of defeats the purpose of having a single storage layer and, alternately
- using object storage in a block-storage like mode for indexes does not scale at all - block storage access patterns do not work well at object storage latencies and I/O capacity.
There are several other components on top of generic object storage access that are used by Dovecot Pro to address these concerns. This allows Dovecot Pro's object storage support to store all data (messages, indexes, FTS) in a single object storage bucket while remaining performant and scalable.
These additional components are expensive to develop and maintain, and mainly useful for very large installations, so they will remain part of the commercial Dovecot Pro package when the basic object storage support is released.
michael
participants (7)
-
Dave McGuire
-
dclist＠list.jmatt.net
-
Jerry
-
Joseph Tam
-
M. Balridge
-
Michael Slusarz
-
mj