[Dovecot] NoSQL Storage Backend

[Robert Brockway]

On Tue, 8 Feb 2011, Marten Lehmann wrote:

> The problem with traditional filesystems is, that they are very limited. You 
> cannot easily extend it, even with RAID 10 all messages may be lost due to a 
> RAID controller failure and just-in-time backups of a heavy loaded system 
> with consistently changing messages folders are virtually impossible.

Hi Marten.  There are many options that allow you to snapshop an active 
filesystem and guarantee a consistent backup.  Eg LVM on Linux or many 
other OSes. WAFL on a NetApp, etc.

In my experience a lot of high end mail systems store mail on the 
filesystem in Maildir format.

> Modern object stores like Scality, CouchDB or MongoDB can be very handy, as 
> they easily allow to keep an arbitrary amount of copies of each message on

This is certainly an interesting idea.

> different servers, you can simply add a storage node and new storage is 
> immediately available and mailboxes aren't bound to a certain server any 
> longer, but messages are evenly distributed across all storage nodes, thus 
> distributing load as well.

Being able to easily rebalance load would be useful.  I've known of 
several well known companies with a lot of mail users where mail was 
stored on filesystems across many systems and significant imbalances 
developed over time.  There was no easy way to rebalance them without 
causing an outage for at least some users.

> Isn't this option of big interest for every bigger mailbox provider? How hard 
> would it be to develop? Would it be possible to get this developed on a paid 
> basis by the dovecot core team?

In my experience OSS developers are quite open to being paid to add 
features :)

Cheers,

Rob

-- 
Email: robert at timetraveller.org		Linux counter ID #16440
IRC: Solver (OFTC & Freenode)
Web: http://www.practicalsysadmin.com
Contributing member of Software in the Public Interest (http://spi-inc.org/)
Open Source: The revolution that silently changed the world