[Dovecot] Scalability plans: Abstract out filesystem and make it someone else's problem

[Seth Mattinen]

Timo Sirainen wrote:
> On Mon, 2009-08-10 at 14:33 -0700, Seth Mattinen wrote:
>> Timo Sirainen wrote:
>>> This is something I figured out a few months ago, mainly because this
>>> one guy at work (hi, Stu) kept telling me my multi-master replication
>>> plan sucked and we should use some existing scalable database. (I guess
>>> it didn't go exactly like that, but that's the result anyway.)
>>>
>> Ick, some people (myself included) hate the idea of storing mail in a 
>> database versus simple and almost impossible to screw up plain text 
>> files of maildir. 
> 
> Nothing forces you to switch from maildir, if you're happy with it :)
> But if you want to support millions of users, it's simpler to distribute
> the storage and disk I/O evenly across hundreds of servers using a
> database that was designed for it. And by databases I mean here some of
> those key/value-like databases, not SQL. (What's a good collective name
> for those dbs anyway? BASE and NoSQL are a couple names I've seen.)
> 


Why is a database a better choice than a clustered filesystem? It seems
that you're adding a huge layer of complexity (a database) for something
that's already solved (clusters). Queue directories and clusters don't
mix well, but a read-heavy maildir/dbox environment shouldn't suffer the
same problem.

~Seth