Timo Sirainen wrote:
On Aug 11, 2009, at 12:41 AM, Seth Mattinen wrote:
Nothing forces you to switch from maildir, if you're happy with it :) But if you want to support millions of users, it's simpler to distribute the storage and disk I/O evenly across hundreds of servers using a database that was designed for it. And by databases I mean here some of those key/value-like databases, not SQL. (What's a good collective name for those dbs anyway? BASE and NoSQL are a couple names I've seen.)
Why is a database a better choice than a clustered filesystem?
Show me a clustered filesystem that can guarantee that each file is stored in at least 3 different data centers and can scale linearly by simply adding more servers (let's say at least up to thousands).
Easy, AFS. It is known to support tens of thousands of clients [1] and it's not exactly new. Like supporting the quirks of NFS, the quirks of a clustered filesystem could be found and dealt with, too.
Key/value databases are hardly a magic bullet for redundancy. You don't get 3 copies in different datacenters by simply switching to a database-style storage.
[1] http://www-conf.slac.stanford.edu/AFSBestPractices/Slides/MorganStanley.pdf
Clustered filesystems are also complex. They're much more complex than what Dovecot really requires.
I mention it because you stated wanting to outsource the storage portion. The complexity of whatever database engine you choose or supporting a clustered filesystem (like NFS) is a wash since you're not maintaining either one personally.
~Seth