[Dovecot] RFC: Storing indexes in a (remote) database to increase dovecot scalability

[Timo Sirainen]

I just wrote a "Scalability plans" mail, which explains what I've been
recently thinking about. I think that would solve most of your problems.

On Mon, 2009-08-10 at 11:20 +0100, Mark Zealey wrote:
> 1) One major issue we have with dovecot is the number of iops it
> requires to keep its files in sync on our big disk arrays. If this were
> a mysql database, we'd have better ways of keeping these in sync, for
> example switching the speed at which it flushes the updates to disk,
> growth using partitioning/sharding, read-only slave instances etc,
> specific ssd storage for indexes/caches. I know that we could create a
> separate ssd disk array and export that over nfs too, but see issue (2)
> below. If indexes were stored in a mysql database we could choose to
> have a different reliability for the indexes (ie if the database server
> crashes we have 1 second of index changes lost, whereas for the email
> itself we can ensure it is always kept in sync on our other disk
> arrays).

Seems pretty complex to me, but it should be possible to implement MySQL
FS backend and keep indexes/mails there. They would anyway pretty much
have to be just blobs, but I guess that's not an issue for you?

> 2) nfs locking issues. No matter how hard we try, we always find there
> are some issues with concurrency over nfs. 

Solved again.

> 3) As we store all of our mail systems over nfs, effectively each export
> is a single-threaded connection to the server. No matter how fast the
> central nfs server you can realistically not go above 10k nfs ops/sec on
> the fastest of local networks due to latency issues. With mysql you
> could have multiple parallel connections to the central database store
> (1 per process?) which would mean more nfs ops available for actually
> serving the files.

lib-storage parallelization with async I/O backend should also solve
this. Even when using mysql the lib-storage parallelization would be
required to take advantage of multiple connections.

> 4) Caching. Much more tuneable on a database server than in a
> filesystem. Eg in mysql there are loads of options to tune the various
> innodb/query/key caches, on nfs there are very few even on advanced
> netapps etc

I'm not sure about this. I think the in-memory index cache would solve
the worst problem.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://dovecot.org/pipermail/dovecot/attachments/20090810/d3211758/attachment.bin