[Dovecot] Solr clusters

[Timo Sirainen]

On 7.11.2013, at 5.36, Timo Sirainen <tss at iki.fi> wrote:

> Has anyone implemented larger Dovecot+Solr clusters and would be willing to give some details about how it works for you? My understanding about it so far is:
> 
> - SolrCloud isn’t usable with Dovecot. Replication isn’t useful, because nobody wants to pay for double the disk space for indexes that could be regenerated anyway. The autosharding isn’t very useful also, because: I think the shard keys could be created in two possible ways: a) Mails would be entirely randomly distributed across the cluster. This would make updates efficient, because the writes would be fully distributed across all servers. But I think it would also make reads somewhat inefficient, since all the servers would have to be searched and the results combined. Also if a server is lost, there’s no easy way to reindex back the missing data, because it would contain a piece of pretty much all the users’ data. b) Shard keys could be created so that the same user would typically go only to 1-2 servers. It would be possible (at least in theory) to find a broken server’s list of users and reindex only their data, but I’m not sure if this method is any easier than the non-SolrCloud setup.

Of course, if you do use NFS (or other shared filesystem) and you trust it enough not to easily lose all of the data, you don’t need replication and the auto-sharding could be useful after all..