On Fri, Mar 02, 2007 at 04:12:10PM +0000, Mike Brudenell wrote:
- Because sessions could be on different servers memory mapping of
index files doesn't work well with NFS, so set mmap_disable=yes in dovecot.conf.
If your load balancer is set up to have persistent servers based upon user criteria if some sort, you could actually store the indexes on local drives on each machine. Worst case scenario, if user flipped to a different box in the cluster, Dovecot would have to rebuild its index increasing CPU and I/O. Best case, you see a performance gain by using local store and reducing NFS traffic.
- Dovecot relies on the mtime timestamp of mailbox files so the NFS-mounted mailstore needs to be mounted with these options on each server: actimeo=0 (Are there any other mount options we should use too?)
This is absolutely critical, yes. Without this, all sorts of nasty things can happen. You can also use the "noac" attribute, at least as of Solaris 8.
- We will be using filesystem quotas on users mailboxes. We understand that Dovecot's index files are best on no-quota filestores so
will store these separately.
The only issue here is dotlocking. If your MTA or if Dovecot uses dotlocks of any kind, you'll need to be sure those are also outside of your quota-restricted filestore. Otherwise, you are going to need to be sure that the filesystem quota is never actually reached, and that the limit is artificially set lower than the filesystem quota in some other manner.
Q1. Would it be better to store the index files on NFS-shared
filestore and direct users to any of the IMAP server machines? Or to store
the index files on local disk and direct each user to their 'preferred' IMAP server machine?
Our plan is to store index files on local store and load balance to persistent servers. Sure, the persistent cache table expires over time, but then again, the indexes get out of date over time anyway.
Q2. Does Dovecot (or "something") clean out old index files that haven't been accessed for a while? Eg, when a user has temporarily come through on a different IMAP server to normal. Or do the index
files sit there untouched for evermore?
They sit untouched forever. Feel free to remove them after they get to be of certain age.
Q3. Storing the index files on the NetApp filer would give us the
ability to share them between servers and grow their volume as need be,
but at the cost of performance. How big do the index files get? Are they typically a few kBytes per message? Per mailbox? Per user? Or more?
I considered this, but my concern is reliability. Dovecot's index files seem a bit "delicate" in recent patches and I'm afraid of possible issues of sharing them between servers, especially if there are multiple IMAP sessions open on different servers. I'm not familiar with ultimate size at this point, but it depends on the maximum size of the mailboxes and folders I suppose.
Q4. We will be using Exim as the MTA, which can deliver direct to
Maildir mailboxes. However I understand that Dovecot's "deliver" LDA
adds the benefit of updating the index files as each message is
delivered. Is this a significant gain? Or is there little difference in
actual use?
No experience with the LDA component.
Q5. We have around 20,000 mail accounts and will therefore be seeing
lots of concurrent IMAP sessions, usually secure (SSL) ones. I have
seen mention that this can give rise to "Too many open files" errors
under Solaris. How do we avoid this when we are likely to have several thousand concurrent IMAP sessions per server machine?
Yow. Thousands of concurrent IMAP sessions *per* server? All using SSL? With only 20,000 mail accounts? Are you sure about that? That seems like an awfully high active-reader ratio given the low number of accounts. Still, if true, it is what it is and needs to be accomodated.
Obviously increasing the number of systems in your cluster is one way to fight it. I know there was a recent bug in Dovecot that was causing file descriptor leaks, but if I recall it was fixed in a recent patch.
Still, if you are going to have thousands of concurrent IMAP sessions, I would consider making sure you have a good number of systems in your cluster.
I'm not sure if you are using a webmail client, such as Squirrelmail, but if so you may also want to consider running an IMAP proxy server to keep sessions open and persistent between page loads.
-- Dean Brooks dean@iglou.com