Mike Brudenell wrote:
Our plan is to store index files on local store and load balance to persistent servers. Sure, the persistent cache table expires over time, but then again, the indexes get out of date over time anyway.
Assuming our trick of implementing preferred servers works OK I'm tempted to use local disk for the indexes too. I just need to get a feel for how big these things grow. Even knowing roughly how much it needs per message (bytes? Kbytes?) might give me a clue to start with. Suggestions, anyone?
We have indexes on a separate partition for our staff. It works out at 650GB folders + 90GB inboxes + 8.3GB indexes (both folders + inboxes). For one of our undergrad partitions it is 35GB folders + 150GB inboxes + 6.4GB indexes.
This is using mbox format, and I rather suspect most of the staff folders are accessed infrequently and may not even have any Dovecot indexes yet!
Yow. Thousands of concurrent IMAP sessions *per* server? All using SSL? With only 20,000 mail accounts? Are you sure about that? That seems like an awfully high active-reader ratio given the low number of accounts. Still, if true, it is what it is and needs to be accomodated.
I've just checked and right at this moment we have between 600 and 800 IMAP server sessions running on each of our 5 servers. I'm fairly certain that at our peak times we see 1,000-2,000 concurrently on each server. I'll start eye-balling them over the next few weekdays.
The main point I was trying to make is that there would be significantly more than the 256 that was mentioned in previous posts on this subject.
We use "login_process_per_connection = no" and change the number of file descriptors for the Dovecot master process with a "plimit -n 4096". You can also change the default in Solaris by modifying /etc/system (check the list archives for details - we discussed this recently).
I'm not sure if you are using a webmail client, such as Squirrelmail, but if so you may also want to consider running an IMAP proxy server to keep sessions open and persistent between page loads.
Thankfully this shouldn't be an issue for us. We use the University of Cambridge's "Prayer" webmail software.
We use Prayer too, but I considered using imapproxy (http://www.imapproxy.org) for, say, Squirrelmail. I'd have thought that Dovecot's indexes would make the proxy unnecessary, though, unless authentication is very expensive (and Dovecot can cache that too). imapproxy can't usefully cache SELECTs, so Dovecot would probably have to read the cache/index files for each connection.
We have NetApps too, and I've been thinking about moving the mail store to them (possibly converting to maildir on the way), but unfortunately, the Powers That Be have decided all staff are moving to Exchange (with message store on the NetApps via iSCSI) and the students may be moved to an outsourced solution!
Best Wishes, Chris
-- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094