Just to expand and give some more information in response to Dean's
helpful message, in case it helps anyone else when replying to my
enquiry...
On 2 Mar 2007, at 16:28, Dean Brooks wrote:
On Fri, Mar 02, 2007 at 04:12:10PM +0000, Mike Brudenell wrote:
If your load balancer is set up to have persistent servers based upon user criteria if some sort, you could actually store the indexes on local drives on each machine. Worst case scenario, if user flipped to a different box in the cluster, Dovecot would have to rebuild its index increasing CPU and I/O. Best case, you see a performance gain by using local store and reducing NFS traffic.
I believe for IMAP sessions we only have the option of maintaining
persistence based on the IP address of the person making the request,
not on user credentials used for authentication etc.
However if we go down the "try to keep a user on their 'preferred'
IMAP server for every session" path I'm planning to use the load
balancer's fail-over facility...
Basically for each load-balanced service you can set up two pools of
servers. If ANY of the servers in Pool 1 is available they get used;
if none are then a server in Pool 2 is used.
We already give each user their own personal DNS name to access
'their' mail server: for example mine would be
pmb1.imap.york.ac.uk. This currently maps direct to my IMAP server:
imap0.york.ac.uk.
But what I am thinking of is pointing imap0.york.ac.uk at our load
balancer and setting this with the preferred IMAP server for me as
the only machine listed in Pool 1, but with all the other IMAP
servers in Pool 2. Thus whenever I connect I will get routed through
to to 'my' IMAP server. However if it is unavailable then I will get
routed to one of the others and persistence will then keep the rest
of my session there.
Q1. Would it be better to store the index files on NFS-shared filestore and direct users to any of the IMAP server machines? Or to store the index files on local disk and direct each user to their 'preferred'
IMAP server machine?Our plan is to store index files on local store and load balance to persistent servers. Sure, the persistent cache table expires over time, but then again, the indexes get out of date over time anyway.
Assuming our trick of implementing preferred servers works OK I'm
tempted to use local disk for the indexes too. I just need to get a
feel for how big these things grow. Even knowing roughly how much it
needs per message (bytes? Kbytes?) might give me a clue to start
with. Suggestions, anyone?
Q2. Does Dovecot (or "something") clean out old index files that
haven't been accessed for a while? Eg, when a user has temporarily come through on a different IMAP server to normal. Or do the index files sit there untouched for evermore?They sit untouched forever. Feel free to remove them after they
get to be of certain age.
So if we were to have a cron job scan and delete old ones (like we do
with /tmp now) we should be OK? There wouldn't be any Nasty Things
happen if we deleted an index file that turned out to still be in
use, even though it hadn't apparently been used for ages?
Q5. We have around 20,000 mail accounts and will therefore be seeing lots of concurrent IMAP sessions, usually secure (SSL) ones. I have seen mention that this can give rise to "Too many open files" errors under Solaris. How do we avoid this when we are likely to have several thousand concurrent IMAP sessions per server machine?
Yow. Thousands of concurrent IMAP sessions *per* server? All using SSL? With only 20,000 mail accounts? Are you sure about that? That seems like an awfully high active-reader ratio given the low number of accounts. Still, if true, it is what it is and needs to be
accomodated.
I've just checked and right at this moment we have between 600 and
800 IMAP server sessions running on each of our 5 servers. I'm
fairly certain that at our peak times we see 1,000-2,000 concurrently
on each server. I'll start eye-balling them over the next few weekdays.
The main point I was trying to make is that there would be
significantly more than the 256 that was mentioned in previous posts
on this subject.
I'm not sure if you are using a webmail client, such as Squirrelmail, but if so you may also want to consider running an IMAP proxy server to keep sessions open and persistent between page loads.
Thankfully this shouldn't be an issue for us. We use the University
of Cambridge's "Prayer" webmail software. Rather than being
something you run under a web server such as Apache it's actually a
custom-written HTTP to IMAP gateway and so gives a number of very
special benefits:
- Persistent browser-to-gateway sessions (where supported)
- Persistent gateway-to-imap-server sessions
- Aggressive caching
- GZipping of data en route to browser (benefits slow connections)
It is VERY speedy to use: a number of our users have been impressed,
even when working at the end of a slow dialup link around the world
from us in Australia.
Cheers, Mike B-)
-- The Computing Service, University of York, Heslington, York Yo10 5DD, UK Tel:+44-1904-433811 FAX:+44-1904-433740
- Unsolicited commercial e-mail is NOT welcome at this e-mail address. *