[Dovecot] Newbie questions: Load-balanced Dovecot with NFS storage

[Mike Brudenell]

Just to expand and give some more information in response to Dean's  
helpful message, in case it helps anyone else when replying to my  
enquiry...


On 2 Mar 2007, at 16:28, Dean Brooks wrote:

> On Fri, Mar 02, 2007 at 04:12:10PM +0000, Mike Brudenell wrote:
>
> If your load balancer is set up to have persistent servers based upon
> user criteria if some sort, you could actually store the indexes on
> local drives on each machine.  Worst case scenario, if user flipped to
> a different box in the cluster, Dovecot would have to rebuild its
> index increasing CPU and I/O.  Best case, you see a performance gain
> by using local store and reducing NFS traffic.

I believe for IMAP sessions we only have the option of maintaining  
persistence based on the IP address of the person making the request,  
not on user credentials used for authentication etc.

However if we go down the "try to keep a user on their 'preferred'  
IMAP server for every session" path I'm planning to use the load  
balancer's fail-over facility...

Basically for each load-balanced service you can set up two pools of  
servers.  If ANY of the servers in Pool 1 is available they get used;  
if none are then a server in Pool 2 is used.

We already give each user their own personal DNS name to access  
'their' mail server:  for example mine would be  
pmb1.imap.york.ac.uk.  This currently maps direct to my IMAP server:  
imap0.york.ac.uk.

But what I am thinking of is pointing imap0.york.ac.uk at our load  
balancer and setting this with the preferred IMAP server for me as  
the only machine listed in Pool 1, but with all the other IMAP  
servers in Pool 2.  Thus whenever I connect I will get routed through  
to to 'my' IMAP server.  However if it is unavailable then I will get  
routed to one of the others and persistence will then keep the rest  
of my session there.


>> Q1. Would it be better to store the index files on NFS-shared
>> filestore and
>>     direct users to any of the IMAP server machines?  Or to store
>> the index
>>     files on local disk and direct each user to their 'preferred'  
>> IMAP
>>     server machine?
>
> Our plan is to store index files on local store and load balance to
> persistent servers.  Sure, the persistent cache table expires over
> time, but then again, the indexes get out of date over time anyway.

Assuming our trick of implementing preferred servers works OK I'm  
tempted to use local disk for the indexes too.  I just need to get a  
feel for how big these things grow.  Even knowing roughly how much it  
needs per message (bytes? Kbytes?) might give me a clue to start  
with.  Suggestions, anyone?


>> Q2. Does Dovecot (or "something") clean out old index files that  
>> haven't
>>     been accessed for a while?  Eg, when a user has temporarily come
>>     through on a different IMAP server to normal.  Or do the index
>> files
>>     sit there untouched for evermore?
>
> They sit untouched forever.  Feel free to remove them after they  
> get to
> be of certain age.

So if we were to have a cron job scan and delete old ones (like we do  
with /tmp now) we should be OK?  There wouldn't be any Nasty Things  
happen if we deleted an index file that turned out to still be in  
use, even though it hadn't apparently been used for ages?


>> Q5. We have around 20,000 mail accounts and will therefore be seeing
>> lots
>>     of concurrent IMAP sessions, usually secure (SSL) ones.  I have
>> seen
>>     mention that this can give rise to "Too many open files" errors
>> under
>>     Solaris.  How do we avoid this when we are likely to have several
>>     thousand concurrent IMAP sessions per server machine?
>
> Yow.  Thousands of concurrent IMAP sessions *per* server?  All using
> SSL?  With only 20,000 mail accounts?  Are you sure about that?  That
> seems like an awfully high active-reader ratio given the low number of
> accounts.  Still, if true, it is what it is and needs to be  
> accomodated.

I've just checked and right at this moment we have between 600 and  
800 IMAP server sessions running on each of our 5 servers.  I'm  
fairly certain that at our peak times we see 1,000-2,000 concurrently  
on each server.  I'll start eye-balling them over the next few weekdays.

The main point I was trying to make is that there would be  
significantly more than the 256 that was mentioned in previous posts  
on this subject.


> I'm not sure if you are using a webmail client, such as Squirrelmail,
> but if so you may also want to consider running an IMAP proxy server
> to keep sessions open and persistent between page loads.

Thankfully this shouldn't be an issue for us.  We use the University  
of Cambridge's "Prayer" webmail software.  Rather than being  
something you run under a web server such as Apache it's actually a  
custom-written HTTP to IMAP gateway and so gives a number of very  
special benefits:

   * Persistent browser-to-gateway sessions (where supported)
   * Persistent gateway-to-imap-server sessions
   * Aggressive caching
   * GZipping of data en route to browser (benefits slow connections)

It is VERY speedy to use: a number of our users have been impressed,  
even when working at the end of a slow dialup link around the world  
from us in Australia.

Cheers,
Mike B-)

-- 
The Computing Service, University of York, Heslington, York Yo10 5DD, UK
Tel:+44-1904-433811  FAX:+44-1904-433740

* Unsolicited commercial e-mail is NOT welcome at this e-mail address. *