[dovecot] Re: Architectural questions

[Timo Sirainen]

On Tue, Oct 22, 2002 at 01:49:22PM +0200, Thomas Wouters wrote:
> Yes, the Alteons we use can be configured quite flexibly. We can easily
> configure, e.g., two servers as 'primary' and two fallback servers, or do
> load-balancing based on the output of a script, or any number of things. We
> only use the general load-balancing (actually just
> active-connection-balancing) while keeping sessions on the same server
> (based on remote IP) but we could look into the more intricate methods. A
> single IMAP server with a backup is a good start though.

OK, they'd probably be better than any Dovecot proxies I think.

> > Dovecot could actually do that itself too, authenticate user and then
> > either locally handle it or transfer it to another node based on some
> > configuration.
> 
> You mean if you have a frontend with several backends, and the frontend
> proxies for the backends (with several frontends possible, for redundancy,)
> hmm, that might work. Diablo (the news server software) works like this,
> somewhat, too, and we also use it behind Alteon switches :)

Not necessarily split to frontend/backend.. Well, that's possible too but I
was thinking that every running dovecot could handle authentication and
transferring connection elsewhere (via another TCP connection, login using
some internal password, maybe use TLS too).

> Yes. Keeping things on local disk sounds good. As long as opening the same
> mailbox on another server doesn't break anything (or breaks 'cleanly',
> doesn't delete the wrong mails etc) we can definately live with much worse
> performance for those cases. My professional estimate is that they will be
> very, very rare ;P

Nothing breaks if same mailbox is opened from different computers with
different indexes (or no indexes).

> But for the time being we're more concerned with the SORT extention :) I've
> read the spec, and besides the natural "ugh" at having to parse the subject
> that way, it seems doable...

The subject sorting looked very "ugh" to me too :)

> except that charset support for UTF-8, as well
> as US-ASCII, is mandatory. I don't know any libraries that convert to/from
> UTF-8 (though ASCII<->UTF-8 is obviously simple :) and though it's probably
> easy to roll your own for iso8859-1 I'm not sure if you had a solution in
> mind yet. Also, supporting the other character sets (like UW-IMAP does) is
> probably a lot trickier.

iconv() does it all, comes with glibc. Only bigger thing to do is to parse
the headers and convert the =?xxx?yyy?= things. I think everything should
go either through UTF8 or without any conversion if both header and search
charsets are same.

> Other than the charset issues I could probably whip up a working SORT
> implementation given enough time... but it probably wouldn't be
> super-efficient, as quicksort is so much easier to start with than, say,
> a mergesort ;P

I can think of two ways to do it:

1) save search results to array, sort the array, send it to clients, delete
the array.

2) sort all mails writing results into btree file, keep the file updated
whenever new mails are added or deleted. then do the search in that order so
we can just write out the results without any sorting.

I like the 2) more, but that works only if the sort condition isn't changed.
Or if it is changed, then we'd need to have multiple btree files.. And in
general it slows down things if sorting isn't done often.