[Dovecot] Dovecot performance under high load (vs. Courier)

[Michael M Slusarz]

Quoting Timo Sirainen <tss at iki.fi>:

> Well, I had completely forgotten about it :) Reading my old mail:
>
>> There isn't a whole lot of state to be saved really. Mailbox GUID,  
>> UIDVALIDITY,
>> HIGHESTMODSEQ gives the mailbox state. Then you have the  
>> language/etc. states.
>> Clients could restore their earlier state from days ago, as long as Dovecot
>> still has the necessary .log records available (similar to how  
>> QRESYNC works).
>
> Yeah .. Perhaps something like:
>
> 1. if client issues LOGOUT XSTATE
>
> 2. And server sees that it can actually save all of the state (some  
> things are a bit tricky, and probably not worth the trouble in  
> initial implementation)
>
> 3. Then the server server sends
> * OK XSTATE <string>
> * BYE

This makes sense. Although wouldn't it be:

* OK [XSTATE <string>] State saved.

> 4. The client can pipeline after LOGIN/AUTHENTICATE:
> a XSTATERESTORE <string>
> a OK Yeah!
> or
> a NO Not gonna work.

Couple of suggestions here:

1) Maybe allow XSTATERESTORE to be sent BEFORE authentication  
also/instead?  The way that Dovecot would restore state might be  
different from the way another IMAP server would restore state.  It's  
possible that another server could optimize things if, at  
authentication time, it knew it was going to restore state.

i.e.:

a XSTATERESTORE <string>
a OK Will attempt to restore state.
b (LOGIN/AUTHENTICATE command)
* OK [XSTATERESTOREOK] State restored.
   -- or --
* OK [XSTATERESTORENO] State NOT restored.
b OK Logged in.

2) Could extend LOGIN/AUTHENTICATE to accept XSTATERESTORE parameter.
Pros: saves round-trip.
Cons: extending LOGIN/AUTHENTICATE at this stage of IMAP 4 development  
is probably overkill (Although this implementation already requires  
extending the LOGOUT command)

> Perhaps even a real RFC for this thing? .. If it's worth it.. Would  
> save at least a few X bytes from network traffic :)

It could potentially be a few more than X bytes.  Here's an extreme  
example of the potential savings:

Initial connection:
-------------------
1 (LOGIN/AUTHENTICATE)
1 OK Logged in.
2 CAPABILITY
* CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE  
SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT  
CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC  
ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE  
ACL RIGHTS=texk
2 OK Capability completed.
3 ID ("name" "foo" "version" "1.0")
* ID ("name" "foo2" "version" "bar2" "os" "linux")
3 OK ID completed
4 ENABLE QRESYNC
* ENABLED QRESYNC
4 OK Enabled.
5 COMPARATOR "de;*" i;basic
* COMPARATOR i;basic
5 OK Will use i;basic for collation
6 LANGUAGE DE
* LANGUAGE (DE)
* NAMESPACE (("" "/")) (("Other Users/" "/" "TRANSLATION" ("Andere  
Ben&APw-tzer/"))) (("Public Folders/" "/" "TRANSLATION" ("Gemeinsame  
Postf&AM8-cher/")))
6 OK Sprachwechsel durch LANGUAGE-Befehl ausgefuehrt
[IMAP session]
50 LOGOUT XSTATE
* OK [XSTATE 123abc]
* BYE

Subsequent connection:
----------------------
1 XSTATERESTORE 123abc
1 OK Will attempt to restore state.
2 (LOGIN/AUTHENTICATE)
* OK [XSTATERESTOREOK] State restored.
2 OK Angemeldet.

Given this (admittedly) extreme example, the savings are 689 bytes  
(+126 bytes for staterestore overhead, -815 bytes for state setup).   
Additionally, the server/client have to process 4 less IMAP commands.   
This is a significant savings IMHO.

Whether or not this is appropriate for a real RFC, it would probably  
be useful to document in RFC fashion regardless.

michael