Quoting Timo Sirainen tss@iki.fi:

On Thu, 2011-06-09 at 14:45 +0200, Vincent Richomme wrote:

...

Hi,

I plan to develop a new web service where one of its module is a webmail and I am thinking about some implementation details. From a interface point of view I wanted to adopt the same logic a outlook(desktop) or yahoo webmail and not split emails into pages. I mean I just want a single datatable view where user can scroll to see his old messages and data are updated only when user release the vertical scrollbar.

We have implemented this in IMP. Best of luck - it is not an easy
thing to implement correctly.

...

When I look at current webmail (roundcube for instance) I can see that a cache is used to stored emails headers and I was wondering if it was really necessary ?

You mean Roundcube's own local cache?

For webmail, our theory is that caching is pretty much only important
when viewing the mailbox list. There is an expectation from a user
that when viewing a message, it may take a bit of time to grab the
data and render. Caching body data and full header text for all users
and all messages will instead most likely cause your cache to fill to
quickly, and provides little practical benefit.

So instead focus on caching for the mailbox list. We cache all
envelope data, imapdate, size, and flags. We also cache *specific*
headers that are unobtainable from envelope data, but are used when
rendering the message list (e.g. X-Priority, List Headers). But there
is no need to cache the entire header text - some of it would be
duplicative (e.g. envelope headers), and a good chunk of the remaining
data is worthless for most use cases (e.g. Received).

Finally, we cache flags. But note: you absolutely MUST implement
CONDSTORE support on your client or else mailbox caching is
essentially worthless. A client that has to grab the flag data on
every connection pretty much eliminates all benefits gained from
caching in the first place. Most other webmail implementations claim
they cache, but they have no CONDSTORE support, so their caching is
either broken (flag changes from other clients don't appear) or of
limited value (since an IMAP server may need to parse through the
entire list of requested messages to grab this information, which may
be a slow operation if using something like mboxes on the storage side).

...

Once a user has passed the login process I would like to retrieve only the emails that will be displayed (actually a bit more, I am thinking of 150) but I am wondering if it's a good idea to not use a cache for headers. For instance if I have thousands of users on the same machine, will it support it ? Will it be fast enough ? Of course I will try by myself the different options but would be curious to have some opinions.

Dovecot's cache is also pretty fast. But then again it is easier to scale web servers than IMAP servers by just adding more servers.

I'll admit that a webmail cache is not as important when using with a
server that already supports caching natively (Dovecot, Cyrus).
However, you still gain benefits because the local cache is in a
format that is directly usable by the client program - in other words,
you save on reparsing the IMAP data -> local data structure.

But a webmail cache is a necessity when using IMAP servers that don't
natively cache (e.g. Courier).

michael