[Dovecot] v3.0 architecture

[pod]

Timo Sirainen <tss at iki.fi> writes:

> The big problem is what the protocol should be. Use some existing RPC
> protocol? It should be something extensible so that a plugin in imap
> process can talk to a plugin in storage process, without the base
> processes knowing anything about the details (e.g. imap-quota plugin
> asking quota usage from storage's quota plugin). In any case the  client
> side API should be asynchronous. That can make it annoyingly  difficult to
> use though. Wonder if I could switch to erlang or  something for the
> imap/pop3 processes :)

I realise I'm coming a little late to this conversation.  Various people
have suggested various RPC mechanisms.  Might I humbly suggest also
looking at 9P (or 9P2000) the Plan9 filesystem protocol. [1]

The general design philosophy in Plan9 is not to use an increasing number
of application specific RPC mechanisms but instead to implement everything
in a synthetic filesystem tree.  The 9P protocol handles the basic
operations required to navigate the filesystem tree and read/write to the
files within.  The semantics of "RPC" are then relegated to the
specification for the layout of the filesystem and meaning assigned to
reading or writing to files within that tree.  This allows for
extensibility (and incidently discoverability) - new features or "plugins"
appear as a new file or directory hierarchy in the presented filesystem
layout.

As a concrete example the so called upas/fs [2] presents a view of a
mailbox independent of the actual storage mechanism used for the data
contained in the mailbox.  From the manpage

    The mailbox itself becomes a directory under /mail/fs. Each message in
    the mailbox becomes a numbered directory in the mailbox directory, and
    each attachment becomes a numbered directory in the message
    directory. Since an attachment may itself be a mail message, this
    structure can recurse ad nauseam.

See [2] for more details.  I'm not suggesting this layout would be exactly
right for the Dovecot storage protocol but it may provide a starting
point.

It's probably worth noting the irony, given this is a maillist about
Dovecot, in that this approach almost obsoletes the need for an IMAP
protocol in the first place (the upas/fs style layout as documented
doesn't really provide sufficient support for server side search for
example).

Footnotes: 
[1]  http://plan9.bell-labs.com/magic/man2html/5/0intro
[2]  http://plan9.bell-labs.com/magic/man2html/4/upasfs