[Dovecot] v3.0 architecture
I just had a thought. It's probably going to be too big of a change
for v2.0, so something to think about for v3.0:
Supporting multiple UIDs is difficult. Currently expire-tool and
v2.0's LMTP server are implemented in a way that it's always running
as root and only temporarily dropping privileges. So any security hole
in there gives the attacker root privileges. Not very nice. With a
single UID it's possible (or will be) to just use that one UID for the
process without running as root.
User sharing mailboxes to another user with different UID is even more
difficult. Currently the only practical way to implement this is to
assign read-write permissions to some group and have all mail files
owned by that group. Not very nice.
Sharing mailboxes between users located in different servers is
impossible if the disk isn't somehow shared.
All of these can be solved the same way:
- imap, pop3, lmtp, etc. processes will be running without any
privileges, very much like imap/pop3-login nowadays. Nobody-chrooted
to empty directory.
- Create storage processes that are started as needed. They can be
configured in different security/performance settings: One storage
process per user, one storage process per UID, setuid-root style
shared storage process for multiple UIDs.
- Mail processes talk to storage processes via some protocol. They
can talk via UNIX socket or TCP/IP. If an existing connection can't
handle the target UID, a new connection is made that either reuses an
existing storage proces or creates a new one.
- IMAP can finally implement truly simultaneously running queries
(where it makes sense) by creating more storage processes if existing
ones are busy handling longer-running commands. This also allows using
the same storage process for multiple simultaneous IMAP connections
and mail deliveries (and etc). The storage processes can also hang
around and keep mailboxes open after all connections have closed it
(dynamically figured out when it makes sense) to handle webmail
clients that create lots of short-living connections or POP3 clients
who poll for new mail every minute.
The big problem is what the protocol should be. Use some existing RPC
protocol? It should be something extensible so that a plugin in imap
process can talk to a plugin in storage process, without the base
processes knowing anything about the details (e.g. imap-quota plugin
asking quota usage from storage's quota plugin). In any case the
client side API should be asynchronous. That can make it annoyingly
difficult to use though. Wonder if I could switch to erlang or
something for the imap/pop3 processes :)
Timo Sirainen schrieb:
- Mail processes talk to storage processes via some protocol. They
can talk via UNIX socket or TCP/IP. If an existing connection can't handle the target UID, a new connection is made that either reuses an existing storage proces or creates a new one.
This sounds really, really cool! I am thinking of the possibilities of separating storage to frontend. That way one could build dynamically distributing mailboxes. Use local ressources of many storage boxes instead of using FC storages all the way. Or just "replicate" mails twice to have them redundantly stored (hadn't you the idea of a replication part, anyway?).
The big problem is what the protocol should be. Use some existing RPC protocol? It should be something extensible so that a plugin in imap process can talk to a plugin in storage process, without the base processes knowing anything about the details (e.g. imap-quota plugin asking quota usage from storage's quota plugin). In any case the client side API should be asynchronous. That can make it annoyingly difficult to use though. Wonder if I could switch to erlang or something for the imap/pop3 processes :)
Maybe one could "lend" some parts of erlang based database http://couchdb.apache.org/ ... :)
Regards, Sebastian
The big problem is what the protocol should be. Use some existing RPC protocol? It should be something extensible so that a plugin in imap process can talk to a plugin in storage process, without the base processes knowing anything about the details (e.g. imap-quota plugin asking quota usage from storage's quota plugin).
Googles Protocol Buffers offer both flexibility and speed.
On 147, 05 27, 2009 at 01:31:41PM +0400, Max Ivanov wrote:
The big problem is what the protocol should be. Use some existing RPC protocol? It should be something extensible so that a plugin in imap process can talk to a plugin in storage process, without the base processes knowing anything about the details (e.g. imap-quota plugin asking quota usage from storage's quota plugin).
Googles Protocol Buffers offer both flexibility and speed.
"Protocol buffers are Google's ... blah-blah-blah ... using a variety of languages - Java, C++, or Python."
I can't find good old plain C in this "variety of languages" :(
"Protocol buffers are Google's ... blah-blah-blah ... using a variety of languages - Java, C++, or Python."
I can't find good old plain C in this "variety of languages" :(
Protocol buffers is flexible message format specification, there are plenty implementations of it , including C based: http://code.google.com/p/protobuf-c/
The big problem is what the protocol should be. Use some existing
RPC protocol? It should be something extensible so that a plugin in
imap process can talk to a plugin in storage process, without the
base processes knowing anything about the details (e.g. imap-quota
plugin asking quota usage from storage's quota plugin). In any case
the client side API should be asynchronous. That can make it
annoyingly difficult to use though. Wonder if I could switch to
erlang or something for the imap/pop3 processes :)
process-to-process D-Bus!
On Wednesday 27 May 2009 20.14.01 Aria Stewart wrote:
The big problem is what the protocol should be. Use some existing RPC protocol? It should be something extensible so that a plugin in imap process can talk to a plugin in storage process, without the base processes knowing anything about the details (e.g. imap-quota plugin asking quota usage from storage's quota plugin). In any case the client side API should be asynchronous. That can make it annoyingly difficult to use though. Wonder if I could switch to erlang or something for the imap/pop3 processes :)
process-to-process D-Bus!
Pleeeaaaasseeee. Do not need to bloat dovecot with heavy lib-depends, AT ALL.
Daniel
Timo Sirainen skrev:
The big problem is what the protocol should be. Use some existing RPC protocol? It should be something extensible so that a plugin in imap process can talk to a plugin in storage process, without the base processes knowing anything about the details (e.g. imap-quota plugin asking quota usage from storage's quota plugin). In any case the client side API should be asynchronous. That can make it annoyingly difficult to use though. Wonder if I could switch to erlang or something for the imap/pop3 processes :)
SOAP / BEEP: http://www.faqs.org/rfcs/rfc3288.html
You will need to thread calls, waiting for long-running commands. Unless you switch to Erlang, of course :-)
If you don't overdo the XML document part, you can use your own implementation or use saxon (sp?) or another serial XML processor.
/Peter
Peter Lindgren
Timo Sirainen tss@iki.fi writes:
The big problem is what the protocol should be. Use some existing RPC protocol? It should be something extensible so that a plugin in imap process can talk to a plugin in storage process, without the base processes knowing anything about the details (e.g. imap-quota plugin asking quota usage from storage's quota plugin). In any case the client side API should be asynchronous. That can make it annoyingly difficult to use though. Wonder if I could switch to erlang or something for the imap/pop3 processes :)
I realise I'm coming a little late to this conversation. Various people have suggested various RPC mechanisms. Might I humbly suggest also looking at 9P (or 9P2000) the Plan9 filesystem protocol. [1]
The general design philosophy in Plan9 is not to use an increasing number of application specific RPC mechanisms but instead to implement everything in a synthetic filesystem tree. The 9P protocol handles the basic operations required to navigate the filesystem tree and read/write to the files within. The semantics of "RPC" are then relegated to the specification for the layout of the filesystem and meaning assigned to reading or writing to files within that tree. This allows for extensibility (and incidently discoverability) - new features or "plugins" appear as a new file or directory hierarchy in the presented filesystem layout.
As a concrete example the so called upas/fs [2] presents a view of a mailbox independent of the actual storage mechanism used for the data contained in the mailbox. From the manpage
The mailbox itself becomes a directory under /mail/fs. Each message in
the mailbox becomes a numbered directory in the mailbox directory, and
each attachment becomes a numbered directory in the message
directory. Since an attachment may itself be a mail message, this
structure can recurse ad nauseam.
See [2] for more details. I'm not suggesting this layout would be exactly right for the Dovecot storage protocol but it may provide a starting point.
It's probably worth noting the irony, given this is a maillist about Dovecot, in that this approach almost obsoletes the need for an IMAP protocol in the first place (the upas/fs style layout as documented doesn't really provide sufficient support for server side search for example).
Footnotes: [1] http://plan9.bell-labs.com/magic/man2html/5/0intro [2] http://plan9.bell-labs.com/magic/man2html/4/upasfs
pod skrev:
It's probably worth noting the irony, given this is a maillist about Dovecot, in that this approach almost obsoletes the need for an IMAP protocol in the first place (the upas/fs style layout as documented doesn't really provide sufficient support for server side search for example).
Hmmmm, there are more things in IMAP than just this. Mail clients (user agents) that are independent of the server platform, for instance.
/Peter
Peter Lindgren
Peter Lindgren peter@norrskenkonsult.com writes:
Hmmmm, there are more things in IMAP than just this. Mail clients (user agents) that are independent of the server platform, for instance.
I don't understand what you mean. Nothing I've said implies a dependence on the server platform. The presentation of the hierarchy representing the mailbox is analogous to how one might design an HTTP URI hierarchy to present a RESTful interface to a mailbox.
On Jun 2, 2009, at 11:39 AM, pod wrote:
Timo Sirainen tss@iki.fi writes:
The big problem is what the protocol should be. Use some existing RPC protocol? It should be something extensible so that a plugin in imap process can talk to a plugin in storage process, without the base processes knowing anything about the details (e.g. imap-quota plugin asking quota usage from storage's quota plugin). In any case the
client side API should be asynchronous. That can make it annoyingly
difficult to use though. Wonder if I could switch to erlang or something for the imap/pop3 processes :)I realise I'm coming a little late to this conversation. Various
people have suggested various RPC mechanisms. Might I humbly suggest also looking at 9P (or 9P2000) the Plan9 filesystem protocol. [1]The general design philosophy in Plan9 is not to use an increasing
number of application specific RPC mechanisms but instead to implement
everything in a synthetic filesystem tree.
I do kind of like that idea, but I don't really se how it would be
practical here, especially if high performance is wanted.
- I'm not implementing Dovecot to Plan9, so the "filesystem" would
still have to be wrapped in some kind of a protocol. I suppose you
could get them visible to filesystem using FUSE, but that would still
be Linux-only.
- Latency over network is pretty high, so a nice clean filesystem
layout wouldn't probably be possible without sacrificing performance.
And a non-clean layout probably would just make it horrible to use in
all ways.
Actually I think even the current lib-storage API won't be low-enough
latency. Most IMAP commands should be able to be done by sending a
single request and then reading responses. Well, I'm not going to
start coding this anytime soon.
Anyway, I'm still more concerned about how to implement the client
side so that a single process can asynchronously process commands and
handle multiple connections, without the code looking awful difficult
to understand mess. I think it might be possible with C, but I'm not
aware of any existing code that does it as cleanly as synchronous code
looks like.
Timo Sirainen tss@iki.fi writes:
I do kind of like that idea, but I don't really se how it would be practical here, especially if high performance is wanted.
I don't really see why a priori it would be any less performant than any other particular RPC mechanism.
- I'm not implementing Dovecot to Plan9, so the "filesystem" would still
have to be wrapped in some kind of a protocol. I suppose you could get them visible to filesystem using FUSE, but that would still be Linux-only.
I probably didn't explain well enough. One doesn't need to be implementing for or on a Plan9 system and there's no need for there to be any involvement with the OS or kernel's notion of filesystem.
9P is a _wire_ protocol for expressing filesystem hierarchy and I/O on files and dirs within that filesystem. I would like to say "think of it as NFS simplified" but even that will, I suspect, for lots of people draw with it far too much irrelevant baggage. It is a perfectly tractable proposition to implement both 9P servers and clients, e.g. wmii [1], solely with the assistance of a userland library, e.g. libixp [2].
- Latency over network is pretty high, so a nice clean filesystem layout
wouldn't probably be possible without sacrificing performance. And a non-clean layout probably would just make it horrible to use in all ways.
Agreed, careful design of the layout is rather important. But, I suggest, it requires no more or less care than goes into the design of a more traditional RPC mechanism - how are errors signalled, can more than one RPC be in flight at any one time, how is data marshalling done, etc.
Using a synthetic filesystem at least provides a layer of abstraction that might help. I don't claim it makes it easier - it just provides a layer in which some of these questions are already answered. It's an alternate way to factorise the problem.
Actually I think even the current lib-storage API won't be low-enough latency. Most IMAP commands should be able to be done by sending a single request and then reading responses. Well, I'm not going to start coding this anytime soon.
I'm afraid I am sufficiently unfamiliar with the lib-storage API to comment on how straightforwardly one might map it to a 9P-using world.
Anyway, I'm still more concerned about how to implement the client side so that a single process can asynchronously process commands and handle multiple connections, without the code looking awful difficult to understand mess. I think it might be possible with C, but I'm not aware of any existing code that does it as cleanly as synchronous code looks like.
Very valid concerns. I don't think I am able to offer further insight though :-(
Please forgive me if I come across as overly evangelistic. I do not intend to. It is an area of personal interest to me and it felt like it mapped onto your problem nicely. Thanks for listening.
Footnotes: [1] http://wmii.suckless.org/ [2] http://libs.suckless.org/libixp
On Jul 14, 2009, at 8:25 AM, Marc Dequènes (Duck) wrote:
Restarting IMAP/POP3 mail server: dovecotILoading modules from
directory: /usr/lib/dovecot/modules/imap Edlopen(/usr/lib/dovecot/modules/imap/lib10_quota_plugin.so)
failed: /usr/lib/dovecot/modules/imap/lib10_quota_plugin.so:
undefined symbol: xdr_ext_getquota_argsThis is because the Debian package is built with automake 1.9. 1.10
would work, or you could apply this patch:http://hg.dovecot.org/dovecot-1.2/rev/66040bffc1a5
-- To UNSUBSCRIBE, email to debian-bugs-dist-REQU...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
The problem is also in CentOS 5.3, the mentioned patch also solves the problem, but applying the new ones
http://hg.dovecot.org/dovecot-1.2/rev/1b5a1ac430a7 http://hg.dovecot.org/dovecot-1.2/rev/b6aaca59301a
raised the error again.
Regards
+----------------------------------------------------------------+ | Juan C. Blanco | | | | Centro de Calculo | | | Facultad de Informatica U.P.M. | E-mail: jcblanco@fi.upm.es | | Campus de Montegancedo | | | Boadilla del Monte | Tel.: (+34) 91 336 7466 | | 28660 MADRID (Spain) | Fax : (+34) 91 336 6913 | +----------------------------------------------------------------+
participants (9)
-
Andrey Panin
-
Aria Stewart
-
Juan C. Blanco
-
LEVAI Daniel
-
Max Ivanov
-
Peter Lindgren
-
pod
-
reg9009
-
Timo Sirainen