[Dovecot] Writing a new mailbox access module
Having finished a few other projects, I'm contemplating trying to write a new mailbox module which could be useful for us here. I would appreciate any guidance.
Basically, I need (or rather, could use) a module that provides a merge of two mailboxes (which are themselves accessed by other modules), cascading them, so that:
The user sees one mailbox M, which contains all the messages in the mailboxes A and B.
A and B are both already accessible by existing Dovecot mailbox access modules, but possibly by different modules (e.g. A is an mbox, B is a Maildir).
Any write operation (creation of a new message, etc.) is eventually performed on B.
As soon as a message that was in A is accessed (either for read or for write), it is transferred to B.
I would use such a module to combine the spool file in /var/mail with some INBOX in the user's home directory. The incentive is the following:
Traditionally, users receive their mail in the spool, commonly in /var/spool. Mailers, like mail, elm, pine, mutt, etc. move each read message to some record mailbox (commonly ~/mbox, but with the more advanced mailers it can be configured to be some other mailbox, even a Maildir).
Users keep working this way. IMAP will not substitute the regular mail access when inside the network here. But it should be as convenient as possible to use it from the outside (e.g. at home, or from an Internet Café, or using some Webmail interface).
Mainly for administrative reasons, it is desirable that mail keeps being delivered to the spool, and not directly to the user's home directory. One of the reasons is that mail never fails to be delivered as long as there is space in the spool. When the user is out of quota on his home directory, the mail does not get rejected, and a failure only occurs when the user initiates some operation (like reading the message). Another reason is simply to be compatible with the way traditional mailers and users work.
IMAP, and IMAP clients (e.g. Thunderbird, the most common one here), on the other hand, usually do not recognize the concept of a spool. There is an INBOX, where all mail arrives, and this is where all new messages should arrive, and remain, unless they are deleted moved elsewhere.
It would of course not be desirable to just move messages automatically after the user reads them, because it would confuse users when a message suddenly disappears from a mailbox all by itself.
The best way I can think of to be consistent with both the traditional way mailers work (moving messages from spool to ~/mbox after they are read) and the way IMAP is commonly used (INBOX contains all received messages not deleted or moved, including messages that just arrived), would be using such a module as I suggested, with (using the previous terminology) e.g. M=INBOX, A=spool, and B=~/mbox.
I have just skimmed again through the mailbox code (which I last looked at over a year ago, back when it was in alpha releases), not going too much into details. From past experience and current skimming, working with that code doesn't seem too straightforward, though.
I can think of how to implement the logic I described on its own (i.e. as a library with a few functions), but not sure how difficult it will be to interface it with Dovecot. Is there any developer/module writer's guide somewhere that explains the mailbox access module interface?
I'd appreciate some basic guidance to start with. If I get to working on it seriously, expect lots of questions from me about the code :-)
Regards, -- Tom
-- Tom Alsberg - hacker (being the best description fitting this space) Web page: http://www.cs.huji.ac.il/~alsbergt/ DISCLAIMER: The above message does not even necessarily represent what my fingers have typed on the keyboard, save anything further.
On Sun, 2007-05-13 at 14:29 +0300, Tom Alsberg wrote:
The user sees one mailbox M, which contains all the messages in the mailboxes A and B.
A and B are both already accessible by existing Dovecot mailbox access modules, but possibly by different modules (e.g. A is an mbox, B is a Maildir).
Any write operation (creation of a new message, etc.) is eventually performed on B.
As soon as a message that was in A is accessed (either for read or for write), it is transferred to B.
So individual messages are separately moved to ~/mbox while they're being read? And by read what exactly do you mean? When IMAP client reads the message, or when the message is marked as \Seen?
The biggest problem I see with this is message UIDs. IMAP requires that the mailbox has growing UIDs. The message UIDs preferrably also shouldn't change, otherwise the clients will see them as new messages. This can't really work with the current Dovecot code, for example:
- Move message with UID 2 to ~/mbox
- Move message with UID 1 to ~/mbox
To avoid UID changing the 2rd step would require that the message is inserted into the mbox file as the first message. Dovecot has no code to do that.
Another possibility would be to allow messages to be in the mbox file non-sorted. That would probably be easier to implement than insertion code. Actually I think that would only require removing a few checks from mbox-parse.c
I have just skimmed again through the mailbox code (which I last looked at over a year ago, back when it was in alpha releases), not going too much into details. From past experience and current skimming, working with that code doesn't seem too straightforward, though.
I've been trying to make it easier to implement new storages in CVS HEAD, but there is still a bit of work to be done. Cydir is the simplest possible storage backend, so looking at that could be useful.
I can think of how to implement the logic I described on its own (i.e. as a library with a few functions), but not sure how difficult it will be to interface it with Dovecot. Is there any developer/module writer's guide somewhere that explains the mailbox access module interface?
Unfortunately not. I've been thinking about writing something to wiki -> Design about this too.
participants (2)
-
Timo Sirainen
-
Tom Alsberg