[Dovecot] Future enhancement of imap: sorting?
After solving a problem with a sweet little application I discovered (more on that below), I wanted to get some feedback about a potential enhancement for IMAP: standardized, MUA independant, server-side sorting.
Currently, there are two basic solutions for sorting: handle it during delivery or handle it in the MUA. However, if you handle it in the MUA, you lose some of the client independancy and portability that imap was designed to address, as there is no mechanism to sync sorting rules between clients, and many lightweight clients don't support sorting at all. On the flip side, in distributed mail envrionments sorting at delivery time isn't feasible, as you typically have the entire mailspool owned by a single UID and mounted via NFS. My basic point is that the two paradigms for sorting have fundemental flaws that severaly limit them.
It occured to me that it would be ideal to enable sorting in the imap daemon, enabling users to implement sorting rules which are stored in their directory (similar to the other imap metadata), and which get invoked whenever mail is checked. Perhaps that is beyond the reasonable scope of the imap protocol, but in my mind sorting is a critical task that is poorly addressed and needs to be standardized. I'd love to hear opinions on this.
See, in our mail cluster environment, it's basically impossible to use procmail without forwarding your mail to our shell server (nor are we able to use the dovecot LDA). So, the question presented to me by one of my business partners was, since Outlook was shortsighted and didn't include the ability to sync sorting rules between clients, nor did they implement sorting functionality whatsoever on the windows mobile version, what options does he have to ensure proper sorting of his mail without having to leave outlook running 24x7? He doesn't want to have to wade through mailing list email on his mobile outlook, nor does he want to have to ensure his filter rules are consistent in both his outlook and our webmail.
So, I poked around in the ports tree, and found a gem: imapfilter.
It's configured via lua, supports a very rich array of actions and queries, and has proven to be very robust and lightweight. I helped him get started with some sample rules, added it to run every minute as a cron job, and boom: robust, fully featured, MUA independant filtering in an environment that can't support procmail or the dovecot LDA.
As nice as it is, it's still a personal solution. In my mind we need to have standardized, server-side, MUA-independant sorting rules that get invoked when a mailbox is checked. Thoughts?
Andy
Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972
On Thu, 2008-03-06 at 20:59 -0500, Andy Dills wrote:
Filtering is a better name for this, took me a few paragraphs before I understood what you meant by sorting. :)
You could already create a plugin that runs a command when it sees new messages in a mailbox. I'm not really interested in writing this feature..
A somewhat related feature that I'm going to implement is virtual mailboxes where messages are shown according to specified search rules.
On Fri, 7 Mar 2008, Timo Sirainen wrote:
Heh, sorry about that. In my mind, sorting implies directing an object to a location based on some arbitrary criteria, whereas filtering (in my mind) implies rejecting or accepting based on some arbitrary criteria.
I didn't mean to imply this as a request for functionality...it would be worthless as a feature in dovecot without support in the IMAP protocol and thus the MUAs. I was thinking more along the lines of a future enchanement for the actual IMAP protocol, as this seems like something that should have been addressed but wasn't, leading to an array of flawed solutions. I assume you feel that this is beyond the scope of what IMAP should handle natively? On one hand it feels like it is, but on the other hand it seems like something in desperate need of standardization.
The idea for virtual folders based on search criteria is quite interesting, by the way. How would you enable users to create them from their client?
Thanks, Andy
Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972
On Thu, 2008-03-06 at 21:23 -0500, Andy Dills wrote:
I think Sieve (+ ManageSieve) could already do all of this. It's just normally meant to be used with MDA, but nothing would prevent from running the script when IMAP server sees it for the first time.
I'm just going to make it read the configuration from a file. How it gets there is not my problem, at least not yet. :) So it'll probably get some kind of a web configuration tool.
There do exist a couple of drafts related to this, but I think they're all abandoned now.
On Fri, 7 Mar 2008, Timo Sirainen wrote:
I hadn't considered invoking Sieve from imap, I'll look into that as a more scalable solution.
Makes sense. This feature has a large amount of potential and I would love to see it implemented. The possibilities are exciting, such as a adding a button to atmail's addressbook to create a virtual folder for that contact, which would contain mail sent from them, with a sub folder containing mail sent to them. Even something as simple as being able to associate a message with multiple folders without requiring multiple copies on disk is a nice improvement.
Do you have any timelines in mind for this feature? I'd be happy to help work out the bugs. No pressure, I'm just excited to offer new and useful functionality to our users.
Andy
Andy Dills Xecunet, Inc. www.xecu.net 301-682-9972
participants (2)
-
Andy Dills
-
Timo Sirainen