[Dovecot] Pigeonhole sieve re-filter extension?
Dear Stephan Bosch, dear Dovecot community,
"sieve-filter" re-filtering of existing mails is already a great progress for everyone using sieve! But as far as I know this functionality is just accessible by the server administrator.
In the meanwhile everyone knows the comfort of re-executing filter scripts on his own existing mailboxes - triggered by himself!
This service is already very widespread. Examples are MUA's like Outlook or Thunderbird which provide this service just on the client side. Or almost every web-mail provider offers this.
Is there already an managesieve protocol extension that allows client triggered server side re-filtering?
If not, maybe you could design a RFC for this?
What do you think?
Best regards.
T.B.
On 5/6/2014 12:43 PM, T.B. wrote:
Dear Stephan Bosch, dear Dovecot community,
"sieve-filter" re-filtering of existing mails is already a great progress for everyone using sieve! But as far as I know this functionality is just accessible by the server administrator.
In the meanwhile everyone knows the comfort of re-executing filter scripts on his own existing mailboxes - triggered by himself!
This service is already very widespread. Examples are MUA's like Outlook or Thunderbird which provide this service just on the client side. Or almost every web-mail provider offers this.
Is there already an managesieve protocol extension that allows client triggered server side re-filtering?
If not, maybe you could design a RFC for this?
I proposed something like this in the past, but it didn't catch much attention.
For now, there's this:
https://tools.ietf.org/html/rfc6785
It could be used to create a magic folder that does the re-filtering when a message is moved to it.
In any case, functionality like this is a prerequisite for implementing what you desire. Once this is implemented (there are plans, albeit not very concrete), we can propose a new extension.
Regards,
Stephan.
On 05/06/2014 01:05 PM, Stephan Bosch wrote:
For now, there's this:
https://tools.ietf.org/html/rfc6785
It could be used to create a magic folder that does the re-filtering when a message is moved to it.
There are some similarities, but I think this is not what we want here. We don't need imap events, we don't need new sieve extensions, we don't need metadata. All we need is one simple imap (or managesieve) command that executes current (or specified) script on specified folder (or maybe later a set of messages, like in IMAP SEARCH). The implementation then would be much simpler. -- Aleksander 'A.L.E.C' Machniak LAN Management System Developer [http://lms.org.pl] Roundcube Webmail Developer [http://roundcube.net] --------------------------------------------------- PGP: 19359DC1 @@ GG: 2275252 @@ WWW: http://alec.pl
On 05/06/2014 01:05 PM, Stephan Bosch wrote:
I proposed something like this in the past, but it didn't catch much attention. The only reason I can imagine is the classic chicken-and-egg problem. IMHO the demand is there!
Sieve is still hard to use ... the integration in many MUA's is not end-user friendly. No normal user wants to learn a new complex language just to sort some mails to sub-folders. Many interfaces just allow preconfigured Sieve scripts, e.g. for vacation absence. So at the moment- sieve is mainly used by administrators and enthusiasts. Few Managesieve clients already have some user-friendly kind of point and click or drag and drop GUI - But they are coming!
The wish to move or filter mail comes from EXISTING mails - usually you have a big INBOX and want to do something with it because you lost the overwiew - you want to delete mails or restructure them in sub-folders. But there is NO solution atm. The normal user now has two possibilities: Run to the administrator an beg him to start re-filtering [ -> unacceptable] - or he uses some other client side filtering which you can find in Outlook or Thunderbird. To do client side filtering he has to re-create filter rules with the MUA's own filter GUI. Both is not an acceptable workflow - not at all. No normal user will do that. A third manual alternative would be to use the search function to do something with the mails - but also in addition to the sieve rules, which is also an additional step.
An Managesieve protocol extension to trigger refiltering is THE "missing piece" for an consistent and acceptable workflow in the modern world where multiple devices use the same IMAP account.
IMHO the implementation would be relatively easy: The functionality is already implemented ("sieve-filter"). And call parameters of "sieve-filter" like -e and -W need an equivalent in the Managesieve protocol. Of course everything should be cast in a RFC :) What do you think?
Am 06.05.2014 13:29, schrieb A.L.E.C:
There are some similarities, but I think this is not what we want here. We don't need imap events, we don't need new sieve extensions, we don't need metadata. All we need is one simple imap (or managesieve) command that executes current (or specified) script on specified folder (or maybe later a set of messages, like in IMAP SEARCH). The implementation then would be much simpler.
Exactly - nothing left to be added.
Kind Regards,
T.B.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tue, 6 May 2014, T.B. wrote:
Am 06.05.2014 13:29, schrieb A.L.E.C:
There are some similarities, but I think this is not what we want here. We don't need imap events, we don't need new sieve extensions, we don't need metadata. All we need is one simple imap (or managesieve) command that executes current (or specified) script on specified folder (or maybe later a set of messages, like in IMAP SEARCH). The implementation then would be much simpler.
Exactly - nothing left to be added.
Just musing:
the antispam-plugin triggers e.g. an external command, if you fork it, give it another name and another setting's prefix, you can configure it to spawn sieve-filter on the special "Re-filter" folder, maybe delaying its run to gather all messages from a bulk move operation or something like that. That said, on small systems even a cron may work.
I never used the sieve-filter tool, though. And the man page warns about its useage.
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux)
iQEVAwUBU2nUR3z1H7kL/d9rAQLkNgf/V7/nTTG5AMeoltnXIPAds7c/o11p/puh 9MRe57lXQLDaINJFcE9cdRVrDwtAXSrVAmj1LIpHC9qpVrG6FyguHCn2+cpIUvup TGGcNxtuX1RM6GbEJbGXXgt4KT5r761cPixupDebOypfYLKm18ENxohEF8U+TUHU kes7n8Nyk67wYZhxHt31ubDB+fpIOPTXQKqfjuSxCXSNri2YXvRaciD3+g1UA9XJ hl6fjrDqt3MK+/V/i1LI5kqXh4DUsZysy+DA6eYaF+htSeovCF2fVVVB+pLJxWHw 5jaIz8SZNn7fiW4bLo7buAsyGLmqRIkIhA6VoyioyR9jaKps3Pa1ZQ== =jmvo -----END PGP SIGNATURE-----
Am 07.05.2014 08:35, schrieb Steffen Kaiser:
Just musing:
the antispam-plugin triggers e.g. an external command, if you fork it, give it another name and another setting's prefix, you can configure it to spawn sieve-filter on the special "Re-filter" folder, maybe delaying its run to gather all messages from a bulk move operation or something like that. That said, on small systems even a cron may work.
I never used the sieve-filter tool, though. And the man page warns about its useage.
- -- Steffen Kaiser
I think it would not be a problem to develop a solution to remotely trigger re-filtering for me, myself and I. But that is not the point here. Clients like the Thunderbird Sieve Extension (https://github.com/thsmi/sieve, https://addons.mozilla.org/de/thunderbird/addon/sieve/) or the diverse webmail MUA's will only start implementing such a feature if there is a official draft or specification.
The whole point of my initiative here is that Managesieve finally becomes the capability to replicate the features the users know from their local client side filtering (Thunderbird, Outlook) which provide the feature of re-filtering. Even big webmail providers like the Global Mail Exchange / GMX here in Germany provide re-filtering in their webgui.
I use the sieve-filter tool very often for myself - everytime when I create a new subfolder and create a new fileinto rule, I refilter my Inbox to clean it up and have a consistent subfolder with all old and new mails that are matching the rule.
The man page of the sieve-filter tool is 2 and a half years old ;) (http://pigeonhole.dovecot.org/doc/man1/sieve-filter.1.html) Sadly even the Wiki page doesn't mention it directly: http://wiki2.dovecot.org/HowTo/RefilterMail
The tool works like a charm, but if your sieve script does not do what you actually intended, you can delete all your mails. But that's the same with Thunderbirds or Outlooks client side techniques. The only difference is that both MUA's provide a good GUI which explains and summarizes what the filter rules will do and thereby prevent not intended behavior of the rules.
Since the new german Dovecot book (http://www.dovecot-buch.de/) recommends the sieve-filter tool for refiltering, it will get much more attention in the future.
Best regards,
T.B.
T.B. schreef op 7-5-2014 10:40:
I think it would not be a problem to develop a solution to remotely trigger re-filtering for me, myself and I. But that is not the point here. Clients like the Thunderbird Sieve Extension (https://github.com/thsmi/sieve, https://addons.mozilla.org/de/thunderbird/addon/sieve/) or the diverse webmail MUA's will only start implementing such a feature if there is a official draft or specification.
Yes, I agree.
The whole point of my initiative here is that Managesieve finally becomes the capability to replicate the features the users know from their local client side filtering (Thunderbird, Outlook) which provide the feature of re-filtering. Even big webmail providers like the Global Mail Exchange / GMX here in Germany provide re-filtering in their webgui.
I don't think this should be a ManageSieve feature. ManageSieve currently does not need/have access to the user's mailbox. It therefore also doesn't have the syntax elements and mechanisms in place to select mailboxes and ranges of messages. I think the only sensible place for this feature is IMAP.
I use the sieve-filter tool very often for myself - everytime when I create a new subfolder and create a new fileinto rule, I refilter my Inbox to clean it up and have a consistent subfolder with all old and new mails that are matching the rule.
The man page of the sieve-filter tool is 2 and a half years old ;) (http://pigeonhole.dovecot.org/doc/man1/sieve-filter.1.html) Sadly even the Wiki page doesn't mention it directly: http://wiki2.dovecot.org/HowTo/RefilterMail
I haven't received much feedback about this command line tool. So either everyone is happy with it, or it is rarely used. :)
Since the new german Dovecot book (http://www.dovecot-buch.de/) recommends the sieve-filter tool for refiltering, it will get much more attention in the future.
Only from Germans at first, although it will be translated soon I guess.
Anyway, I will give this idea a closer look somewhat soon. The main problem with IMAPSieve is not the METADATA support or the other Sieve extensions needed for it, it is the atomic nature of the IMAP commands for which it is used: either the whole command succeeds or the whole command fails. This makes things difficult for the Sieve interpreter, as it needs to keep record of what it has done for when a rollback is needed. Especially for "redirect" this is a huge pain.
However, as you rightly say, this new feature can be simpler than that. It can reduce the atomicity to include only the processing of individual messages and e.g. return a response indicating which messages were successfully processed. This way, the state at client and server can still remain consistent without too much trouble. I think I'll make a proof-of-concept first and then condense my experience into a proper specification. This can take a while though; there is much Dovecot stuff on my list at the moment.
Regards,
Stephan.
Hi Stephan Bosch, any update on this?
Regards,
T.B.
Am 08.05.2014 um 14:37 schrieb Stephan Bosch:
T.B. schreef op 7-5-2014 10:40:
I think it would not be a problem to develop a solution to remotely trigger re-filtering for me, myself and I. But that is not the point here. Clients like the Thunderbird Sieve Extension (https://github.com/thsmi/sieve, https://addons.mozilla.org/de/thunderbird/addon/sieve/) or the diverse webmail MUA's will only start implementing such a feature if there is a official draft or specification.
Yes, I agree.
The whole point of my initiative here is that Managesieve finally becomes the capability to replicate the features the users know from their local client side filtering (Thunderbird, Outlook) which provide the feature of re-filtering. Even big webmail providers like the Global Mail Exchange / GMX here in Germany provide re-filtering in their webgui.
I don't think this should be a ManageSieve feature. ManageSieve currently does not need/have access to the user's mailbox. It therefore also doesn't have the syntax elements and mechanisms in place to select mailboxes and ranges of messages. I think the only sensible place for this feature is IMAP.
I use the sieve-filter tool very often for myself - everytime when I create a new subfolder and create a new fileinto rule, I refilter my Inbox to clean it up and have a consistent subfolder with all old and new mails that are matching the rule.
The man page of the sieve-filter tool is 2 and a half years old ;) (http://pigeonhole.dovecot.org/doc/man1/sieve-filter.1.html) Sadly even the Wiki page doesn't mention it directly: http://wiki2.dovecot.org/HowTo/RefilterMail
I haven't received much feedback about this command line tool. So either everyone is happy with it, or it is rarely used. :)
Since the new german Dovecot book (http://www.dovecot-buch.de/) recommends the sieve-filter tool for refiltering, it will get much more attention in the future.
Only from Germans at first, although it will be translated soon I guess.
Anyway, I will give this idea a closer look somewhat soon. The main problem with IMAPSieve is not the METADATA support or the other Sieve extensions needed for it, it is the atomic nature of the IMAP commands for which it is used: either the whole command succeeds or the whole command fails. This makes things difficult for the Sieve interpreter, as it needs to keep record of what it has done for when a rollback is needed. Especially for "redirect" this is a huge pain.
However, as you rightly say, this new feature can be simpler than that. It can reduce the atomicity to include only the processing of individual messages and e.g. return a response indicating which messages were successfully processed. This way, the state at client and server can still remain consistent without too much trouble. I think I'll make a proof-of-concept first and then condense my experience into a proper specification. This can take a while though; there is much Dovecot stuff on my list at the moment.
Regards,
Stephan.
Op 9/21/2015 om 12:16 AM schreef T.B.:
Hi Stephan Bosch, any update on this?
Unfortunately, no.
It is not high up my list at the moment. :/
Regards,
Stephan.
Regards,
T.B.
Am 08.05.2014 um 14:37 schrieb Stephan Bosch:
T.B. schreef op 7-5-2014 10:40:
I think it would not be a problem to develop a solution to remotely trigger re-filtering for me, myself and I. But that is not the point here. Clients like the Thunderbird Sieve Extension (https://github.com/thsmi/sieve, https://addons.mozilla.org/de/thunderbird/addon/sieve/) or the diverse webmail MUA's will only start implementing such a feature if there is a official draft or specification.
Yes, I agree.
The whole point of my initiative here is that Managesieve finally becomes the capability to replicate the features the users know from their local client side filtering (Thunderbird, Outlook) which provide the feature of re-filtering. Even big webmail providers like the Global Mail Exchange / GMX here in Germany provide re-filtering in their webgui.
I don't think this should be a ManageSieve feature. ManageSieve currently does not need/have access to the user's mailbox. It therefore also doesn't have the syntax elements and mechanisms in place to select mailboxes and ranges of messages. I think the only sensible place for this feature is IMAP.
I use the sieve-filter tool very often for myself - everytime when I create a new subfolder and create a new fileinto rule, I refilter my Inbox to clean it up and have a consistent subfolder with all old and new mails that are matching the rule.
The man page of the sieve-filter tool is 2 and a half years old ;) (http://pigeonhole.dovecot.org/doc/man1/sieve-filter.1.html) Sadly even the Wiki page doesn't mention it directly: http://wiki2.dovecot.org/HowTo/RefilterMail
I haven't received much feedback about this command line tool. So either everyone is happy with it, or it is rarely used. :)
Since the new german Dovecot book (http://www.dovecot-buch.de/) recommends the sieve-filter tool for refiltering, it will get much more attention in the future.
Only from Germans at first, although it will be translated soon I guess.
Anyway, I will give this idea a closer look somewhat soon. The main problem with IMAPSieve is not the METADATA support or the other Sieve extensions needed for it, it is the atomic nature of the IMAP commands for which it is used: either the whole command succeeds or the whole command fails. This makes things difficult for the Sieve interpreter, as it needs to keep record of what it has done for when a rollback is needed. Especially for "redirect" this is a huge pain.
However, as you rightly say, this new feature can be simpler than that. It can reduce the atomicity to include only the processing of individual messages and e.g. return a response indicating which messages were successfully processed. This way, the state at client and server can still remain consistent without too much trouble. I think I'll make a proof-of-concept first and then condense my experience into a proper specification. This can take a while though; there is much Dovecot stuff on my list at the moment.
Regards,
Stephan.
participants (4)
-
A.L.E.C
-
Steffen Kaiser
-
Stephan Bosch
-
T.B.