Using filter in an imapsieve script?
Hello list
I currently have an issue with an imapsieve script on my dovecot server
CentOS Linux release 7.3.1611 (Core) Dovecot 2.2.26.0 (23d1de6) Pigeonhole 2.2.26.0
The goal is to "fire" an imapsieve script upon mailclient saves message to sent folder I setup the following in 90-plugin.conf:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_extensions = +vnd.dovecot.filter +vnd.dovecot.pipe +vnd.dovecot.execute sieve_filter_bin_dir = /etc/dovecot/sieve-filters sieve_pipe_bin_dir = /etc/dovecot/sieve-filters sieve_execute_bin_dir = /etc/dovecot/sieve-filters sieve_filter_exec_timeout = 10000 sieve_pipe_exec_timeout = 10000 sieve_execute_exec_timeout = 10000 imapsieve_mailbox1_name = Sent imapsieve_mailbox1_causes = COPY APPEND imapsieve_mailbox1_before = file:/home/vmail/domain/user/dovecot-crypt-sent.sieve }
and the content of the sieve script is:
require ["environment", "vnd.dovecot.filter", "variables", "imapsieve", "vnd.dovecot.pipe", "vnd.dovecot.execute"];
if anyof (environment :is "imap.cause" "APPEND", environment :is "imap.cause" "COPY") { filter "gpgit" "myuser@mydomain.tld"; }
gpgit is a perl script that encrypts a given message using the users pub key and returns back the encrypted message As a sieve script for lmtp/lda it works fine and encrypted messages show up in the mailbox. But it does not work if it is used as imap sieve script There is nothing in the logs that indicates a problem with the imap sieve script. Is it possible that 'filter' is not supported for imapsieve as it changes the content of a message?
Thanks for any hint
tobi
Op 4/5/2017 om 11:48 AM schreef Tobi:
Hello list
I currently have an issue with an imapsieve script on my dovecot server
CentOS Linux release 7.3.1611 (Core) Dovecot 2.2.26.0 (23d1de6) Pigeonhole 2.2.26.0
The goal is to "fire" an imapsieve script upon mailclient saves message to sent folder I setup the following in 90-plugin.conf:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_extensions = +vnd.dovecot.filter +vnd.dovecot.pipe +vnd.dovecot.execute sieve_filter_bin_dir = /etc/dovecot/sieve-filters sieve_pipe_bin_dir = /etc/dovecot/sieve-filters sieve_execute_bin_dir = /etc/dovecot/sieve-filters sieve_filter_exec_timeout = 10000 sieve_pipe_exec_timeout = 10000 sieve_execute_exec_timeout = 10000 imapsieve_mailbox1_name = Sent imapsieve_mailbox1_causes = COPY APPEND imapsieve_mailbox1_before = file:/home/vmail/domain/user/dovecot-crypt-sent.sieve }
and the content of the sieve script is:
require ["environment", "vnd.dovecot.filter", "variables", "imapsieve", "vnd.dovecot.pipe", "vnd.dovecot.execute"];
if anyof (environment :is "imap.cause" "APPEND", environment :is "imap.cause" "COPY") { filter "gpgit" "myuser@mydomain.tld"; }
gpgit is a perl script that encrypts a given message using the users pub key and returns back the encrypted message As a sieve script for lmtp/lda it works fine and encrypted messages show up in the mailbox. But it does not work if it is used as imap sieve script There is nothing in the logs that indicates a problem with the imap sieve script. Is it possible that 'filter' is not supported for imapsieve as it changes the content of a message?
Thanks for any hint
Did you add the imap_sieve plugin to mail_plugins for imap?
Once you enable mail_debug, your logs should show Sieve activity in imap.
Regards,
Stephan.
Hi Stephan
yes the imap_sieve plugin is added to the mail_plugins for imap. Thanks for the hint with mail_debug. After enabling it I can see that the program seems to be called, so filter should not be the problem. But the result is that the message appears unencrypted in my sent folder
Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: imapsieve: mailbox Sent: APPEND event Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Pigeonhole version 0.4.16 (fed8554) initializing Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: include: sieve_global is not set; it is currently not possible to include
:global' scripts. Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.4.16 (fed8554) loaded Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.4.16 (fed8554) loaded Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: imapsieve: Static mailbox rule [1]: mailbox=Spam' from=*' causes=(COPY) => before=file:/home/vmail/brain-force.ch/tobster/dovecot-mail-filter.sieve' after=(none) Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: imapsieve: Static mailbox rule [2]: mailbox=Sent' from=*' causes=(COPY APPEND) => before=file:/home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.sieve' after=(none) Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: imapsieve: Matched static mailbox rule [2] Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: file storage: Using active Sieve script path: /home/vmail/brain-force.ch/tobster/.dovecot.sieve Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: file storage: Using script storage path: /home/vmail/brain-force.ch/tobster/sieve Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: file storage: Relative path to sieve storage in active link: sieve/ Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: file storage: Using Sieve script path: /home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.sieve Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: file script: Opened scriptdovecot-crypt-sent' from/home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.sieve' Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Opening script 1 of 1 from/home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.sieve' Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Loading script /home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.sieve Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Script binary /home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.svbin successfully loaded Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: binary save: not saving binary /home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.svbin, because it is already stored Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Executing script from `/home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.svbin' Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: action filter: running program: gpgit Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: Mailbox Sent: Opened mail UID=3800 because: mail stream
From my understanding the logs looks fine. Just here
Debug: sieve: action filter: running program: gpgit
I wonder if the parameter given to gpgit should be logged as well? Calling gpgit without the userparameter would explain why the message appears unencrypted in sent mailbox.
Best regards
tobi
Am 06.04.2017 um 00:58 schrieb Stephan Bosch:
Op 4/5/2017 om 11:48 AM schreef Tobi:
Hello list
I currently have an issue with an imapsieve script on my dovecot server
CentOS Linux release 7.3.1611 (Core) Dovecot 2.2.26.0 (23d1de6) Pigeonhole 2.2.26.0
The goal is to "fire" an imapsieve script upon mailclient saves message to sent folder I setup the following in 90-plugin.conf:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_extensions = +vnd.dovecot.filter +vnd.dovecot.pipe +vnd.dovecot.execute sieve_filter_bin_dir = /etc/dovecot/sieve-filters sieve_pipe_bin_dir = /etc/dovecot/sieve-filters sieve_execute_bin_dir = /etc/dovecot/sieve-filters sieve_filter_exec_timeout = 10000 sieve_pipe_exec_timeout = 10000 sieve_execute_exec_timeout = 10000 imapsieve_mailbox1_name = Sent imapsieve_mailbox1_causes = COPY APPEND imapsieve_mailbox1_before = file:/home/vmail/domain/user/dovecot-crypt-sent.sieve }
and the content of the sieve script is:
require ["environment", "vnd.dovecot.filter", "variables", "imapsieve", "vnd.dovecot.pipe", "vnd.dovecot.execute"];
if anyof (environment :is "imap.cause" "APPEND", environment :is "imap.cause" "COPY") { filter "gpgit" "myuser@mydomain.tld"; }
gpgit is a perl script that encrypts a given message using the users pub key and returns back the encrypted message As a sieve script for lmtp/lda it works fine and encrypted messages show up in the mailbox. But it does not work if it is used as imap sieve script There is nothing in the logs that indicates a problem with the imap sieve script. Is it possible that 'filter' is not supported for imapsieve as it changes the content of a message?
Thanks for any hint Did you add the imap_sieve plugin to mail_plugins for imap?
Once you enable mail_debug, your logs should show Sieve activity in imap.
Regards,
Stephan.
To further debug I wrote a little shell wrapper for my gpgit script. That wrapper now is called from imap sieve script. The wrapper writes exit code of gpgit and the mail content returned by gpgit into a logfile. I can see that gpgit returns 0 and the mail content returned is encrypted. But still the mail that appears in sent mailbox is NOT encrypted. How can that be? It seems that for what reason ever the mail is stored as it's passed to the filter and not how it's returned by filter. Any idea whats going on?
Thanks for any idea how to solve this issue
tobi
Am 06.04.2017 um 08:31 schrieb Tobi:
Hi Stephan
yes the imap_sieve plugin is added to the mail_plugins for imap. Thanks for the hint with mail_debug. After enabling it I can see that the program seems to be called, so filter should not be the problem. But the result is that the message appears unencrypted in my sent folder
Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: imapsieve: mailbox Sent: APPEND event Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Pigeonhole version 0.4.16 (fed8554) initializing Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: include: sieve_global is not set; it is currently not possible to include
:global' scripts. Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.4.16 (fed8554) loaded Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.4.16 (fed8554) loaded Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: imapsieve: Static mailbox rule [1]: mailbox=Spam' from=*' causes=(COPY) => before=file:/home/vmail/brain-force.ch/tobster/dovecot-mail-filter.sieve' after=(none) Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: imapsieve: Static mailbox rule [2]: mailbox=Sent' from=*' causes=(COPY APPEND) => before=file:/home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.sieve' after=(none) Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: imapsieve: Matched static mailbox rule [2] Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: file storage: Using active Sieve script path: /home/vmail/brain-force.ch/tobster/.dovecot.sieve Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: file storage: Using script storage path: /home/vmail/brain-force.ch/tobster/sieve Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: file storage: Relative path to sieve storage in active link: sieve/ Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: file storage: Using Sieve script path: /home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.sieve Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: file script: Opened scriptdovecot-crypt-sent' from/home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.sieve' Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Opening script 1 of 1 from/home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.sieve' Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Loading script /home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.sieve Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Script binary /home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.svbin successfully loaded Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: binary save: not saving binary /home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.svbin, because it is already stored Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: Executing script from `/home/vmail/brain-force.ch/tobster/dovecot-crypt-sent.svbin' Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: sieve: action filter: running program: gpgit Apr 6 08:20:26 mbox2 dovecot: imap(tobster@brain-force.ch): Debug: Mailbox Sent: Opened mail UID=3800 because: mail streamFrom my understanding the logs looks fine. Just here
Debug: sieve: action filter: running program: gpgit I wonder if the parameter given to gpgit should be logged as well? Calling gpgit without the userparameter would explain why the message appears unencrypted in sent mailbox.
Best regards
tobi
Am 06.04.2017 um 00:58 schrieb Stephan Bosch:
Op 4/5/2017 om 11:48 AM schreef Tobi:
Hello list
I currently have an issue with an imapsieve script on my dovecot server
CentOS Linux release 7.3.1611 (Core) Dovecot 2.2.26.0 (23d1de6) Pigeonhole 2.2.26.0
The goal is to "fire" an imapsieve script upon mailclient saves message to sent folder I setup the following in 90-plugin.conf:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_extensions = +vnd.dovecot.filter +vnd.dovecot.pipe +vnd.dovecot.execute sieve_filter_bin_dir = /etc/dovecot/sieve-filters sieve_pipe_bin_dir = /etc/dovecot/sieve-filters sieve_execute_bin_dir = /etc/dovecot/sieve-filters sieve_filter_exec_timeout = 10000 sieve_pipe_exec_timeout = 10000 sieve_execute_exec_timeout = 10000 imapsieve_mailbox1_name = Sent imapsieve_mailbox1_causes = COPY APPEND imapsieve_mailbox1_before = file:/home/vmail/domain/user/dovecot-crypt-sent.sieve }
and the content of the sieve script is:
require ["environment", "vnd.dovecot.filter", "variables", "imapsieve", "vnd.dovecot.pipe", "vnd.dovecot.execute"];
if anyof (environment :is "imap.cause" "APPEND", environment :is "imap.cause" "COPY") { filter "gpgit" "myuser@mydomain.tld"; }
gpgit is a perl script that encrypts a given message using the users pub key and returns back the encrypted message As a sieve script for lmtp/lda it works fine and encrypted messages show up in the mailbox. But it does not work if it is used as imap sieve script There is nothing in the logs that indicates a problem with the imap sieve script. Is it possible that 'filter' is not supported for imapsieve as it changes the content of a message?
Thanks for any hint Did you add the imap_sieve plugin to mail_plugins for imap?
Once you enable mail_debug, your logs should show Sieve activity in imap.
Regards,
Stephan.
Op 4/5/2017 om 11:48 AM schreef Tobi:
Hello list
OK, I know why this is happening now.
I currently have an issue with an imapsieve script on my dovecot server
CentOS Linux release 7.3.1611 (Core) Dovecot 2.2.26.0 (23d1de6) Pigeonhole 2.2.26.0
The goal is to "fire" an imapsieve script upon mailclient saves message to sent folder I setup the following in 90-plugin.conf:
plugin { sieve_plugins = sieve_imapsieve sieve_extprograms sieve_extensions = +vnd.dovecot.filter +vnd.dovecot.pipe +vnd.dovecot.execute sieve_filter_bin_dir = /etc/dovecot/sieve-filters sieve_pipe_bin_dir = /etc/dovecot/sieve-filters sieve_execute_bin_dir = /etc/dovecot/sieve-filters sieve_filter_exec_timeout = 10000 sieve_pipe_exec_timeout = 10000 sieve_execute_exec_timeout = 10000 imapsieve_mailbox1_name = Sent imapsieve_mailbox1_causes = COPY APPEND imapsieve_mailbox1_before = file:/home/vmail/domain/user/dovecot-crypt-sent.sieve }
This is all OK.
and the content of the sieve script is:
require ["environment", "vnd.dovecot.filter", "variables", "imapsieve", "vnd.dovecot.pipe", "vnd.dovecot.execute"];
if anyof (environment :is "imap.cause" "APPEND", environment :is "imap.cause" "COPY") { filter "gpgit" "myuser@mydomain.tld"; }
This is not, because from https://tools.ietf.org/html/rfc6785#section-3.1:
For all cases that fall under IMAP events in Sieve, the implicit keep means that the message is treated as it would have been if no Sieve script were run. For APPEND and COPY, the message is stored into the target mailbox normally. For flag changes, the message is left in the mailbox. If actions have been taken that change the message, those changes are considered transient and MUST NOT be retained for any "keep" action (because IMAP messages are immutable). No error is generated, but the original message, without the changes, is kept.
So, your implicit "keep" will ignore the changes made by the filter command.
gpgit is a perl script that encrypts a given message using the users pub key and returns back the encrypted message As a sieve script for lmtp/lda it works fine and encrypted messages show up in the mailbox. But it does not work if it is used as imap sieve script There is nothing in the logs that indicates a problem with the imap sieve script. Is it possible that 'filter' is not supported for imapsieve as it changes the content of a message?
In the simplest case, it looks that way, yes.
However, there may be a way around that. Unlike "keep", the "fileinto" command will store the changed message.
I tested the following Sieve script in place of yours:
require "variables"; require "fileinto"; require "imapsieve"; require "environment"; require "vnd.dovecot.filter";
Obtain the destination mailbox name
if environment :matches "imap.mailbox" "*" { set "mailbox" "${1}"; }
No need to check imap.cause like you did, since the condition you
formulated is always true with your
configuration.
Encrypt
if filter "gpgit" { # Create an encrypted copy of the message fileinto "${mailbox}";
# Since implicit keep is canceled, original saved/copied messageis marked as \Deleted, soon to be expunged. stop; }
If encryption fails, the original message is kept in place
This works.
However, I now remember I tested this in the past and there was one snag. The effect of this is that the message is stored twice in the Sent mailbox:
- The first is the original message. Since it was discarded, it has the \Deleted flag set and it will disappear at the next EXPUNGE
- The encrypted message stored with "fileinto".
When I tested this a little more than a year ago, Thunderbird got confused and kept showing the original message in the Sent folder and not the encrypted one. Only a restart of Thunderbird would fix that. Other mail clients may have similar issues. This is valid IMAP behavior (think other client deleting the message right after it was saved), so the server is not to blame.
Regards,
Stephan.
Am 08.04.2017 um 10:00 schrieb Stephan Bosch:
However, I now remember I tested this in the past and there was one snag. The effect of this is that the message is stored twice in the Sent mailbox:
- The first is the original message. Since it was discarded, it has the \Deleted flag set and it will disappear at the next EXPUNGE
- The encrypted message stored with "fileinto".
I changed my sieve script as you described and get the following:
if save encrypted to same mailbox as original message only the original message can be seen in TB. Even restart TB does not change anything. But the enc message is in Sent on server, just TB does **never** show it
if I save the encrypted message to another mailbox then I can see the ecrypted msg in TB and the original msg in Sent
Do you think that the problem could be solved by using another client instead of TB? Is there a way around this issue on serverside? Like for example use pipe? Or does pipe store the original message as well on server?
Cheers
tobi
Hi Stephan
Found a "way" to solve the issue in Thunderbird. If I go into "Properties" of the Sent folder and chose "repair" then TB loads everything from the server again and tada only the encrypted message is shown. But as I have about 4000 msg in my sent that's not a real solution :-)
Cheers
tobi
p.s. as you really helped me a lot already is there something I can do for you in revenge? Do you have for example an Amazon Wishlist or something like that?
Am 09.04.2017 um 10:07 schrieb Tobi:
Am 08.04.2017 um 10:00 schrieb Stephan Bosch:
However, I now remember I tested this in the past and there was one snag. The effect of this is that the message is stored twice in the Sent mailbox:
- The first is the original message. Since it was discarded, it has the \Deleted flag set and it will disappear at the next EXPUNGE
- The encrypted message stored with "fileinto".
I changed my sieve script as you described and get the following:
if save encrypted to same mailbox as original message only the original message can be seen in TB. Even restart TB does not change anything. But the enc message is in Sent on server, just TB does **never** show it
if I save the encrypted message to another mailbox then I can see the ecrypted msg in TB and the original msg in Sent
Do you think that the problem could be solved by using another client instead of TB? Is there a way around this issue on serverside? Like for example use pipe? Or does pipe store the original message as well on server?
Cheers
tobi
participants (2)
-
Stephan Bosch
-
Tobi