AW: AW: AW: IMAPSieve plugin will not run rspamd script
Hi John,
the prefix is just a sign of my desperation - I tried all sorts of variations yesterday and now forgot to undo it.
There’s a very detailed tutorial available (German language) with Debian 10. Just the sieve scripts are little different. I spend the time to raise a Debian 10 with this tutorial – and with exact the same result as with my up to date configuration.
So it would be interesting for me to know if there is anyone here who has managed to get this to work on Debian?
Jens
Von: John Fawcett john@voipsupport.it Gesendet: Montag, 1. Juli 2024 18:26 An: postfix_dovecot@gmx.de Betreff: Re: AW: AW: IMAPSieve plugin will not run rspamd script
On 01/07/2024 18:08, postfix_dovecot@gmx.de mailto:postfix_dovecot@gmx.de wrote:
Hi John,
is there any other log than system?
Jens
Hi Jens
I meant the same type of logging you already posted (i.e. as below) for exactly the same test case, but where the rules use the mailbox name that includes the prefix you set for the namespace (i.e.INBOX/Spam).
There is probably a good reason why you have configured a prefix for your namespace, but my suspicion is that the example imapsieve config you copied assumes no prefix set (it was not part of the example so we cannot see it).
I personally tried the settings from the example and they work fine and I have no prefix. I also did the opposite test and I get the behaviour you reported when I have prefix set. In order to solve it (without changing the prefix) I had to state the full mailbox name including the prefix in the imapsieve rules. In that case it worked. What I'd like to check in your logging is if there is a clue why it didn't work for you when yu used INBOX/Spam in the rules.
John
imap(mail@test.example mailto:mail@test.example )<1797>: Debug: Module loaded: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so
imap(mail@test.example mailto:mail@test.example )<1797>: Debug: imapsieve: mailbox INBOX/Spam: APPEND event
imap(mail@test.example mailto:mail@test.example )<1797>: Debug: sieve: Pigeonhole version 0.5.19 (4eae2f79) initializing
imap(mail@test.example mailto:mail@test.example )<1797>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts.
imap(mail@test.example mailto:mail@test.example )<1797>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded
imap(mail@test.example mailto:mail@test.example )<1797>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded
imap(mail@test.example mailto:mail@test.example )<1797>: Debug: imapsieve: Static mailbox rule [1]: mailbox=Spam' from=*' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none)
imap(mail@test.example mailto:mail@test.example )<1797>: Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Spam' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none)
imap(mail@test.example mailto:mail@test.example )<1795><iC6JmQIcnqvAqB4a>: Debug: imapsieve: mailbox INBOX: FLAG event (changed flags: \Deleted)
imap(mail@test.example mailto:mail@test.example )<1795><iC6JmQIcnqvAqB4a>: Debug: sieve: Pigeonhole version 0.5.19 (4eae2f79) initializing
imap(mail@test.example mailto:mail@test.example )<1795><iC6JmQIcnqvAqB4a>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts.
imap(mail@test.example mailto:mail@test.example )<1795><iC6JmQIcnqvAqB4a>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded
imap(mail@test.example mailto:mail@test.example )<1795><iC6JmQIcnqvAqB4a>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded
imap(mail@test.example mailto:mail@test.example )<1795><iC6JmQIcnqvAqB4a>: Debug: imapsieve: Static mailbox rule [1]: mailbox=Spam' from=*' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none)
imap(mail@test.example mailto:mail@test.example )<1795><iC6JmQIcnqvAqB4a>: Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Spam' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none)
imap(mail@test.example mailto:mail@test.example )<1795><iC6JmQIcnqvAqB4a>: Debug: imapsieve: mailbox INBOX: FLAG event (changed flags: \Seen)
On 01/07/2024 22:09, postfix_dovecot--- via dovecot wrote:
Hi John,
the prefix is just a sign of my desperation - I tried all sorts of variations yesterday and now forgot to undo it.
There’s a very detailed tutorial available (German language) with Debian 10. Just the sieve scripts are little different. I spend the time to raise a Debian 10 with this tutorial – and with exact the same result as with my up to date configuration.
So it would be interesting for me to know if there is anyone here who has managed to get this to work on Debian?
Jens
Hi Jens
changing random things is rarely a good way to solve these kinds of issues. My advice, if you don't need the inbox namespace prefix set for a specific reason, would be to go with the default prefix (ie. blank) and then set up the imapsieve rules as per your original post and repeat the test. If that does not work then post the debug logging and configuration from that test.
There could be multiple places where this is failing but the very first point is to have your sieve rules match. They will show something like the following in the logging. Notice the "Matched static mailbox rule" message. If that is not happening it's pointless to look further down the line into issues in the scripts themselves.
Jul 01 22:57:55 imola.site24.it dovecot[1145722]:
imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve:
mailbox INBOX: MOVE event
Jul 01 22:57:55 imola.site24.it dovecot[1145722]:
imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: Mailbox Spam:
UID 2: Expunge requested
Jul 01 22:57:55 imola.site24.it dovecot[1145722]:
imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: duplicate db:
Initialize
Jul 01 22:57:55 imola.site24.it dovecot[1145722]:
imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve:
Pigeonhole version 0.5.21 (f6cd4b8e) initializing
Jul 01 22:57:55 imola.site24.it dovecot[1145722]:
imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: include:
sieve_global is not set; it is currently not possible to include
:global' scripts. Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.21 (f6cd4b8e) loaded Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.21 (f6cd4b8e) loaded Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: Static mailbox rule [1]: mailbox=Spam' from=*' causes=(COPY) => before=file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none)
Jul 01 22:57:55 imola.site24.it dovecot[1145722]:
imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve:
Static mailbox rule [2]: mailbox=*' from=Spam' causes=(COPY APPEND) =>
before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none)
Jul 01 22:57:55 imola.site24.it dovecot[1145722]:
imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve:
Matched static mailbox rule [2]
As for Debian, sorry I can't help on that, I'm using Fedora :-)
John
Hi John!
You are of course absolutely right - wild experimentation is rarely a good approach. But you are surely familiar with the situation when you have done everything logical and then, in your desperation, you start to change things that you would not otherwise change ;)
I've actually gotten to the point where I'm going to give up on the idea and run a cron job at night. Then the spam won't be detected in real time - but what the heck?
As I wrote yesterday, I implemented a very popular tutorial for Debian 10 in Germany 1:1. The result was the same. Nevertheless, I reset my VM, where I do everything with the current releases, and attaching the log here. I would be very happy if it still works.
To be sure, I deleted the whole inbox and recreated the account on Outlook to create a fresh, new folder structure given by the Dovecot configuration. I also added the plugin vnd.dovecot.debug to the sieve_global_extensions line and added >debug_log "report-spam.sieve was running!";< to the report-spam.sieve file. So this string should appear in the log if the script were running at all. But it doesn't. So I still guess, it's not getting triggered.
That's the log if I move a mail from Junk (Spam) to Inbox:
# journalctl -u dovecot -f
imap-login: Login: user=info@demo.example, method=PLAIN, rip=192.168.30.26, lip=192.168.30.34, mpid=1827, TLS, session=<dJVhwj4cJP3AqB4a>
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Loading modules from directory: /usr/lib/dovecot/modules
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Module loaded: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Effective uid=10000, gid=10000, home=/var/mail/vhosts/demo.example/info
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: open(/proc/self/io) failed: Permission denied
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: maildir++: root=/var/mail/vhosts/demo.example/info/Maildir, index=, indexpvt=, control=, inbox=/var/mail/vhosts/demo.example/info/Maildir, alt=
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: none: root=, index=, indexpvt=, control=, inbox=, alt=
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Mailbox INBOX/Junk: Mailbox opened
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: mailbox INBOX/Junk: APPEND event
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: duplicate db: Initialize
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: Pigeonhole version 0.5.19 (4eae2f79) initializing
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include :global' scripts. imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: Static mailbox rule [1]: mailbox=Junk' from=*' causes=(COPY) => before=file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none)
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Junk' causes=(COPY) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none)
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Mailbox INBOX/Junk: Mailbox opened
imap(info@demo.example)<1825>
# doveconf -n # 2.3.19.1 (9b53102964): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 6.1.0-21-amd64 x86_64 Debian 12.5 # Hostname: ServerIV-home.demo.example auth_mechanisms = plain login mail_debug = yes mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { hidden = no ignore_on_failure = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX/ separator = / subscriptions = yes type = private } passdb { driver = pam } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /etc/dovecot/conf.d/custom-sieve/global_after.sieve sieve_before = /etc/dovecot/conf.d/custom-sieve/global_before.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment +vnd.dovecot.debug sieve_pipe_bin_dir = /usr/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service lmtp { inet_listener lmtp { address = 127.0.0.1 ::1 port = 24 } } ssl_cert =
-----Ursprüngliche Nachricht----- Von: John Fawcett via dovecot dovecot@dovecot.org Gesendet: Montag, 1. Juli 2024 23:16 An: dovecot@dovecot.org Betreff: Re: AW: AW: AW: IMAPSieve plugin will not run rspamd script
Hi Jens
changing random things is rarely a good way to solve these kinds of issues. My advice, if you don't need the inbox namespace prefix set for a specific reason, would be to go with the default prefix (ie. blank) and then set up the imapsieve rules as per your original post and repeat the test. If that does not work then post the debug logging and configuration from that test.
There could be multiple places where this is failing but the very first point is to have your sieve rules match. They will show something like the following in the logging. Notice the "Matched static mailbox rule" message. If that is not happening it's pointless to look further down the line into issues in the scripts themselves.
Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: mailbox INBOX: MOVE event
Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: Mailbox Spam: UID 2: Expunge requested
Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: duplicate db: Initialize
Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Pigeonhole version 0.5.21 (f6cd4b8e) initializing
Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include :global' scripts. Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.21 (f6cd4b8e) loaded Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.21 (f6cd4b8e) loaded Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: Static mailbox rule [1]: mailbox=Spam' from=*' causes=(COPY) => before=file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none)
Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Spam' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none)
Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: Matched static mailbox rule [2]
As for Debian, sorry I can't help on that, I'm using Fedora :-)
John
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
Hi Jens
that log looks to me like it is for moving a message from Inbox to Junk rather than Junk to Inbox.
The issue I see is that how you defined the rules does not match your namespace naming scheme. Event being received is the following. In that event you can see the name of the mailbox is "INBOX/Junk".
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: mailbox INBOX/Junk: APPEND eventHowever the rule you defined has the mailbox as "Junk".
Static mailbox rule [1]: mailbox=`Junk' from=`*' causes=(COPY) => before=`file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none)The logging you show confirms that the rule is not matching, else you would get a log line like the one I showed with the text "Debug: imapsieve: Matched static mailbox rule". So I have two possible alternative solutions, that in my tests I found both working:
(1) you define your inbox namespace with blank prefix instead of prefix=INBOX/
OR
(2) you adjust the rules as follows
imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = INBOX/Junk imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = INBOX/Junk imapsieve_mailbox2_name = *
If you apply one of those solutions and repeat the test unsuccessfully, it would be good to see the logging from that test. John
On 02/07/2024 11:28, postfix_dovecot--- via dovecot wrote:
Hi John!
You are of course absolutely right - wild experimentation is rarely a good approach. But you are surely familiar with the situation when you have done everything logical and then, in your desperation, you start to change things that you would not otherwise change ;)
I've actually gotten to the point where I'm going to give up on the idea and run a cron job at night. Then the spam won't be detected in real time - but what the heck?
As I wrote yesterday, I implemented a very popular tutorial for Debian 10 in Germany 1:1. The result was the same. Nevertheless, I reset my VM, where I do everything with the current releases, and attaching the log here. I would be very happy if it still works.
To be sure, I deleted the whole inbox and recreated the account on Outlook to create a fresh, new folder structure given by the Dovecot configuration. I also added the plugin vnd.dovecot.debug to the sieve_global_extensions line and added >debug_log "report-spam.sieve was running!";< to the report-spam.sieve file. So this string should appear in the log if the script were running at all. But it doesn't. So I still guess, it's not getting triggered.
That's the log if I move a mail from Junk (Spam) to Inbox:
# journalctl -u dovecot -f imap-login: Login: user=info@demo.example, method=PLAIN, rip=192.168.30.26, lip=192.168.30.34, mpid=1827, TLS, session=<dJVhwj4cJP3AqB4a> imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Loading modules from directory: /usr/lib/dovecot/modules imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Module loaded: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Effective uid=10000, gid=10000, home=/var/mail/vhosts/demo.example/info imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: open(/proc/self/io) failed: Permission denied imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: maildir++: root=/var/mail/vhosts/demo.example/info/Maildir, index=, indexpvt=, control=, inbox=/var/mail/vhosts/demo.example/info/Maildir, alt= imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Mailbox INBOX/Junk: Mailbox opened imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: mailbox INBOX/Junk: APPEND event imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: duplicate db: Initialize imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: Pigeonhole version 0.5.19 (4eae2f79) initializing imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include
:global' scripts. imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: Static mailbox rule [1]: mailbox=Junk' from=*' causes=(COPY) => before=file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none) imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Junk' causes=(COPY) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none) imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Mailbox INBOX/Junk: Mailbox opened imap(info@demo.example)<1825>: Debug: imapsieve: mailbox INBOX: FLAG event (changed flags: \Deleted) imap(info@demo.example)<1825> : Debug: Mailbox INBOX: UID 1: Expunge requested imap(info@demo.example)<1825> : Debug: Mailbox INBOX: UID 1: Mail expunged # doveconf -n # 2.3.19.1 (9b53102964): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 6.1.0-21-amd64 x86_64 Debian 12.5 # Hostname: ServerIV-home.demo.example auth_mechanisms = plain login mail_debug = yes mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { hidden = no ignore_on_failure = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX/ separator = / subscriptions = yes type = private } passdb { driver = pam } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * sieve =file:~/sieve;active=~/.dovecot.sieve sieve_after = /etc/dovecot/conf.d/custom-sieve/global_after.sieve sieve_before = /etc/dovecot/conf.d/custom-sieve/global_before.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment +vnd.dovecot.debug sieve_pipe_bin_dir = /usr/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service lmtp { inet_listener lmtp { address = 127.0.0.1 ::1 port = 24 } } ssl_cert =
-----Ursprüngliche Nachricht----- Von: John Fawcett via dovecotdovecot@dovecot.org
Gesendet: Montag, 1. Juli 2024 23:16 An:dovecot@dovecot.org Betreff: Re: AW: AW: AW: IMAPSieve plugin will not run rspamd scriptHi Jens
changing random things is rarely a good way to solve these kinds of issues. My advice, if you don't need the inbox namespace prefix set for a specific reason, would be to go with the default prefix (ie. blank) and then set up the imapsieve rules as per your original post and repeat the test. If that does not work then post the debug logging and configuration from that test.
There could be multiple places where this is failing but the very first point is to have your sieve rules match. They will show something like the following in the logging. Notice the "Matched static mailbox rule" message. If that is not happening it's pointless to look further down the line into issues in the scripts themselves.
Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: mailbox INBOX: MOVE event Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: Mailbox Spam: UID 2: Expunge requested Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: duplicate db: Initialize Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Pigeonhole version 0.5.21 (f6cd4b8e) initializing Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include
:global' scripts. Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.21 (f6cd4b8e) loaded Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.21 (f6cd4b8e) loaded Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: Static mailbox rule [1]: mailbox=Spam' from=*' causes=(COPY) => before=file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none) Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Spam' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none) Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: Matched static mailbox rule [2]As for Debian, sorry I can't help on that, I'm using Fedora :-)
John
dovecot mailing list --dovecot@dovecot.org To unsubscribe send an email todovecot-leave@dovecot.org
dovecot mailing list --dovecot@dovecot.org To unsubscribe send an email todovecot-leave@dovecot.org
Hi Jens
just one update and some more insight looking at the code
Your causes should be COPY APPEND (whether applying proposed solution (1) or (2). I updated inline below for solution (2).
Logic for that: I was testing from Thunderbird which is generating a MOVE event. In the imapsieve plugin code MOVE and COPY are treated almost the same, so my rule with COPY fire on MOVE too. In your case the event you're getting is APPEND, but none of your rules have that as a cause so won't match even if the mailbox matches.
John
On 02/07/2024 12:07, John Fawcett via dovecot wrote:
Hi Jens
that log looks to me like it is for moving a message from Inbox to Junk rather than Junk to Inbox.
The issue I see is that how you defined the rules does not match your namespace naming scheme. Event being received is the following. In that event you can see the name of the mailbox is "INBOX/Junk".
imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: mailbox INBOX/Junk: APPEND event
However the rule you defined has the mailbox as "Junk".
Static mailbox rule [1]: mailbox=
Junk' from=*' causes=(COPY) => before=`file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none)The logging you show confirms that the rule is not matching, else you would get a log line like the one I showed with the text "Debug: imapsieve: Matched static mailbox rule". So I have two possible alternative solutions, that in my tests I found both working:
(1) you define your inbox namespace with blank prefix instead of prefix=INBOX/
OR
(2) you adjust the rules as follows
imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY APPEND imapsieve_mailbox1_name = INBOX/Junk imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY APPEND imapsieve_mailbox2_from = INBOX/Junk imapsieve_mailbox2_name = *
If you apply one of those solutions and repeat the test unsuccessfully, it would be good to see the logging from that test. John
On 02/07/2024 11:28, postfix_dovecot--- via dovecot wrote:
Hi John!
You are of course absolutely right - wild experimentation is rarely a good approach. But you are surely familiar with the situation when you have done everything logical and then, in your desperation, you start to change things that you would not otherwise change ;)
I've actually gotten to the point where I'm going to give up on the idea and run a cron job at night. Then the spam won't be detected in real time - but what the heck?
As I wrote yesterday, I implemented a very popular tutorial for Debian 10 in Germany 1:1. The result was the same. Nevertheless, I reset my VM, where I do everything with the current releases, and attaching the log here. I would be very happy if it still works.
To be sure, I deleted the whole inbox and recreated the account on Outlook to create a fresh, new folder structure given by the Dovecot configuration. I also added the plugin vnd.dovecot.debug to the sieve_global_extensions line and added >debug_log "report-spam.sieve was running!";< to the report-spam.sieve file. So this string should appear in the log if the script were running at all. But it doesn't. So I still guess, it's not getting triggered.
That's the log if I move a mail from Junk (Spam) to Inbox:
# journalctl -u dovecot -f imap-login: Login: user=info@demo.example, method=PLAIN, rip=192.168.30.26, lip=192.168.30.34, mpid=1827, TLS, session=<dJVhwj4cJP3AqB4a> imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Loading modules from directory: /usr/lib/dovecot/modules imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Module loaded: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Effective uid=10000, gid=10000, home=/var/mail/vhosts/demo.example/info imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: open(/proc/self/io) failed: Permission denied imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: maildir++: root=/var/mail/vhosts/demo.example/info/Maildir, index=, indexpvt=, control=, inbox=/var/mail/vhosts/demo.example/info/Maildir, alt= imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Mailbox INBOX/Junk: Mailbox opened imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: mailbox INBOX/Junk: APPEND event imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: duplicate db: Initialize imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: Pigeonhole version 0.5.19 (4eae2f79) initializing imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include
:global' scripts. imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: Static mailbox rule [1]: mailbox=Junk' from=*' causes=(COPY) => before=file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none) imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Junk' causes=(COPY) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none) imap(info@demo.example)<1827><dJVhwj4cJP3AqB4a>: Debug: Mailbox INBOX/Junk: Mailbox opened imap(info@demo.example)<1825>: Debug: imapsieve: mailbox INBOX: FLAG event (changed flags: \Deleted) imap(info@demo.example)<1825> : Debug: Mailbox INBOX: UID 1: Expunge requested imap(info@demo.example)<1825> : Debug: Mailbox INBOX: UID 1: Mail expunged # doveconf -n # 2.3.19.1 (9b53102964): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 6.1.0-21-amd64 x86_64 Debian 12.5 # Hostname: ServerIV-home.demo.example auth_mechanisms = plain login mail_debug = yes mail_location = maildir:~/Maildir mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve namespace inbox { hidden = no ignore_on_failure = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX/ separator = / subscriptions = yes type = private } passdb { driver = pam } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * sieve =file:~/sieve;active=~/.dovecot.sieve sieve_after = /etc/dovecot/conf.d/custom-sieve/global_after.sieve sieve_before = /etc/dovecot/conf.d/custom-sieve/global_before.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment +vnd.dovecot.debug sieve_pipe_bin_dir = /usr/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service lmtp { inet_listener lmtp { address = 127.0.0.1 ::1 port = 24 } } ssl_cert =
-----Ursprüngliche Nachricht----- Von: John Fawcett via dovecotdovecot@dovecot.org Gesendet: Montag,
- Juli 2024 23:16 An:dovecot@dovecot.org Betreff: Re: AW: AW: AW: IMAPSieve plugin will not run rspamd script
Hi Jens
changing random things is rarely a good way to solve these kinds of issues. My advice, if you don't need the inbox namespace prefix set for a specific reason, would be to go with the default prefix (ie. blank) and then set up the imapsieve rules as per your original post and repeat the test. If that does not work then post the debug logging and configuration from that test.
There could be multiple places where this is failing but the very first point is to have your sieve rules match. They will show something like the following in the logging. Notice the "Matched static mailbox rule" message. If that is not happening it's pointless to look further down the line into issues in the scripts themselves.
Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: mailbox INBOX: MOVE event Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: Mailbox Spam: UID 2: Expunge requested Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: duplicate db: Initialize Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Pigeonhole version 0.5.21 (f6cd4b8e) initializing Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include
:global' scripts. Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.21 (f6cd4b8e) loaded Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.21 (f6cd4b8e) loaded Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: Static mailbox rule [1]: mailbox=Spam' from=*' causes=(COPY) => before=file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none) Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Spam' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none) Jul 01 22:57:55 imola.site24.it dovecot[1145722]: imap(test@site24.it)<1145777><mlKa2TUccLdOhguc>: Debug: imapsieve: Matched static mailbox rule [2]As for Debian, sorry I can't help on that, I'm using Fedora :-)
John
dovecot mailing list --dovecot@dovecot.org To unsubscribe send an email todovecot-leave@dovecot.org
dovecot mailing list --dovecot@dovecot.org To unsubscribe send an email todovecot-leave@dovecot.org
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
Gotcha!!!!!!!!
It looks like you nailed it John! (Log and config attached)
The log looks now very different and my log entry is also shown.
Now I can continue working at this point. Adjusting the events i.e.
The reason was actually simple - but I spent a few evenings searching and was blind. I could have figured it out on my own! Thank you so much John for took such care of this issue!!!!!
Jens
imap(info@demo.example): Debug: Loading modules from directory: /usr/lib/dovecot/modules
imap(info@demo.example): Debug: Module loaded: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so
imap(info@demo.example): Debug: Effective uid=10000, gid=10000, home=/var/mail/vhosts/demo.example/info
imap(info@demo.example): Debug: open(/proc/self/io) failed: Permission denied
imap(info@demo.example): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir
imap(info@demo.example): Debug: maildir++: root=/var/mail/vhosts/demo.example/info/Maildir, index=, indexpvt=, control=, inbox=/var/mail/vhosts/demo.example/info/Maildir, alt=
imap(info@demo.example): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none
imap(info@demo.example): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt=
imap(info@demo.example): Debug: Mailbox INBOX/Junk: Mailbox opened
imap(info@demo.example): Debug: imapsieve: mailbox INBOX/Junk: APPEND event
imap(info@demo.example): Debug: duplicate db: Initialize
imap(info@demo.example): Debug: sieve: Pigeonhole version 0.5.19 (4eae2f79) initializing
imap(info@demo.example): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts.
imap(info@demo.example): Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded
imap(info@demo.example): Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded
imap(info@demo.example): Debug: imapsieve: Static mailbox rule [1]: mailbox=INBOX/Junk' from=*' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none)
imap(info@demo.example): Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=INBOX/Junk' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none)
imap(info@demo.example): Debug: imapsieve: Matched static mailbox rule [2]
imap(info@demo.example): Debug: imapsieve: Matched static mailbox rule [1]
imap(info@demo.example): Debug: sieve: file storage: Using Sieve script path: /usr/lib/dovecot/sieve/report-spam.sieve
imap(info@demo.example): Debug: sieve: file storage: script: Opened script report-spam' from /usr/lib/dovecot/sieve/report-spam.sieve'
imap(info@demo.example): Debug: sieve: file storage: Using Sieve script path: /usr/lib/dovecot/sieve/report-ham.sieve
imap(info@demo.example): Debug: sieve: file storage: script: Opened script report-ham' from /usr/lib/dovecot/sieve/report-ham.sieve'
imap(info@demo.example): Debug: Mailbox INBOX/Junk: Mailbox opened
imap(info@demo.example): Debug: sieve: multi-script: Start execute sequence
imap(info@demo.example): Debug: sieve: Opening script 1 of 2 from `/usr/lib/dovecot/sieve/report-spam.sieve'
imap(info@demo.example): Debug: sieve: Loading script /usr/lib/dovecot/sieve/report-spam.sieve
imap(info@demo.example): Debug: sieve: Script binary /usr/lib/dovecot/sieve/report-spam.svbin successfully loaded
imap(info@demo.example): Debug: sieve: binary /usr/lib/dovecot/sieve/report-spam.svbin: save: not saving binary, because it is already stored
imap(info@demo.example): Debug: sieve: Executing script from `/usr/lib/dovecot/sieve/report-spam.svbin'
imap(info@demo.example): Debug: sieve: multi-script: Run script `/usr/lib/dovecot/sieve/report-spam.svbin'
imap(info@demo.example): Debug: sieve: Started running script `/usr/lib/dovecot/sieve/report-spam.svbin'
imap(info@demo.example): sieve: DEBUG: report-spam.sieve was running!
imap(info@demo.example): Debug: sieve: Finished running script `/usr/lib/dovecot/sieve/report-spam.svbin' (status=ok, resource usage: no usage recorded)
imap(info@demo.example): Debug: sieve: multi-script: Execute result
imap(info@demo.example): Debug: sieve: uid=3: Executing result (status=ok, commit=no)
imap(info@demo.example): Debug: sieve: uid=3: Starting execution of actions
imap(info@demo.example): Debug: sieve: uid=3: Executing actions
imap(info@demo.example): Debug: sieve: uid=3: Executing pipe action
imap(info@demo.example): Debug: sieve: action pipe: running program: sa-learn-spam.sh
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Created
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Pass environment: USER=info@demo.example
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Pass environment: HOME=/var/mail/vhosts/demo.example/info
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Pass environment: HOST=ServerIV-home.demo.example
imap(info@demo.example): Debug: Mailbox INBOX/Junk: UID 3: Opened mail because: mail stream
imap(info@demo.example): Debug: sieve: uid=3: Finished executing pipe action (status=ok, keep=implicit)
imap(info@demo.example): Debug: sieve: uid=3: Finished executing actions (status=ok, keep=implicit, executed=yes)
imap(info@demo.example): Debug: sieve: uid=3: Finished executing result (no commit, status=ok, keep=yes)
imap(info@demo.example): Debug: sieve: multi-script: Sequence active
imap(info@demo.example): Debug: sieve: Opening script 2 of 2 from `/usr/lib/dovecot/sieve/report-ham.sieve'
imap(info@demo.example): Debug: sieve: Loading script /usr/lib/dovecot/sieve/report-ham.sieve
imap(info@demo.example): Debug: sieve: Script binary /usr/lib/dovecot/sieve/report-ham.svbin successfully loaded
imap(info@demo.example): Debug: sieve: binary /usr/lib/dovecot/sieve/report-ham.svbin: save: not saving binary, because it is already stored
imap(info@demo.example): Debug: sieve: Executing script from `/usr/lib/dovecot/sieve/report-ham.svbin'
imap(info@demo.example): Debug: sieve: multi-script: Run script `/usr/lib/dovecot/sieve/report-ham.svbin'
imap(info@demo.example): Debug: sieve: Started running script `/usr/lib/dovecot/sieve/report-ham.svbin'
imap(info@demo.example): Debug: sieve: Finished running script `/usr/lib/dovecot/sieve/report-ham.svbin' (status=ok, resource usage: no usage recorded)
imap(info@demo.example): Debug: sieve: multi-script: Execute result
imap(info@demo.example): Debug: sieve: uid=3: Executing result (status=ok, commit=no)
imap(info@demo.example): Debug: sieve: uid=3: Starting execution of actions
imap(info@demo.example): Debug: sieve: uid=3: Executing actions
imap(info@demo.example): Debug: sieve: uid=3: Executing pipe action
imap(info@demo.example): Debug: sieve: action pipe: running program: sa-learn-ham.sh
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Created
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Pass environment: USER=info@demo.example
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Pass environment: HOME=/var/mail/vhosts/demo.example/info
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Pass environment: HOST=ServerIV-home.demo.example
imap(info@demo.example): Debug: sieve: uid=3: Finished executing pipe action (status=ok, keep=implicit)
imap(info@demo.example): Debug: sieve: uid=3: Finished executing actions (status=ok, keep=implicit, executed=yes)
imap(info@demo.example): Debug: sieve: uid=3: Finished executing result (no commit, status=ok, keep=yes)
imap(info@demo.example): Debug: sieve: multi-script: Sequence active
imap(info@demo.example): Debug: sieve: multi-script: Finishing sequence (status=ok)
imap(info@demo.example): Debug: sieve: uid=3: Executing result (status=ok, commit=yes)
imap(info@demo.example): Debug: sieve: uid=3: Starting execution of actions
imap(info@demo.example): Debug: sieve: uid=3: Executing actions
imap(info@demo.example): Debug: sieve: uid=3: Finished executing actions (status=ok, keep=implicit, executed=yes)
imap(info@demo.example): Debug: sieve: uid=3: Execute implicit keep (status=ok)
imap(info@demo.example): Debug: sieve: uid=3: Start storing into mailbox INBOX/Junk
imap(info@demo.example): Debug: sieve: uid=3: Executing implicit keep action
imap(info@demo.example): Debug: sieve: uid=3: Execute storing into mailbox 'INBOX/Junk'
imap(info@demo.example): Debug: Mailbox INBOX/Junk: Mailbox opened
imap(info@demo.example): Debug: sieve: uid=3: Updated existing mail in mailbox 'INBOX/Junk'
imap(info@demo.example): Debug: sieve: uid=3: Finished executing implicit keep action (status=ok)
imap(info@demo.example): Debug: sieve: uid=3: Finalizing actions
imap(info@demo.example): Debug: sieve: uid=3: Finalize pipe action (status=ok, action_status=ok, commit_status=ok, pre-commit=yes)
imap(info@demo.example): Debug: sieve: uid=3: Commit pipe action
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Establishing connection
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Forked child process
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Connected to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Finished streaming payload to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Finished input to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Disconnected
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Waiting for program to finish after 0 msecs (timeout = 10000 msecs)
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Child process ended
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Destroy
imap(info@demo.example): Debug: sieve: uid=3: pipe action: piped message to program `sa-learn-spam.sh'
imap(info@demo.example): Debug: sieve: uid=3: Finalize pipe action (status=ok, action_status=ok, commit_status=ok, pre-commit=yes)
imap(info@demo.example): Debug: sieve: uid=3: Commit pipe action
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Establishing connection
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Forked child process
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Connected to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Finished streaming payload to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Finished input to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Disconnected
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Waiting for program to finish after 0 msecs (timeout = 10000 msecs)
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Child process ended
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Destroy
imap(info@demo.example): Debug: sieve: uid=3: pipe action: piped message to program `sa-learn-ham.sh'
imap(info@demo.example): Debug: sieve: uid=3: Finished finalizing actions (status=ok, keep=implicit, committed=yes)
imap(info@demo.example): Debug: sieve: uid=3: Finalize implicit keep (status=ok)
imap(info@demo.example): Debug: sieve: uid=3: Finalize implicit keep action(status=ok, action_status=ok, commit_status=ok)
imap(info@demo.example): Debug: sieve: uid=3: Commit implicit keep action
imap(info@demo.example): Debug: sieve: uid=3: Commit storing into mailbox 'INBOX/Junk'
imap(info@demo.example): Debug: sieve: uid=3: left message in mailbox 'INBOX/Junk'
imap(info@demo.example): Debug: sieve: uid=3: Finish implicit keep action
imap(info@demo.example): Debug: sieve: uid=3: Finishing actions
imap(info@demo.example): Debug: sieve: uid=3: Finish pipe action
imap(info@demo.example): Debug: sieve: uid=3: Finish pipe action
imap(info@demo.example): Debug: sieve: uid=3: Finished executing result (final, status=ok, keep=yes)
imap(info@demo.example): Debug: sieve: multi-script: Sequence finished (status=ok, keep=yes)
imap(info@demo.example): Debug: sieve: multi-script: Destroy
imap(info@demo.example): Debug: Mailbox INBOX/Junk: Mailbox opened
imap(info@demo.example)<2107><Zu7qvEEcHyjAqB4a>: Debug: imapsieve: mailbox INBOX: FLAG event (changed flags: \Deleted)
imap(info@demo.example)<2107><Zu7qvEEcHyjAqB4a>: Debug: Mailbox INBOX: UID 3: Expunge requested
imap(info@demo.example)<2107><Zu7qvEEcHyjAqB4a>: Debug: Mailbox INBOX: UID 3: Mail expunged
#dovecot -n # 2.3.19.1 (9b53102964): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.19 (4eae2f79)
# OS: Linux 6.1.0-21-amd64 x86_64 Debian 12.5
# Hostname: ServerIV-home.demo.example
auth_mechanisms = plain login
mail_debug = yes
mail_location = maildir:~/Maildir
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
namespace inbox {
hidden = no
ignore_on_failure = no
inbox = yes
list = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts}
mailbox Junk {
auto = subscribe
special_use = \Junk}
mailbox Sent {
auto = subscribe
special_use = \Sent}
mailbox "Sent Messages" {
auto = subscribe
special_use = \Sent}
mailbox Trash {
auto = subscribe
special_use = \Trash}
prefix = INBOX/
separator = /
subscriptions = yes
type = private
}
passdb {
driver = pam
}
passdb {
args = scheme=CRYPT username_format=%u /etc/dovecot/users
driver = passwd-file
}
plugin {
imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-spam.sieve
imapsieve_mailbox1_causes = COPY APPEND
imapsieve_mailbox1_name = INBOX/Junk
imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve
imapsieve_mailbox2_causes = COPY APPEND
imapsieve_mailbox2_from = INBOX/Junk
imapsieve_mailbox2_name = *
sieve = file:~/sieve;active=~/.dovecot.sieve
sieve_after = /etc/dovecot/conf.d/custom-sieve/global_after.sieve
sieve_before = /etc/dovecot/conf.d/custom-sieve/global_before.sieve
sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment +vnd.dovecot.debug
sieve_pipe_bin_dir = /usr/lib/dovecot/sieve
sieve_plugins = sieve_imapsieve sieve_extprograms
}
protocols = imap lmtp sieve
service auth {
unix_listener /var/spool/postfix/private/auth {
mode = 0666}
}
service lmtp {
inet_listener lmtp {
address = 127.0.0.1 ::1
port = 24}
}
ssl_cert =
ssl_client_ca_dir = /etc/ssl/certs
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
userdb {
driver = passwd
}
userdb {
args = username_format=%u /etc/dovecot/users
driver = passwd-file
}
verbose_proctitle = yes
protocol lmtp {
mail_plugins = " sieve"
}
protocol imap {
mail_plugins = " imap_sieve"
}
-----Ursprüngliche Nachricht----- Von: John Fawcett via dovecot dovecot@dovecot.org Gesendet: Dienstag, 2. Juli 2024 12:34 An: dovecot@dovecot.org Betreff: Re: AW: AW: AW: AW: IMAPSieve plugin will not run rspamd script
Hi Jens
just one update and some more insight looking at the code
Your causes should be COPY APPEND (whether applying proposed solution
(1) or (2). I updated inline below for solution (2).
Logic for that: I was testing from Thunderbird which is generating a MOVE event. In the imapsieve plugin code MOVE and COPY are treated almost the same, so my rule with COPY fire on MOVE too. In your case the event you're getting is APPEND, but none of your rules have that as a cause so won't match even if the mailbox matches.
John
Hi Jens
that looks much better, though not yet completely solved. What you're now getting is both rules firing when you move a message to Junk, whereas you only want the first one firing. I guess you will have no rules firing when you move from Junk to elsewhere. My suspicion is that for APPEND event the target and destination mailboxes are set to the same value. i.e. APPEND event for moving to Junk will have mailbox=INBOX/Junk and from=INBOX/Junk which fires both rules, whereas when moving from Junk to INBOX mailbox=INBOX and from=INBOX, leading to KO for rule one on "mailbox" and KO on rule 2 for "from".
Static mailbox rule [1]: mailbox=INBOX/Junk' from=*' causes=(COPY
APPEND) => before=`file:/usr/lib/dovecot/sieve/report-spam.sieve'
after=(none)
Static mailbox rule [2]: mailbox=*' from=INBOX/Junk' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none)
Whenever I do the move of the message I get a MOVE (which is treated like COPY). I don't see these APPEND events. Not sure why your email client generates APPEND instead of MOVE. That could be a point to investigate. If you're going to have to manage APPEND events you'll probably need different rules. If you'll going to manage MOVE/COPY and APPEND events then you can probably combine sets of rules (adjusting the sequential numbering).
The following work for MOVE/COPY events
imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = INBOX/Junk imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = INBOX/Junk imapsieve_mailbox2_name = *
The following should work for APPEND events, but it won't support wildcards fo rule 2 else it will fire when moving to Junk as well as from Junk. I'm pretty sure you can't specify negation e.g. imapsieve_mailbox2_name = !INBOX/Junk, which is what would be needed to make it work for moving from Junk to any mailbox. As written rule 2 fires for moving from Junk to INBOX
imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = APPEND imapsieve_mailbox1_name = INBOX/Junk imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_name = INBOX
John
On 02/07/2024 14:51, postfix_dovecot--- via dovecot wrote:
Gotcha!!!!!!!!
It looks like you nailed it John! (Log and config attached)
The log looks now very different and my log entry is also shown.
Now I can continue working at this point. Adjusting the events i.e.
The reason was actually simple - but I spent a few evenings searching and was blind. I could have figured it out on my own! Thank you so much John for took such care of this issue!!!!!
Jens
imap(info@demo.example): Debug: Loading modules from directory: /usr/lib/dovecot/modules
imap(info@demo.example): Debug: Module loaded: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so
imap(info@demo.example): Debug: Effective uid=10000, gid=10000, home=/var/mail/vhosts/demo.example/info
imap(info@demo.example): Debug: open(/proc/self/io) failed: Permission denied
imap(info@demo.example): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir
imap(info@demo.example): Debug: maildir++: root=/var/mail/vhosts/demo.example/info/Maildir, index=, indexpvt=, control=, inbox=/var/mail/vhosts/demo.example/info/Maildir, alt=
imap(info@demo.example): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none
imap(info@demo.example): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt=
imap(info@demo.example): Debug: Mailbox INBOX/Junk: Mailbox opened
imap(info@demo.example): Debug: imapsieve: mailbox INBOX/Junk: APPEND event
imap(info@demo.example): Debug: duplicate db: Initialize
imap(info@demo.example): Debug: sieve: Pigeonhole version 0.5.19 (4eae2f79) initializing
imap(info@demo.example): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts.
imap(info@demo.example): Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded
imap(info@demo.example): Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.19 (4eae2f79) loaded
imap(info@demo.example): Debug: imapsieve: Static mailbox rule [1]: mailbox=
INBOX/Junk' from=*' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none)imap(info@demo.example): Debug: imapsieve: Static mailbox rule [2]: mailbox=
*' from=INBOX/Junk' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none)imap(info@demo.example): Debug: imapsieve: Matched static mailbox rule [2]
imap(info@demo.example): Debug: imapsieve: Matched static mailbox rule [1]
imap(info@demo.example): Debug: sieve: file storage: Using Sieve script path: /usr/lib/dovecot/sieve/report-spam.sieve
imap(info@demo.example): Debug: sieve: file storage: script: Opened script
report-spam' from/usr/lib/dovecot/sieve/report-spam.sieve'imap(info@demo.example): Debug: sieve: file storage: Using Sieve script path: /usr/lib/dovecot/sieve/report-ham.sieve
imap(info@demo.example): Debug: sieve: file storage: script: Opened script
report-ham' from/usr/lib/dovecot/sieve/report-ham.sieve'imap(info@demo.example): Debug: Mailbox INBOX/Junk: Mailbox opened
imap(info@demo.example): Debug: sieve: multi-script: Start execute sequence
imap(info@demo.example): Debug: sieve: Opening script 1 of 2 from `/usr/lib/dovecot/sieve/report-spam.sieve'
imap(info@demo.example): Debug: sieve: Loading script /usr/lib/dovecot/sieve/report-spam.sieve
imap(info@demo.example): Debug: sieve: Script binary /usr/lib/dovecot/sieve/report-spam.svbin successfully loaded
imap(info@demo.example): Debug: sieve: binary /usr/lib/dovecot/sieve/report-spam.svbin: save: not saving binary, because it is already stored
imap(info@demo.example): Debug: sieve: Executing script from `/usr/lib/dovecot/sieve/report-spam.svbin'
imap(info@demo.example): Debug: sieve: multi-script: Run script `/usr/lib/dovecot/sieve/report-spam.svbin'
imap(info@demo.example): Debug: sieve: Started running script `/usr/lib/dovecot/sieve/report-spam.svbin'
imap(info@demo.example): sieve: DEBUG: report-spam.sieve was running!
imap(info@demo.example): Debug: sieve: Finished running script `/usr/lib/dovecot/sieve/report-spam.svbin' (status=ok, resource usage: no usage recorded)
imap(info@demo.example): Debug: sieve: multi-script: Execute result
imap(info@demo.example): Debug: sieve: uid=3: Executing result (status=ok, commit=no)
imap(info@demo.example): Debug: sieve: uid=3: Starting execution of actions
imap(info@demo.example): Debug: sieve: uid=3: Executing actions
imap(info@demo.example): Debug: sieve: uid=3: Executing pipe action
imap(info@demo.example): Debug: sieve: action pipe: running program: sa-learn-spam.sh
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Created
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Pass environment:USER=info@demo.example
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Pass environment: HOME=/var/mail/vhosts/demo.example/info
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Pass environment: HOST=ServerIV-home.demo.example
imap(info@demo.example): Debug: Mailbox INBOX/Junk: UID 3: Opened mail because: mail stream
imap(info@demo.example): Debug: sieve: uid=3: Finished executing pipe action (status=ok, keep=implicit)
imap(info@demo.example): Debug: sieve: uid=3: Finished executing actions (status=ok, keep=implicit, executed=yes)
imap(info@demo.example): Debug: sieve: uid=3: Finished executing result (no commit, status=ok, keep=yes)
imap(info@demo.example): Debug: sieve: multi-script: Sequence active
imap(info@demo.example): Debug: sieve: Opening script 2 of 2 from `/usr/lib/dovecot/sieve/report-ham.sieve'
imap(info@demo.example): Debug: sieve: Loading script /usr/lib/dovecot/sieve/report-ham.sieve
imap(info@demo.example): Debug: sieve: Script binary /usr/lib/dovecot/sieve/report-ham.svbin successfully loaded
imap(info@demo.example): Debug: sieve: binary /usr/lib/dovecot/sieve/report-ham.svbin: save: not saving binary, because it is already stored
imap(info@demo.example): Debug: sieve: Executing script from `/usr/lib/dovecot/sieve/report-ham.svbin'
imap(info@demo.example): Debug: sieve: multi-script: Run script `/usr/lib/dovecot/sieve/report-ham.svbin'
imap(info@demo.example): Debug: sieve: Started running script `/usr/lib/dovecot/sieve/report-ham.svbin'
imap(info@demo.example): Debug: sieve: Finished running script `/usr/lib/dovecot/sieve/report-ham.svbin' (status=ok, resource usage: no usage recorded)
imap(info@demo.example): Debug: sieve: multi-script: Execute result
imap(info@demo.example): Debug: sieve: uid=3: Executing result (status=ok, commit=no)
imap(info@demo.example): Debug: sieve: uid=3: Starting execution of actions
imap(info@demo.example): Debug: sieve: uid=3: Executing actions
imap(info@demo.example): Debug: sieve: uid=3: Executing pipe action
imap(info@demo.example): Debug: sieve: action pipe: running program: sa-learn-ham.sh
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Created
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Pass environment:USER=info@demo.example
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Pass environment: HOME=/var/mail/vhosts/demo.example/info
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Pass environment: HOST=ServerIV-home.demo.example
imap(info@demo.example): Debug: sieve: uid=3: Finished executing pipe action (status=ok, keep=implicit)
imap(info@demo.example): Debug: sieve: uid=3: Finished executing actions (status=ok, keep=implicit, executed=yes)
imap(info@demo.example): Debug: sieve: uid=3: Finished executing result (no commit, status=ok, keep=yes)
imap(info@demo.example): Debug: sieve: multi-script: Sequence active
imap(info@demo.example): Debug: sieve: multi-script: Finishing sequence (status=ok)
imap(info@demo.example): Debug: sieve: uid=3: Executing result (status=ok, commit=yes)
imap(info@demo.example): Debug: sieve: uid=3: Starting execution of actions
imap(info@demo.example): Debug: sieve: uid=3: Executing actions
imap(info@demo.example): Debug: sieve: uid=3: Finished executing actions (status=ok, keep=implicit, executed=yes)
imap(info@demo.example): Debug: sieve: uid=3: Execute implicit keep (status=ok)
imap(info@demo.example): Debug: sieve: uid=3: Start storing into mailbox INBOX/Junk
imap(info@demo.example): Debug: sieve: uid=3: Executing implicit keep action
imap(info@demo.example): Debug: sieve: uid=3: Execute storing into mailbox 'INBOX/Junk'
imap(info@demo.example): Debug: Mailbox INBOX/Junk: Mailbox opened
imap(info@demo.example): Debug: sieve: uid=3: Updated existing mail in mailbox 'INBOX/Junk'
imap(info@demo.example): Debug: sieve: uid=3: Finished executing implicit keep action (status=ok)
imap(info@demo.example): Debug: sieve: uid=3: Finalizing actions
imap(info@demo.example): Debug: sieve: uid=3: Finalize pipe action (status=ok, action_status=ok, commit_status=ok, pre-commit=yes)
imap(info@demo.example): Debug: sieve: uid=3: Commit pipe action
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Establishing connection
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh: Forked child process
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Connected to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Finished streaming payload to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Finished input to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Disconnected
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Waiting for program to finish after 0 msecs (timeout = 10000 msecs)
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Child process ended
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-spam.sh (2120): Destroy
imap(info@demo.example): Debug: sieve: uid=3: pipe action: piped message to program `sa-learn-spam.sh'
imap(info@demo.example): Debug: sieve: uid=3: Finalize pipe action (status=ok, action_status=ok, commit_status=ok, pre-commit=yes)
imap(info@demo.example): Debug: sieve: uid=3: Commit pipe action
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Establishing connection
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh: Forked child process
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Connected to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Finished streaming payload to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Finished input to program
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Disconnected
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Waiting for program to finish after 0 msecs (timeout = 10000 msecs)
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Child process ended
imap(info@demo.example): Debug: program exec:/usr/lib/dovecot/sieve/sa-learn-ham.sh (2121): Destroy
imap(info@demo.example): Debug: sieve: uid=3: pipe action: piped message to program `sa-learn-ham.sh'
imap(info@demo.example): Debug: sieve: uid=3: Finished finalizing actions (status=ok, keep=implicit, committed=yes)
imap(info@demo.example): Debug: sieve: uid=3: Finalize implicit keep (status=ok)
imap(info@demo.example): Debug: sieve: uid=3: Finalize implicit keep action(status=ok, action_status=ok, commit_status=ok)
imap(info@demo.example): Debug: sieve: uid=3: Commit implicit keep action
imap(info@demo.example): Debug: sieve: uid=3: Commit storing into mailbox 'INBOX/Junk'
imap(info@demo.example): Debug: sieve: uid=3: left message in mailbox 'INBOX/Junk'
imap(info@demo.example): Debug: sieve: uid=3: Finish implicit keep action
imap(info@demo.example): Debug: sieve: uid=3: Finishing actions
imap(info@demo.example): Debug: sieve: uid=3: Finish pipe action
imap(info@demo.example): Debug: sieve: uid=3: Finish pipe action
imap(info@demo.example): Debug: sieve: uid=3: Finished executing result (final, status=ok, keep=yes)
imap(info@demo.example): Debug: sieve: multi-script: Sequence finished (status=ok, keep=yes)
imap(info@demo.example): Debug: sieve: multi-script: Destroy
imap(info@demo.example): Debug: Mailbox INBOX/Junk: Mailbox opened
imap(info@demo.example)<2107><Zu7qvEEcHyjAqB4a>: Debug: imapsieve: mailbox INBOX: FLAG event (changed flags: \Deleted)
imap(info@demo.example)<2107><Zu7qvEEcHyjAqB4a>: Debug: Mailbox INBOX: UID 3: Expunge requested
imap(info@demo.example)<2107><Zu7qvEEcHyjAqB4a>: Debug: Mailbox INBOX: UID 3: Mail expunged
#dovecot -n # 2.3.19.1 (9b53102964): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.19 (4eae2f79)
# OS: Linux 6.1.0-21-amd64 x86_64 Debian 12.5
# Hostname: ServerIV-home.demo.example
auth_mechanisms = plain login
mail_debug = yes
mail_location = maildir:~/Maildir
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve
namespace inbox {
hidden = no
ignore_on_failure = no
inbox = yes
list = yes
location =
mailbox Drafts {
auto = subscribe special_use = \Drafts}
mailbox Junk {
auto = subscribe special_use = \Junk}
mailbox Sent {
auto = subscribe special_use = \Sent}
mailbox "Sent Messages" {
auto = subscribe special_use = \Sent}
mailbox Trash {
auto = subscribe special_use = \Trash}
prefix = INBOX/
separator = /
subscriptions = yes
type = private
}
passdb {
driver = pam
}
passdb {
args = scheme=CRYPT username_format=%u /etc/dovecot/users
driver = passwd-file
}
plugin {
imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve
imapsieve_mailbox1_causes = COPY APPEND
imapsieve_mailbox1_name = INBOX/Junk
imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve
imapsieve_mailbox2_causes = COPY APPEND
imapsieve_mailbox2_from = INBOX/Junk
imapsieve_mailbox2_name = *
sieve =file:~/sieve;active=~/.dovecot.sieve
sieve_after = /etc/dovecot/conf.d/custom-sieve/global_after.sieve
sieve_before = /etc/dovecot/conf.d/custom-sieve/global_before.sieve
sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment +vnd.dovecot.debug
sieve_pipe_bin_dir = /usr/lib/dovecot/sieve
sieve_plugins = sieve_imapsieve sieve_extprograms
}
protocols = imap lmtp sieve
service auth {
unix_listener /var/spool/postfix/private/auth {
mode = 0666}
}
service lmtp {
inet_listener lmtp {
address = 127.0.0.1 ::1 port = 24}
}
ssl_cert =
ssl_client_ca_dir = /etc/ssl/certs
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
userdb {
driver = passwd
}
userdb {
args = username_format=%u /etc/dovecot/users
driver = passwd-file
}
verbose_proctitle = yes
protocol lmtp {
mail_plugins = " sieve"
}
protocol imap {
mail_plugins = " imap_sieve"
}
-----Ursprüngliche Nachricht----- Von: John Fawcett via dovecotdovecot@dovecot.org
Gesendet: Dienstag, 2. Juli 2024 12:34 An:dovecot@dovecot.org Betreff: Re: AW: AW: AW: AW: IMAPSieve plugin will not run rspamd scriptHi Jens
just one update and some more insight looking at the code
Your causes should be COPY APPEND (whether applying proposed solution
(1) or (2). I updated inline below for solution (2).
Logic for that: I was testing from Thunderbird which is generating a MOVE event. In the imapsieve plugin code MOVE and COPY are treated almost the same, so my rule with COPY fire on MOVE too. In your case the event you're getting is APPEND, but none of your rules have that as a cause so won't match even if the mailbox matches.
John
dovecot mailing list --dovecot@dovecot.org To unsubscribe send an email todovecot-leave@dovecot.org
(Resending because of size limit exceeded on previous post)
Hi Jens
that looks much better, though not yet completely solved. What you're now getting is both rules firing when you move a message to Junk, whereas you only want the first one firing. I guess you will have no rules firing when you move from Junk to elsewhere. My suspicion is that for APPEND event the target and destination mailboxes are set to the same value. i.e. APPEND event for moving to Junk will have mailbox=INBOX/Junk and from=INBOX/Junk which fires both rules, whereas when moving from Junk to INBOX mailbox=INBOX and from=INBOX, leading to KO for rule one on "mailbox" and KO on rule 2 for "from".
Static mailbox rule [1]: mailbox=INBOX/Junk' from=*' causes=(COPY
APPEND) => before=`file:/usr/lib/dovecot/sieve/report-spam.sieve'
after=(none)
Static mailbox rule [2]: mailbox=*' from=INBOX/Junk' causes=(COPY APPEND) => before=`file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none)
Whenever I do the move of the message I get a MOVE (which is treated like COPY). I don't see these APPEND events. Not sure why your email client generates APPEND instead of MOVE. That could be a point to investigate. If you're going to have to manage APPEND events you'll probably need different rules. If you'll going to manage MOVE/COPY and APPEND events then you can probably combine sets of rules (adjusting the sequential numbering).
The following work for MOVE/COPY events
imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = INBOX/Junk imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = INBOX/Junk imapsieve_mailbox2_name = *
The following should work for APPEND events, but it won't support wildcards fo rule 2 else it will fire when moving to Junk as well as from Junk. I'm pretty sure you can't specify negation e.g. imapsieve_mailbox2_name = !INBOX/Junk, which is what would be needed to make it work for moving from Junk to any mailbox. As written rule 2 fires for moving from Junk to INBOX
imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = APPEND imapsieve_mailbox1_name = INBOX/Junk imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_name = INBOX
John
On 02/07/2024 14:51, postfix_dovecot--- via dovecot wrote:
Gotcha!!!!!!!!
It looks like you nailed it John! (Log and config attached)
The log looks now very different and my log entry is also shown.
Now I can continue working at this point. Adjusting the events i.e.
The reason was actually simple - but I spent a few evenings searching and was blind. I could have figured it out on my own! Thank you so much John for took such care of this issue!!!!!
Jens
Hi John,
again you're hitting the point!
I put a "report-spam.sieve fired!" and "report-ham.sieve fired!" debug message in the scripts and monitored with "journalctl -u dovecot -f | grep fired". The APPEND event (move to junk) triggers BOTH scripts while the APPEND (move from junk to elsewhere) do nothing.
With your second listing it runs as expected. Now I will crawling some RFCs about the commands and a possible solution.
But I wonder am I the only one with this problem? I expected Outlook, even the older 2016, to be very popular?!
Jens
-----Ursprüngliche Nachricht----- Von: John Fawcett via dovecot dovecot@dovecot.org Gesendet: Dienstag, 2. Juli 2024 16:21 An: dovecot@dovecot.org Betreff: Re: AW: AW: AW: AW: AW: IMAPSieve plugin will not run rspamd script
(Resending because of size limit exceeded on previous post)
Hi Jens
that looks much better, though not yet completely solved. What you're now getting is both rules firing when you move a message to Junk, whereas you only want the first one firing. I guess you will have no rules firing when you move from Junk to elsewhere. My suspicion is that for APPEND event the target and destination mailboxes are set to the same value. i.e. APPEND event for moving to Junk will have mailbox=INBOX/Junk and from=INBOX/Junk which fires both rules, whereas when moving from Junk to INBOX mailbox=INBOX and from=INBOX, leading to KO for rule one on "mailbox" and KO on rule 2 for "from".
Static mailbox rule [1]: mailbox=INBOX/Junk' from=*' causes=(COPY APPEND) => before=file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none) Static mailbox rule [2]: mailbox=*' from=INBOX/Junk' causes=(COPY APPEND) => before=file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none)
Whenever I do the move of the message I get a MOVE (which is treated like COPY). I don't see these APPEND events. Not sure why your email client generates APPEND instead of MOVE. That could be a point to investigate. If you're going to have to manage APPEND events you'll probably need different rules. If you'll going to manage MOVE/COPY and APPEND events then you can probably combine sets of rules (adjusting the sequential numbering).
The following work for MOVE/COPY events
imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = INBOX/Junk imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = INBOX/Junk imapsieve_mailbox2_name = *
The following should work for APPEND events, but it won't support wildcards fo rule 2 else it will fire when moving to Junk as well as from Junk. I'm pretty sure you can't specify negation e.g. imapsieve_mailbox2_name = !INBOX/Junk, which is what would be needed to make it work for moving from Junk to any mailbox. As written rule 2 fires for moving from Junk to INBOX
imapsieve_mailbox1_before =file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = APPEND imapsieve_mailbox1_name = INBOX/Junk imapsieve_mailbox2_before =file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = APPEND imapsieve_mailbox2_name = INBOX
John
On 03/07/2024 20:53, postfix_dovecot--- via dovecot wrote:
Hi John,
again you're hitting the point!
I put a "report-spam.sieve fired!" and "report-ham.sieve fired!" debug message in the scripts and monitored with "journalctl -u dovecot -f | grep fired". The APPEND event (move to junk) triggers BOTH scripts while the APPEND (move from junk to elsewhere) do nothing.
With your second listing it runs as expected. Now I will crawling some RFCs about the commands and a possible solution.
But I wonder am I the only one with this problem? I expected Outlook, even the older 2016, to be very popular?!
Jens
Hi Jens
after posting I was thinking over the rule for APPENDing to Inbox and I did have my doubts on that. I think it will fire even if people move email from some other folder to Inbox or even potentially on receiving new email, so probably not an ideal solution. I think the only real solution is to have negative matching rules (which at the moment there isn't)
I don't have access to Outlook 2016. I have tried it with Outlook 1.2024.701.200 (which seems a very strange version number, but that's what it says) and I got a MOVE event just like Thunderbird. I think that a mail client should generate a MOVE event when moving a message between folders that are on the same server. I would only expect APPEND events if the move was happening for example between two different servers or between server and local folders. If you want to investigate further maybe the rawlog_dir feature would help to capture more info.
best of luck!
John
Hi John,
the ham script runs only if the target folder is the inbox. So, if a user moves an email from spam to any other folder, it will not be marked as ham. Meanwhile, I have spent so much time on this to know that it makes no sense to investigate further. I found a few websites describing the same issue with Outlook 2013 - 2016 (and even 2019), so I am stopping the work here to save my time. Regardless of the IMAP scripts, I will write a shell script that goes through all spam and ham folders once a night and sets the markings correctly.
Thank you again for your extraordinary support!
Jens
-----Ursprüngliche Nachricht----- Von: John Fawcett via dovecot dovecot@dovecot.org Gesendet: Mittwoch, 3. Juli 2024 22:25 An: dovecot@dovecot.org Betreff: Re: AW: AW: AW: AW: AW: AW: IMAPSieve plugin will not run rspamd script
On 03/07/2024 20:53, postfix_dovecot--- via dovecot wrote:
Hi John,
again you're hitting the point!
I put a "report-spam.sieve fired!" and "report-ham.sieve fired!" debug message in the scripts and monitored with "journalctl -u dovecot -f | grep fired". The APPEND event (move to junk) triggers BOTH scripts while the APPEND (move from junk to elsewhere) do nothing.
With your second listing it runs as expected. Now I will crawling some RFCs about the commands and a possible solution.
But I wonder am I the only one with this problem? I expected Outlook, even the older 2016, to be very popular?!
Jens
Hi Jens
after posting I was thinking over the rule for APPENDing to Inbox and I did have my doubts on that. I think it will fire even if people move email from some other folder to Inbox or even potentially on receiving new email, so probably not an ideal solution. I think the only real solution is to have negative matching rules (which at the moment there isn't)
I don't have access to Outlook 2016. I have tried it with Outlook 1.2024.701.200 (which seems a very strange version number, but that's what it says) and I got a MOVE event just like Thunderbird. I think that a mail client should generate a MOVE event when moving a message between folders that are on the same server. I would only expect APPEND events if the move was happening for example between two different servers or between server and local folders. If you want to investigate further maybe the rawlog_dir feature would help to capture more info.
best of luck!
John
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
participants (2)
-
John Fawcett
-
postfix_dovecot＠gmx.de