Replacement for antispam plugin
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy
On 10-02-17 09:06, Aki Tuomi wrote:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
Could you elaborate on the fact that it's deprecated? I never saw anything about that? The wiki page wasn't mentioning that it's deprecated since this morning?
Kind regards, Tom
On 10.02.2017 10:35, Tom Hendrikx wrote:
On 10-02-17 09:06, Aki Tuomi wrote:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
Could you elaborate on the fact that it's deprecated? I never saw anything about that? The wiki page wasn't mentioning that it's deprecated since this morning?
Kind regards, Tom
It has not been maintained for ages and we have decided to officially deprecate it now. We believe that IMAPSieve does this much better than this plugin.
Aki
Nice. Finally got around to switching. Had to add filter to my dspam pipe (another one I'm overdue for replacing) . Antispam plugin seems to have automatically converted line endings from "mail-type" '\r\n' into linux plain '\n' .
Like so: sed -e 's/\r$//' | dspam ...
Den 10. feb. 2017 09:06, skrev Aki Tuomi:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy
Aki Tuomi wrote on 10.02.2017 15:06:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy We just implemented this according to the wiki. Works great and without any hiccups. Thanks, Aki
Il 10/02/2017 09:06, Aki Tuomi ha scritto:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
imap_stats plugin is required?
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
Op 2/10/2017 om 2:20 PM schreef Alessio Cecchi:
Il 10/02/2017 09:06, Aki Tuomi ha scritto:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
imap_stats plugin is required?
No, that is just part of the example.
Regards,
Stephan.
On February 12, 2017 at 1:32 PM Stephan Bosch stephan@rename-it.nl wrote:
Op 2/10/2017 om 2:20 PM schreef Alessio Cecchi:
Il 10/02/2017 09:06, Aki Tuomi ha scritto:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
imap_stats plugin is required?
No, that is just part of the example.
Regards,
Stephan.
And removed from the example too, thank you for pointing this out.
Aki
On 10.02.2017 09:06, Aki Tuomi wrote:
Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve.
In my setup, I use the following sieve script globally for all users:
if header :is "X-Spam-Flag" "YES" { fileinto "Junk"; stop; }
This allows processing based on spam flags set by Amavis/SpamAssassin. I wonder if the method shown in
https://wiki.dovecot.org/HowTo/AntispamWithSieve
will cause incoming mail (via LMTP) that is already flagged as spam to be processed by report-spam.sieve and, in consequence, will be learned as spam for a second time, which would of course be undesirable?
-Ralph
On Fri, Feb 10, 2017 at 03:52:52PM +0100, Ralph Seichter wrote:
On 10.02.2017 09:06, Aki Tuomi wrote:
Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve.
In my setup, I use the following sieve script globally for all users:
if header :is "X-Spam-Flag" "YES" { fileinto "Junk"; stop; }
This allows processing based on spam flags set by Amavis/SpamAssassin. I wonder if the method shown in
https://wiki.dovecot.org/HowTo/AntispamWithSieve
will cause incoming mail (via LMTP) that is already flagged as spam to be processed by report-spam.sieve and, in consequence, will be learned as spam for a second time, which would of course be undesirable?
It shouldn't do. Check out https://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve, which explains that sieve is normally only used at delivery time, but the sieve_imapsieve plugin runs a *different* sieve script based on IMAP actions (for example, COPY).
So, when you deliver, your main script tells dovecot WHERE to deliver to. When you issue and IMAP COPY command, the Antispam scripts tell dovecot to pipe the message to spamassassin. The delivery does not involve IMAP and the IMAP COPY does not constitute redelivery.
-- For more information, please reread.
On Fri, Feb 10, 2017 at 5:09 PM, Darac Marjal mailinglist@darac.org.uk wrote:
On Fri, Feb 10, 2017 at 03:52:52PM +0100, Ralph Seichter wrote:
On 10.02.2017 09:06, Aki Tuomi wrote:
Since antispam plugin is deprecated and we would really prefer people
not to use it, we wrote instructions on how to replace it with IMAPSieve.
In my setup, I use the following sieve script globally for all users:
if header :is "X-Spam-Flag" "YES" { fileinto "Junk"; stop; }
This allows processing based on spam flags set by Amavis/SpamAssassin. I wonder if the method shown in
https://wiki.dovecot.org/HowTo/AntispamWithSieve
will cause incoming mail (via LMTP) that is already flagged as spam to be processed by report-spam.sieve and, in consequence, will be learned as spam for a second time, which would of course be undesirable?
It shouldn't do. Check out https://wiki.dovecot.org/Pigeo nhole/Sieve/Plugins/IMAPSieve, which explains that sieve is normally only used at delivery time, but the sieve_imapsieve plugin runs a *different* sieve script based on IMAP actions (for example, COPY).
So, when you deliver, your main script tells dovecot WHERE to deliver to. When you issue and IMAP COPY command, the Antispam scripts tell dovecot to pipe the message to spamassassin. The delivery does not involve IMAP and the IMAP COPY does not constitute redelivery.
-- For more information, please reread.
I think that this needs some change:
# From Spam folder to elsewhere imapsieve_mailbox2_name = * imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve
When a message from Spam is moved to Trash then the report-ham.sieve is being executed.
Can we add an exception for the Trash folder?
Thanks
-- George Kontostanos
On Fri, Feb 10, 2017 at 6:25 PM, George Kontostanos gkontos.mail@gmail.com wrote:
On Fri, Feb 10, 2017 at 5:09 PM, Darac Marjal mailinglist@darac.org.uk wrote:
On Fri, Feb 10, 2017 at 03:52:52PM +0100, Ralph Seichter wrote:
On 10.02.2017 09:06, Aki Tuomi wrote:
Since antispam plugin is deprecated and we would really prefer people
not to use it, we wrote instructions on how to replace it with IMAPSieve.
In my setup, I use the following sieve script globally for all users:
if header :is "X-Spam-Flag" "YES" { fileinto "Junk"; stop; }
This allows processing based on spam flags set by Amavis/SpamAssassin. I wonder if the method shown in
https://wiki.dovecot.org/HowTo/AntispamWithSieve
will cause incoming mail (via LMTP) that is already flagged as spam to be processed by report-spam.sieve and, in consequence, will be learned as spam for a second time, which would of course be undesirable?
It shouldn't do. Check out https://wiki.dovecot.org/Pigeo nhole/Sieve/Plugins/IMAPSieve, which explains that sieve is normally only used at delivery time, but the sieve_imapsieve plugin runs a *different* sieve script based on IMAP actions (for example, COPY).
So, when you deliver, your main script tells dovecot WHERE to deliver to. When you issue and IMAP COPY command, the Antispam scripts tell dovecot to pipe the message to spamassassin. The delivery does not involve IMAP and the IMAP COPY does not constitute redelivery.
-- For more information, please reread.
I think that this needs some change:
# From Spam folder to elsewhere imapsieve_mailbox2_name = * imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve
When a message from Spam is moved to Trash then the report-ham.sieve is being executed.
Can we add an exception for the Trash folder?
Thanks
-- George Kontostanos
Could we use something like this:
imapsieve_mailbox2_name = ! Trash
-- George Kontostanos
On February 10, 2017 at 9:25 AM George Kontostanos gkontos.mail@gmail.com wrote:
[snip]
I think that this needs some change:
# From Spam folder to elsewhere imapsieve_mailbox2_name = * imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve
When a message from Spam is moved to Trash then the report-ham.sieve is being executed.
Can we add an exception for the Trash folder?
This is handled in the sieve script. E.g.:
require "environment"; if environment "imap.mailbox" "Trash" { stop; }
michael
On 10.02.17 18:34, Michael Slusarz wrote:
Can we add an exception for the Trash folder?
This is handled in the sieve script. E.g.:
require "environment"; if environment "imap.mailbox" "Trash" { stop; }
This does not work for me, and I don't really expect it to work either. https://tools.ietf.org/html/rfc6785#section-4.4 states:
The implementation MUST set the Environment [RFC5183] item "imap.mailbox" to the name of the mailbox that the affected message is in, in the case of existing messages, or is targeted to be stored into, in the case of new messages.
The message already exists in the Spam folder, hence imap.mailbox should be "Spam" instead of "Trash", correct?
Is there perhaps another way to ensure that manually deleted spam is not erroneously learned as ham?
-Ralph
On February 10, 2017 at 12:13 PM Ralph Seichter wrote:
On 10.02.17 18:34, Michael Slusarz wrote:
Can we add an exception for the Trash folder? This is handled in the sieve script. E.g.:
require "environment"; if environment "imap.mailbox" "Trash" { stop; }
This does not work for me, and I don't really expect it to work either. https://tools.ietf.org/html/rfc6785#section-4.4 states:
The implementation MUST set the Environment [RFC5183] item "imap.mailbox" to the name of the mailbox that the affected message is in, in the case of existing messages, or is targeted to be stored into, in the case of new messages.
The message already exists in the Spam folder, hence imap.mailbox should be "Spam" instead of "Trash", correct?
Incorrect.
When you move a message to a new mailbox, that is a "new message" event (a new UID in the target mailbox is created; the message count increases). So imap.mailbox is set to the name of the *target* mailbox.
Is there perhaps another way to ensure that manually deleted spam is not erroneously learned as ham?
-Ralph
Hello,
Chiming in on this with a question, and will be getting to it over the weekend or later this evening time permitting.
Does retraining a message as either spam or ham alter message headers for example x-spam or the spamassassin-modified subject header?
If not is it possible to do so after processing? For example, I have a message inadvertently tagged as spam, from Spamassassin it gets an x-spam header added as well as a modified subject. Retraining that message as ham moving it to say any other folder but spam i'd like for that x-spam header to be set to as it is not spam, and the spamassassin subject to be removed. Is this doable?
Thanks. Dave.
On 2/10/17, Michael Slusarz michael.slusarz@dovecot.fi wrote:
On February 10, 2017 at 12:13 PM Ralph Seichter wrote:
On 10.02.17 18:34, Michael Slusarz wrote:
Can we add an exception for the Trash folder? This is handled in the sieve script. E.g.:
require "environment"; if environment "imap.mailbox" "Trash" { stop; }
This does not work for me, and I don't really expect it to work either. https://tools.ietf.org/html/rfc6785#section-4.4 states:
The implementation MUST set the Environment [RFC5183] item "imap.mailbox" to the name of the mailbox that the affected message is in, in the case of existing messages, or is targeted to be stored into, in the case of new messages.
The message already exists in the Spam folder, hence imap.mailbox should be "Spam" instead of "Trash", correct?
Incorrect.
When you move a message to a new mailbox, that is a "new message" event (a new UID in the target mailbox is created; the message count increases). So imap.mailbox is set to the name of the *target* mailbox.
Is there perhaps another way to ensure that manually deleted spam is not erroneously learned as ham?
-Ralph
Op 2/10/2017 om 8:46 PM schreef David Mehler:
Hello,
Chiming in on this with a question, and will be getting to it over the weekend or later this evening time permitting.
Does retraining a message as either spam or ham alter message headers for example x-spam or the spamassassin-modified subject header?
If not is it possible to do so after processing? For example, I have a message inadvertently tagged as spam, from Spamassassin it gets an x-spam header added as well as a modified subject. Retraining that message as ham moving it to say any other folder but spam i'd like for that x-spam header to be set to as it is not spam, and the spamassassin subject to be removed. Is this doable?
I think you could use the "editheader" extension in the report-ham.sieve script to do that. You'll need "variables" as well to modify the subject.
Regards,
Stephan.
On 10.02.17 20:34, Michael Slusarz wrote:
When you move a message to a new mailbox, that is a "new message" event (a new UID in the target mailbox is created; the message count increases). So imap.mailbox is set to the name of the *target* mailbox.
My tests seem to indicate otherwise. Deleting a message currently located in the Junk folder causes the report-ham.sieve script to be invoked, which in my case contains the following:
require ["vnd.dovecot.pipe", "copy", "environment"]; if environment "imap.mailbox" "Trash" { stop; } elsif environment "imap.mailbox" "Junk" { pipe :copy "debug-junk"; } else { pipe :copy "learn-ham"; }
I can see that "learn-ham" is always invoked when a message is deleted from or moved out of the Junk folder, so my guess is that imap.mailbox is neither "Trash" nor "Junk" ("debug-junk" is never called, according to the Dovecot logs). Unfortunately I don't know how to debug this further.
I don't want "learn-ham" to be run when a message located in the Junk folder is manually deleted or moved to Trash, and right now I don't know how to accomplish this.
-Ralph
On Fri, Feb 10, 2017 at 9:59 PM, Ralph Seichter dovecot-ml@seichter.de wrote:
On 10.02.17 20:34, Michael Slusarz wrote:
When you move a message to a new mailbox, that is a "new message" event (a new UID in the target mailbox is created; the message count increases). So imap.mailbox is set to the name of the *target* mailbox.
My tests seem to indicate otherwise. Deleting a message currently located in the Junk folder causes the report-ham.sieve script to be invoked, which in my case contains the following:
require ["vnd.dovecot.pipe", "copy", "environment"]; if environment "imap.mailbox" "Trash" { stop; } elsif environment "imap.mailbox" "Junk" { pipe :copy "debug-junk"; } else { pipe :copy "learn-ham"; }
I can see that "learn-ham" is always invoked when a message is deleted from or moved out of the Junk folder, so my guess is that imap.mailbox is neither "Trash" nor "Junk" ("debug-junk" is never called, according to the Dovecot logs). Unfortunately I don't know how to debug this further.
I don't want "learn-ham" to be run when a message located in the Junk folder is manually deleted or moved to Trash, and right now I don't know how to accomplish this.
-Ralph
Same problem here. As a workaround I tried the following:
# From Spam folder to Inbox imapsieve_mailbox2_name = Inbox imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve
In theory this should trigger the report-ham script, only if a mail is moved from Spam to Inbox. However, it does not seem to work either.
George Kontostanos
Op 2/10/2017 om 10:15 PM schreef George Kontostanos:
On Fri, Feb 10, 2017 at 9:59 PM, Ralph Seichter dovecot-ml@seichter.de wrote:
Same problem here. As a workaround I tried the following:
# From Spam folder to Inbox imapsieve_mailbox2_name = Inbox imapsieve_mailbox2_from = Spam imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve
In theory this should trigger the report-ham script, only if a mail is moved from Spam to Inbox. However, it does not seem to work either.
That is because you found a rather stupid, yet minor bug. The "INBOX" mailbox is rather special, since it is the only mailbox name that is treated case-insensitively. The normal form is "INBOX". Due to the bug, your "Inbox" will not match "INBOX". You can work around this for now by specifying "INBOX" instead. Well, you will not need this workaround for a workaround anyway if the fix I proposed in my earlier message also works for you. But for posterity...
Regards,
Stephan.
Op 2/10/2017 om 8:59 PM schreef Ralph Seichter:
On 10.02.17 20:34, Michael Slusarz wrote:
When you move a message to a new mailbox, that is a "new message" event (a new UID in the target mailbox is created; the message count increases). So imap.mailbox is set to the name of the *target* mailbox. My tests seem to indicate otherwise. Deleting a message currently located in the Junk folder causes the report-ham.sieve script to be invoked, which in my case contains the following:
require ["vnd.dovecot.pipe", "copy", "environment"]; if environment "imap.mailbox" "Trash" { stop; } elsif environment "imap.mailbox" "Junk" { pipe :copy "debug-junk"; } else { pipe :copy "learn-ham"; }
I can see that "learn-ham" is always invoked when a message is deleted from or moved out of the Junk folder, so my guess is that imap.mailbox is neither "Trash" nor "Junk" ("debug-junk" is never called, according to the Dovecot logs). Unfortunately I don't know how to debug this further.
I don't want "learn-ham" to be run when a message located in the Junk folder is manually deleted or moved to Trash, and right now I don't know how to accomplish this.
The "imap.mailbox" environment is the empty string in this case. Why? Well, the Sieve interpreter does not know about it, since the "imapsieve" extension is not activated in the require line.
You could debug this with the non-standard "vnd.dovecot.debug" extension. I debugged it like this:
require ["imapsieve", "environment", "variables", "vnd.dovecot.debug"];
if environment :matches "imap.mailbox" "*" { set "mailbox" "${1}"; }
debug_log "imap.mailbox = ${mailbox}";
if string "${mailbox}" "Trash" { stop; }
debug_log "REPORT HAM";
This will log one or two info messages, depending on what the target mailbox is.
I tested this with the example configuration and it works fine. I've verified this by invoking the imap service directly from the command line, so that it is immediately logged-in and spews al debug messages directly to stderr:
$ sudo /usr/lib/dovecot/imap -u harrie imap(harrie)<>: Debug: Loading modules from directory: /usr/lib/dovecot/modules imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mail_log_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_replication_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib20_virtual_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so imap(harrie)<>: Debug: Module loaded: /usr/lib/dovecot/modules/lib95_imap_sieve_plugin.so imap(harrie)<>: Debug: auth USER input: harrie uid=1000 gid=124 home=/ext/test/home/test/harrie Debug: Effective uid=1000, gid=124, home=/ext/test/home/test/harrie Debug: replication: No mail_replica setting - replication disabled Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Debug: maildir++: root=/ext/test/home/test/harrie/Maildir, index=, indexpvt=, control=, inbox=/ext/test/home/test/harrie/Maildir, alt= Debug: Namespace : type=private, prefix=virtual/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=yes location=virtual:~/Maildir/virtual Debug: fs: root=/ext/test/home/test/harrie/Maildir/virtual, index=, indexpvt=, control=, inbox=, alt= permissions from /ext/test/home/test/harrie/Maildir: mode=0700 gid=default
- PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE LITERAL+ SEARCH=FUZZY NOTIFY URLAUTH URLAUTH=BINARY METADATA SPECIAL-USE] Logged in as harrie 23423 SELECT Spam imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: Namespace : Using
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
- 1 EXISTS
- 1 RECENT
- OK [UNSEEN 1] First unseen.
- OK [UIDVALIDITY 1485122806] UIDs valid
- OK [UIDNEXT 2] Predicted next UID 23423 OK [READ-WRITE] Select completed (0.000 + 0.000 secs).
This is what happens for moving from Spam to a non-Trash mailbox:
2343 MOVE 1 "Frop"
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: imapsieve: mailbox Frop:
MOVE event
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Pigeonhole version
0.5.devel (462a535) initializing
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: include:
sieve_global is not set; it is currently not possible to include
:global' scripts. imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Pigeonhole Sieve PGP Encrypt plugin version 0.2.devel loaded imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.devel (462a535) loaded imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: imapsieve: Static mailbox rule [1]: mailbox=Spam' from=*' causes=(COPY) => before=file:/usr/lib/dovecot/sieve/report-spam.sieve' after=(none)
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: imapsieve: Static mailbox
rule [2]: mailbox=*' from=Spam' causes=(COPY) =>
before=file:/usr/lib/dovecot/sieve/report-ham.sieve' after=(none) imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: imapsieve: Matched static mailbox rule [2] imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Performing auto-detection imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Root exists (/ext/test/home/test/harrie) imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Storage path /ext/test/home/test/harrie/sieve' not found
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Active
script path is unconfigured; using default (path=~/.dovecot.sieve)
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Using
Sieve script path: /ext/test/home/test/harrie/.dovecot.sieve
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage:
Storage path /ext/test/home/test/harrie/.dovecot.sieve' not found imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: storage: No default script location configured imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Using Sieve script path: /usr/lib/dovecot/sieve/report-ham.sieve imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file script: Opened script report-ham' from /usr/lib/dovecot/sieve/report-ham.sieve' imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Opening script 1 of 1 from /usr/lib/dovecot/sieve/report-ham.sieve'
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Loading script
/usr/lib/dovecot/sieve/report-ham.sieve
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Script binary
/usr/lib/dovecot/sieve/report-ham.svbin successfully loaded
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: binary save: not
saving binary /usr/lib/dovecot/sieve/report-ham.svbin, because it is
already stored
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Executing script
from `/usr/lib/dovecot/sieve/report-ham.svbin'
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Info: sieve: report-ham: line 11:
DEBUG: imap.mailbox = Frop
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Info: sieve: report-ham: line 17:
DEBUG: REPORT HAM
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Info: copy from Spam: box=Frop,
uid=2, msgid=201701181731.v0IHVjvo001751@localhost.localdomain, flags=()
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Info: expunge: box=Spam, uid=1,
msgid=201701181731.v0IHVjvo001751@localhost.localdomain, flags=(\Recent)
- OK [COPYUID 1485122805 1 2] Moved UIDs.
- 1 EXPUNGE
- 0 RECENT 2343 OK Move completed (0.089 + 0.000 + 0.088 secs).
Alternatively, towards Trash:
2343 MOVE 1 Trash
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug:
/ext/test/home/test/harrie/Maildir/.Trash/dovecot.index.cache:
Compressed, file_seq changed 0 -> 1486901076, size=32, max_uid=0
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: imapsieve: mailbox Trash:
MOVE event
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: Mailbox Trash: Opened mail
UID=0 because: header Message-ID (Cache file is empty, reset_id=1486901076)
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: imapsieve: Matched static
mailbox rule [2]
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage:
Performing auto-detection
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Root
exists (/ext/test/home/test/harrie)
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage:
Storage path /ext/test/home/test/harrie/sieve' not found imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Active script path is unconfigured; using default (path=~/.dovecot.sieve) imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Using Sieve script path: /ext/test/home/test/harrie/.dovecot.sieve imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Storage path /ext/test/home/test/harrie/.dovecot.sieve' not found
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: storage: No default
script location configured
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file storage: Using
Sieve script path: /usr/lib/dovecot/sieve/report-ham.sieve
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: file script: Opened
script report-ham' from /usr/lib/dovecot/sieve/report-ham.sieve'
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Opening script 1 of
1 from /usr/lib/dovecot/sieve/report-ham.sieve' imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Loading script /usr/lib/dovecot/sieve/report-ham.sieve imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Script binary /usr/lib/dovecot/sieve/report-ham.svbin successfully loaded imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: binary save: not saving binary /usr/lib/dovecot/sieve/report-ham.svbin, because it is already stored imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Debug: sieve: Executing script from /usr/lib/dovecot/sieve/report-ham.svbin'
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Info: sieve: report-ham: line 11:
DEBUG: imap.mailbox = Trash
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Info: copy from Spam: box=Trash,
uid=1, msgid=201701181731.v0IHVjvo001751@localhost.localdomain, flags=()
imap(harrie)</fX4JwpPoFhq2AAAmoV78g>: Info: expunge: box=Spam, uid=2,
msgid=201701181731.v0IHVjvo001751@localhost.localdomain, flags=(\Recent)
- OK [COPYUID 1485122804 2 1] Moved UIDs.
- 1 EXPUNGE
- 0 RECENT 2343 OK Move completed (0.090 + 0.000 + 0.089 secs).
If you want to debug this in even more detail, you can enable trace debugging for the targeted user (or the whole server if you really want to). The imapsieve plugins should obey this as well:
https://github.com/dovecot/pigeonhole/blob/master/INSTALL#L551
This is apparently not currently documented in the wiki.
Regards,
Stephan.
On 12.02.2017 13:25, Stephan Bosch wrote:
The "imap.mailbox" environment is the empty string in this case. Why? Well, the Sieve interpreter does not know about it, since the "imapsieve" extension is not activated in the require line.
Now there's a facepalm moment. ;-) Thank you, with a modified 'require' statement things are working for me.
I see that https://wiki.dovecot.org/HowTo/AntispamWithSieve has already been updated, that's nice.
You could debug this with the non-standard "vnd.dovecot.debug" extension.
Thanks again, I will keep this in mind for future debugging.
-Ralph
Den 12. feb. 2017 15:12, skrev Ralph Seichter:
On 12.02.2017 13:25, Stephan Bosch wrote:
The "imap.mailbox" environment is the empty string in this case. Why? Well, the Sieve interpreter does not know about it, since the "imapsieve" extension is not activated in the require line.
Now there's a facepalm moment. ;-) Thank you, with a modified 'require' statement things are working for me.
I see that https://wiki.dovecot.org/HowTo/AntispamWithSieve has already been updated, that's nice.
Tried to add a small variation for dSpam, but there is some anti-spam functionality that I don't understand.
Mind putting this in after the sa-learn-ham.sh: ?
Or, if you are using dspam, (dropping 'sa-' as that would be misleading)
learn-spam.sh
{{{ #!/bin/sh
sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=spam }}}
learn-ham.sh
{{{ #!/bin/sh
sed -e 's/\r$//' | /usr/bin/dspam --debug --source=error --class=innocent }}}
You could debug this with the non-standard "vnd.dovecot.debug" extension.
Thanks again, I will keep this in mind for future debugging.
-Ralph
On 10.02.2017 16:09, Darac Marjal wrote:
Check out https://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve, which explains that sieve is normally only used at delivery time, but the sieve_imapsieve plugin runs a *different* sieve script based on IMAP actions (for example, COPY).
Thanks for the pointer, Darac. The following confirms that imapsieve_* is not involved during delivery: "Note that the imapsieve extension can only be used in a Sieve script that is invoked from IMAP. When it is used in the active delivery script, it will cause runtime errors."
-Ralph
On Feb 11, 2017, at 12:50 AM, Ralph Seichter dovecot-ml@seichter.de wrote:
Check out https://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve,
My concern is, will you experience any lag while moving message?
According to the doc, this plugin works like “pipe” backend of old “antispam” plugin, when you move a message from INBOX to Junk, the antispam plugin calls sa-learn, and you will experience noticeable lag on webmail (i used Roundcube webmail for testing) until sa-learn finished. It's much worse if you move multiple emails at the same time, because we have to wait for sa-learn to learn all moved messages.
I switched to “spool2dir” backend of old “antispam” plugin, and call sa-learn hourly to learn spam/ham. Since it simply copies moved message(s), no lag on webmail side at all, our users are satisfied.
Zhang Huangbin, founder of iRedMail project: http://www.iredmail.org/ Time zone: GMT+8 (China/Beijing).
On 10.02.17 18:22, Zhang Huangbin wrote:
My concern is, will you experience any lag while moving message?
I don't use direct calls to sa-learn, but store the piped e-mails on disk, and a periodic cron-job picks them up and invokes sa-learn. This way, there is no noticeable lag.
-Ralph
On February 10, 2017 at 10:06 AM Aki Tuomi aki.tuomi@dovecot.fi wrote:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy
Hi everyone,
thank you all for your feedback, questions and comments. We have upgraded the documentation based on this, including information how to exclude Trash folder in ham script.
Aki
On Sun, Feb 12, 2017 at 3:52 PM, Aki Tuomi aki.tuomi@dovecot.fi wrote:
On February 10, 2017 at 10:06 AM Aki Tuomi aki.tuomi@dovecot.fi wrote:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy
Hi everyone,
thank you all for your feedback, questions and comments. We have upgraded the documentation based on this, including information how to exclude Trash folder in ham script.
Aki
Thank you all very much. I am now running into a very weird issue. Whenever an Junk email is “seen” I get the following error:
Debug: sieve: vnd.dovecot.execute extension: no bin or socket directory
specified; extension is unconfigured (both sieve_execute_bin_dir and
sieve_execute_socket_dir are not set)
Feb 12 18:02:54 imap(user@domain): Debug: imapsieve: Static mailbox rule
[1]: mailbox=Junk' from=*' causes=(COPY) =>
before=file:/usr/local/lib/dovecot/sieve/report-spam.sieve' after=(none) Feb 12 18:02:54 imap(user@dmain): Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Junk' causes=(COPY) => before=file:/usr/local/lib/dovecot/sieve/report-ham.sieve' after=(none)
Also it automatically creates a .spamassassin/ folder in the user:
-rw------- 1 vmail vmail 136 Feb 10 17:33 .dovecot.lda-dupes lrwx------ 1 vmail vmail 17 Mar 13 2016 .dovecot.sieve@ -> managesieve.sieve -rw------- 1 vmail vmail 322 Feb 11 03:02 .dovecot.svbin drwx------ 2 vmail vmail 512 Feb 12 18:04 .spamassassin/ drwx------ 5 vmail vmail 512 Feb 10 17:32 Drafts/ drwx------ 5 vmail vmail 512 Feb 12 18:09 Junk/ drwx------ 5 vmail vmail 512 Feb 11 17:36 Saved/ drwx------ 5 vmail vmail 512 Jan 16 11:55 Sent/ drwx------ 5 vmail vmail 512 Feb 12 18:09 Trash/ drwx------ 2 vmail vmail 1024 Feb 12 18:07 cur/ -rw------- 1 vmail vmail 21 Mar 13 2016 dovecot-acl-list -rw------- 1 vmail vmail 68 Oct 17 13:19 dovecot-keywords -rw------- 1 vmail vmail 245 Feb 12 18:07 dovecot-uidlist -rw------- 1 vmail vmail 8 Jun 9 2016 dovecot-uidvalidity -r--r--r-- 1 vmail vmail 0 Mar 12 2016 dovecot-uidvalidity.56e48129 -rw------- 1 vmail vmail 640 Feb 11 22:24 dovecot.index -rw------- 1 vmail vmail 27788 Feb 12 18:07 dovecot.index.cache -rw------- 1 vmail vmail 25996 Feb 12 18:07 dovecot.index.log -rw------- 1 vmail vmail 192 Feb 12 16:04 dovecot.mailbox.log -rw------- 1 vmail vmail 181 Feb 10 17:34 managesieve.sieve drwx------ 2 vmail vmail 512 Feb 12 17:46 new/ -rw------- 1 vmail vmail 39 Jun 9 2016 subscriptions drwx------ 2 vmail vmail 512 Feb 12 18:04 tmp/
Any ideas ? I am attaching my config.
Thanks
# 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: FreeBSD 10.3-RELEASE-p11 amd64 ufs
auth_mechanisms = plain login
auth_verbose = yes
default_client_limit = 2560
default_process_limit = 512
dict {
acl = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext
quota = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf.ext
}
log_path = /var/log/dovecot.log
mail_home = /usr/local/vhosts/mail/%d/%n
mail_location = maildir:/usr/local/vhosts/mail/%d/%n:LAYOUT=fs
mail_max_userip_connections = 20
mail_plugins = quota acl
mail_privileged_group = vmail
mail_shared_explicit_inbox = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date index ihave
duplicate mime foreverypart extracttext imapsieve
mbox_write_locks = fcntl
namespace {
inbox = no
list = children
location =
maildir:/usr/local/vhosts/mail/%%d/%%n:LAYOUT=fs:INDEX=/usr/local/vhosts/indexes/%d/%n/shared/%%u:INDEXPVT=/usr/local/vhosts/indexes/%d/%n/shared/%%u
prefix = shared/%%d/%%n/
separator = /
subscriptions = no
type = shared
}
namespace inbox {
inbox = yes
list = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
separator = /
type = private
}
passdb {
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
acl = vfile
acl_shared_dict = proxy::acl
imapsieve_mailbox1_before =
file:/usr/local/lib/dovecot/sieve/report-spam.sieve
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_name = Junk
imapsieve_mailbox2_before =
file:/usr/local/lib/dovecot/sieve/report-ham.sieve
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_from = Junk
imapsieve_mailbox2_name = *
quota = dict:User quota::proxy::quota
quota_rule2 = Trash:storage=+100M
sieve = /usr/local/vhosts/mail/%d/%n/.dovecot.sieve
sieve_before = /usr/local/vhosts/sieve/before.d/
sieve_dir = /usr/local/vhosts/mail/%d/%n
sieve_global_dir = /usr/local/vhosts/sieve/%d
sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute
sieve_global_path = /usr/local/vhosts/sieve/%d/default.sieve
sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve
sieve_plugins = sieve_imapsieve sieve_extprograms
}
protocols = imap lmtp sieve sieve
service auth-worker {
user = vmail
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-userdb {
mode = 0600
user = vmail
}
user = dovecot
}
service dict {
unix_listener dict {
mode = 0600
user = vmail
}
}
service imap-login {
inet_listener imap {
port = 143
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
process_min_avail = 0
service_count = 1
vsz_limit = 64 M
}
ssl_cert =
George Kontostanos
On 12.02.2017 17:36, George Kontostanos wrote:
it automatically creates a .spamassassin/ folder in the user
That happens because sa-learn is invoked as the user who is logged into IMAP. If you want all users to contribute to a global SpamAssassin database (like I do), you'll need to create your own learning mechanism instead of calling sa-learn directly.
-Ralph
On Sun, Feb 12, 2017 at 7:52 PM, Ralph Seichter dovecot-ml@seichter.de wrote:
On 12.02.2017 17:36, George Kontostanos wrote:
it automatically creates a .spamassassin/ folder in the user
That happens because sa-learn is invoked as the user who is logged into IMAP. If you want all users to contribute to a global SpamAssassin database (like I do), you'll need to create your own learning mechanism instead of calling sa-learn directly.
-Ralph
Actually I think that sa-learn is invoked as user vmail. But of course I might be wrong.
sa-learn-ham.sh
LOG='/var/log/sa-learn.log' exec /usr/local/bin/sa-learn --ham -D >> $LOG 2>&1
I had to give ownership to vmail to sa-learn.log otherwise it would refuse to run.
Do you have any suggestions as per the way sa-learn should be executed?
Thanks for your help.
-- George Kontostanos
On 12.02.17 19:05, George Kontostanos wrote:
Actually I think that sa-learn is invoked as user vmail. But of course I might be wrong.
It might depend on system configuration. On my servers, Sieve scripts are definitely executed as the OS user that matches the current IMAP user.
Do you have any suggestions as per the way sa-learn should be executed?
Instead of calling sa-learn directly, I use a script to store the piped raw message in the file system, using separate directories for ham and spam. This way, there are no delays. A periodic Cron job, running as my global SpamAssassin user, later collects the files and invokes sa-learn, ensuring that sync operations only happen once per run.
-Ralph
Op 2/12/2017 om 5:36 PM schreef George Kontostanos:
On Sun, Feb 12, 2017 at 3:52 PM, Aki Tuomi aki.tuomi@dovecot.fi wrote:
On February 10, 2017 at 10:06 AM Aki Tuomi aki.tuomi@dovecot.fi wrote:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy Hi everyone,
thank you all for your feedback, questions and comments. We have upgraded the documentation based on this, including information how to exclude Trash folder in ham script.
Aki
Thank you all very much. I am now running into a very weird issue. Whenever an Junk email is “seen” I get the following error:
Debug: sieve: vnd.dovecot.execute extension: no bin or socket directory specified; extension is unconfigured (both sieve_execute_bin_dir and sieve_execute_socket_dir are not set) Feb 12 18:02:54 imap(user@domain): Debug: imapsieve: Static mailbox rule [1]: mailbox=
Junk' from=*' causes=(COPY) => before=file:/usr/local/lib/dovecot/sieve/report-spam.sieve' after=(none) Feb 12 18:02:54 imap(user@dmain): Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Junk' causes=(COPY) => before=file:/usr/local/lib/dovecot/sieve/report-ham.sieve' after=(none)
That is not an error. That is merely a debug message indicating that the vnd.dovecot.execute extension is unconfigured.
Don't enable that extension if you're not using the "execute" command/test. Yes, it is in the example, but it has no function either. Only the vnd.dovecot.pipe extension is actually used ("pipe" command).
Also it automatically creates a .spamassassin/ folder in the user:
-rw------- 1 vmail vmail 136 Feb 10 17:33 .dovecot.lda-dupes lrwx------ 1 vmail vmail 17 Mar 13 2016 .dovecot.sieve@ -> managesieve.sieve -rw------- 1 vmail vmail 322 Feb 11 03:02 .dovecot.svbin drwx------ 2 vmail vmail 512 Feb 12 18:04 .spamassassin/ drwx------ 5 vmail vmail 512 Feb 10 17:32 Drafts/ drwx------ 5 vmail vmail 512 Feb 12 18:09 Junk/ drwx------ 5 vmail vmail 512 Feb 11 17:36 Saved/ drwx------ 5 vmail vmail 512 Jan 16 11:55 Sent/ drwx------ 5 vmail vmail 512 Feb 12 18:09 Trash/ drwx------ 2 vmail vmail 1024 Feb 12 18:07 cur/ -rw------- 1 vmail vmail 21 Mar 13 2016 dovecot-acl-list -rw------- 1 vmail vmail 68 Oct 17 13:19 dovecot-keywords -rw------- 1 vmail vmail 245 Feb 12 18:07 dovecot-uidlist -rw------- 1 vmail vmail 8 Jun 9 2016 dovecot-uidvalidity -r--r--r-- 1 vmail vmail 0 Mar 12 2016 dovecot-uidvalidity.56e48129 -rw------- 1 vmail vmail 640 Feb 11 22:24 dovecot.index -rw------- 1 vmail vmail 27788 Feb 12 18:07 dovecot.index.cache -rw------- 1 vmail vmail 25996 Feb 12 18:07 dovecot.index.log -rw------- 1 vmail vmail 192 Feb 12 16:04 dovecot.mailbox.log -rw------- 1 vmail vmail 181 Feb 10 17:34 managesieve.sieve drwx------ 2 vmail vmail 512 Feb 12 17:46 new/ -rw------- 1 vmail vmail 39 Jun 9 2016 subscriptions drwx------ 2 vmail vmail 512 Feb 12 18:04 tmp/
Any ideas ? I am attaching my config.
The likely scenario is that the sa-learn tool is creating a hidden directory inside the $HOME directory of the user for user-specific state information. According to your mail_home and mail_location configuration, that is the same directory as the INBOX mailbox. That is why this may be interpreted as a mailbox by the maildir format. It is generally a bad idea to have those equal; you should put the mail location in a sub-directory of the home directory to prevent problems like this.
https://wiki.dovecot.org/VirtualUsers/Home
Regards,
Stephan.
On Sun, Feb 12, 2017 at 8:56 PM, Stephan Bosch stephan@rename-it.nl wrote:
Op 2/12/2017 om 5:36 PM schreef George Kontostanos:
On Sun, Feb 12, 2017 at 3:52 PM, Aki Tuomi aki.tuomi@dovecot.fi wrote:
On February 10, 2017 at 10:06 AM Aki Tuomi aki.tuomi@dovecot.fi wrote:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy Hi everyone,
thank you all for your feedback, questions and comments. We have upgraded the documentation based on this, including information how to exclude Trash folder in ham script.
Aki
Thank you all very much. I am now running into a very weird issue. Whenever an Junk email is “seen” I get the following error:
Debug: sieve: vnd.dovecot.execute extension: no bin or socket directory specified; extension is unconfigured (both sieve_execute_bin_dir and sieve_execute_socket_dir are not set) Feb 12 18:02:54 imap(user@domain): Debug: imapsieve: Static mailbox rule [1]: mailbox=
Junk' from=*' causes=(COPY) => before=file:/usr/local/lib/dovecot/sieve/report-spam.sieve' after=(none) Feb 12 18:02:54 imap(user@dmain): Debug: imapsieve: Static mailbox rule [2]: mailbox=*' from=Junk' causes=(COPY) => before=file:/usr/local/lib/dovecot/sieve/report-ham.sieve' after=(none)That is not an error. That is merely a debug message indicating that the vnd.dovecot.execute extension is unconfigured.
Don't enable that extension if you're not using the "execute" command/test. Yes, it is in the example, but it has no function either. Only the vnd.dovecot.pipe extension is actually used ("pipe" command).
Also it automatically creates a .spamassassin/ folder in the user:
-rw------- 1 vmail vmail 136 Feb 10 17:33 .dovecot.lda-dupes lrwx------ 1 vmail vmail 17 Mar 13 2016 .dovecot.sieve@ -> managesieve.sieve -rw------- 1 vmail vmail 322 Feb 11 03:02 .dovecot.svbin drwx------ 2 vmail vmail 512 Feb 12 18:04 .spamassassin/ drwx------ 5 vmail vmail 512 Feb 10 17:32 Drafts/ drwx------ 5 vmail vmail 512 Feb 12 18:09 Junk/ drwx------ 5 vmail vmail 512 Feb 11 17:36 Saved/ drwx------ 5 vmail vmail 512 Jan 16 11:55 Sent/ drwx------ 5 vmail vmail 512 Feb 12 18:09 Trash/ drwx------ 2 vmail vmail 1024 Feb 12 18:07 cur/ -rw------- 1 vmail vmail 21 Mar 13 2016 dovecot-acl-list -rw------- 1 vmail vmail 68 Oct 17 13:19 dovecot-keywords -rw------- 1 vmail vmail 245 Feb 12 18:07 dovecot-uidlist -rw------- 1 vmail vmail 8 Jun 9 2016 dovecot-uidvalidity -r--r--r-- 1 vmail vmail 0 Mar 12 2016 dovecot-uidvalidity.56e48129 -rw------- 1 vmail vmail 640 Feb 11 22:24 dovecot.index -rw------- 1 vmail vmail 27788 Feb 12 18:07 dovecot.index.cache -rw------- 1 vmail vmail 25996 Feb 12 18:07 dovecot.index.log -rw------- 1 vmail vmail 192 Feb 12 16:04 dovecot.mailbox.log -rw------- 1 vmail vmail 181 Feb 10 17:34 managesieve.sieve drwx------ 2 vmail vmail 512 Feb 12 17:46 new/ -rw------- 1 vmail vmail 39 Jun 9 2016 subscriptions drwx------ 2 vmail vmail 512 Feb 12 18:04 tmp/
Any ideas ? I am attaching my config.
The likely scenario is that the sa-learn tool is creating a hidden directory inside the $HOME directory of the user for user-specific state information. According to your mail_home and mail_location configuration, that is the same directory as the INBOX mailbox. That is why this may be interpreted as a mailbox by the maildir format. It is generally a bad idea to have those equal; you should put the mail location in a sub-directory of the home directory to prevent problems like this.
https://wiki.dovecot.org/VirtualUsers/Home
Regards,
Stephan.
That makes perfect sense. I might need to find a way to migrate now to the correct structure.
Thanks
-- George Kontostanos
Glad I poked around on the list today!
Thanks to all for the suggestions about integration with dspam.
I'll definitely have to look into this, as I rely on moving messages to a specific folder with various IMAP clients to retrain dspam false positives and negatives.
A quick pair of questions:
Does Dovecot support the IMAP "MOVE" command at this time?
If so, what is the syntax for "COPY or MOVE" for the _causes variables?
I did see messages from 2011 discussing it, but nothing since.
While the script looks like it could be modified for use with dspam (with the great suggestions from others on the list), it has the same problem as "antispam" with bulk moves being serialized and tying of the client until they complete. I'll probably have to break down and look into using FreeBSD's auditd to trigger the actions and then de-queue the successfully processed messages.
Sieve doesn't look like it can handle asynchronous processing, but I'd certainly be interested if I'm missing something there. One less thing to configure and maintain!
Jeff
On 2/12/17 5:52 AM, Aki Tuomi wrote:
On February 10, 2017 at 10:06 AM Aki Tuomi aki.tuomi@dovecot.fi wrote:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy Hi everyone,
thank you all for your feedback, questions and comments. We have upgraded the documentation based on this, including information how to exclude Trash folder in ham script.
Aki
Any opinion on dspam's interoperability with this?
On Fri, Feb 10, 2017 at 1:07 AM -0700, "Aki Tuomi" aki.tuomi@dovecot.fi wrote:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy
Den 12. feb. 2017 21:44, skrev ebroch@whitehorsetc.com:
Any opinion on dspam's interoperability with this?
Just follow the wiki and replace sa-learn scripts with calling dspam. Dspam direct pipe needs mail-line-endings (\r\n) translated into unix line-endings (\r).
like so:
----- learn-spam.sh --------- #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=spam
----- learn-ham.sh ---------- #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=innocent
Theese will be invoked with the owner of the mailbox as current uid, so that is all.
On 2017-02-12 23:00:49 +0100, Håkon Alstadheim wrote:
Just follow the wiki and replace sa-learn scripts with calling dspam. Dspam direct pipe needs mail-line-endings (\r\n) translated into unix line-endings (\r).
tbh ... what do you do about mails which just use \r as separator?
s|\r\n|\n| is safer.
darixlike so:
----- learn-spam.sh --------- #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=spam
----- learn-ham.sh ---------- #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=innocent
Theese will be invoked with the owner of the mailbox as current uid, so that is all.
-- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org
Den 12. feb. 2017 23:56, skrev Marcus Rueckert:
On 2017-02-12 23:00:49 +0100, Håkon Alstadheim wrote:
Just follow the wiki and replace sa-learn scripts with calling dspam. Dspam direct pipe needs mail-line-endings (\r\n) translated into unix line-endings (\r). I had a typo here, (\r\n) gets replaced by just (\n).
tbh ... what do you do about mails which just use \r as separator?
You are mistaken.
Firstly: s/\r$// will remove \r from the end of any line, do nothing if there is no \r. Secondly: All line-endings as seen by transfer-agents is transferred with the same line endings (MTA-MTA: \r\n; locally: usually the same but may be different). "Line ending" here pertains to after each header, and between message-parts. If there are naked (\n) characters in the message body, that is of no concern. Point of filter is to make Dspam-signature parseable for dspam. If there is an (\r) at the end of the header, Dspam will not find a match, and be unable to reclassify the mail.
s|\r\n|\n| is safer.
darixlike so:
----- learn-spam.sh --------- #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=spam
----- learn-ham.sh ---------- #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=innocent
Theese will be invoked with the owner of the mailbox as current uid, so that is all.
Op 2/12/2017 om 11:00 PM schreef Håkon Alstadheim:
Den 12. feb. 2017 21:44, skrev ebroch@whitehorsetc.com:
Any opinion on dspam's interoperability with this?
Just follow the wiki and replace sa-learn scripts with calling dspam. Dspam direct pipe needs mail-line-endings (\r\n) translated into unix line-endings (\r).
like so:
----- learn-spam.sh --------- #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=spam
----- learn-ham.sh ---------- #!/bin/sh sed -e 's/\r$//' | /usr/bin/dspam --source=error --class=innocent
Theese will be invoked with the owner of the mailbox as current uid, so that is all.
Actually, Pigeonhole should be able to do that too:
https://github.com/dovecot/pigeonhole/blob/master/doc/plugins/sieve_extprogr...
Yes, I need to update the wiki.
Regards,
Stephan.
On 02/12/2017 05:28 PM, Stephan Bosch wrote:
Actually, Pigeonhole should be able to do that too:
https://github.com/dovecot/pigeonhole/blob/master/doc/plugins/sieve_extprogr...
Yes, I need to update the wiki.
Regards,
Stephan.
For DSPAM, with --client, one also needs a --user set. http://hg.dovecot.org/dovecot-antispam-plugin/file/5ebc6aae4d7c/src/dspam.c did this.
Is there a way to feed this into the scripts mentioned? I imagine this is imap.user or imap.email, but how would one pass it to the script?
Thank you.
Trever
Greetings to the list :)
I've been meaning to ask a couple of questions about the imapsieve for antispam.
Firstly, I guess that the example at the bottom of the page "https://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve" is meant to link to the new example at "https://wiki2.dovecot.org/HowTo/AntispamWithSieve" now?
Also... in "imapsieve_mailboxX_name = ${directory}" is the string a literal, or is it working with the 'special use' flagged directories from the xx-mailboxes.conf? I'm asking as though I only have a few users to worry about, different mail clients each have their favourite special folders. IE thunderbird uses "junk" but one Outlook express version likes "Junk E-mail" and I have a different favourite from another version of OE too. So in mailboxes.conf I just added extra special use lines which seemed to make things happy. Otherwise I ended up with the server configured 'spam' folder showing as basic folder, and then each client adding it's own chosen spamfolder each time too.
But, if the imapsieve is only matching to literal foldernames, should I just duplicate the trigger lines for each type of junk folder or is there a method to have the sieve script enumerate all the options listed by 'special use' or is there a better method for this? I want to put the spam-mail-filing script as a global sieve script as all users will need it, rather than duplicating out for each user.
Many thanks ,
Andy R
On 10/02/2017 09:06, Aki Tuomi wrote:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy
Some answers inline.
Aki
On 01.03.2017 00:42, Andy R wrote:
Greetings to the list :)
I've been meaning to ask a couple of questions about the imapsieve for antispam.
Firstly, I guess that the example at the bottom of the page "https://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve" is meant to link to the new example at "https://wiki2.dovecot.org/HowTo/AntispamWithSieve" now?
Thanks, fixed.
Also... in "imapsieve_mailboxX_name = ${directory}" is the string a literal, or is it working with the 'special use' flagged directories from the xx-mailboxes.conf? I'm asking as though I only have a few users to worry about, different mail clients each have their favourite special folders. IE thunderbird uses "junk" but one Outlook express version likes "Junk E-mail" and I have a different favourite from another version of OE too. So in mailboxes.conf I just added extra special use lines which seemed to make things happy. Otherwise I ended up with the server configured 'spam' folder showing as basic folder, and then each client adding it's own chosen spamfolder each time too.
It is a mailbox mask, not special use flag.
But, if the imapsieve is only matching to literal foldernames, should I just duplicate the trigger lines for each type of junk folder or is there a method to have the sieve script enumerate all the options listed by 'special use' or is there a better method for this? I want to put the spam-mail-filing script as a global sieve script as all users will need it, rather than duplicating out for each user.
There is no way to match special use folders at the moment, but I like the idea.
Many thanks ,
Andy R
On 10/02/2017 09:06, Aki Tuomi wrote:
Hi! Since antispam plugin is deprecated and we would really prefer people not to use it, we wrote instructions on how to replace it with IMAPSieve. Comments and suggestions are most welcome.
https://wiki.dovecot.org/HowTo/AntispamWithSieve
Aki Tuomi Dovecot oy
On Wed Mar 01 2017 03:11:48 GMT-0500 (Eastern Standard Time), Aki Tuomi aki.tuomi@dovecot.fi wrote:
But, if the imapsieve is only matching to literal foldernames, should I just duplicate the trigger lines for each type of junk folder or is there a method to have the sieve script enumerate all the options listed by 'special use' or is there a better method for this? I want to put the spam-mail-filing script as a global sieve script as all users will need it, rather than duplicating out for each user.
There is no way to match special use folders at the moment, but I like the idea.
If by 'match' you mean, basically, a way to define aliases for different special use folders to a single mailbox name, I suggested this a long time ago, and love the idea.
Hopefully you'll at least add this to your official 'ToDo' (or 'maybe ToDo' list?
:) Thanks
participants (18)
-
Aki Tuomi
-
Alessio Cecchi
-
Andy R
-
Darac Marjal
-
David Mehler
-
ebroch＠whitehorsetc.com
-
George Kontostanos
-
Håkon Alstadheim
-
Jeff Kletsky
-
Marcus Rueckert
-
Michael Slusarz
-
Ralph Seichter
-
Stephan Bosch
-
Tamsy
-
Tanstaafl
-
Tom Hendrikx
-
Trever L. Adams
-
Zhang Huangbin