[Dovecot] A Dovecot Sieve spam filter question.
Hi,
I'd like to make a filtering threshold for users to let them deal with spamassassin spam-level starred < 8 themselves, but spam-level starred higher than 8 should be discarded
how can one do that with the "anyof" option ?
the following doesn't work with a test email with "gtube", which scores > 999 :
# require "fileinto"; if header :matches "X-Spam-Level" "\*\*\*\*\*\*" { fileinto "discard"; }
and I think it would be nice to have not so many lines in the globalsieverc file.
TIA, Regards, Adrian
aja-lists@tni.org wrote:
I'd like to make a filtering threshold for users to let them deal with spamassassin spam-level starred < 8 themselves, but spam-level starred higher than 8 should be discarded
how can one do that with the "anyof" option ?
the following doesn't work with a test email with "gtube", which scores > 999 :
# require "fileinto"; if header :matches "X-Spam-Level" "\*\*\*\*\*\*" { fileinto "discard"; }
and I think it would be nice to have not so many lines in the globalsieverc file.
Try the following:
if header :contains "X-Spam-Level" "*****" { ...
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu Common sense is the collection of prejudices acquired by age eighteen. -- Albert Einstein
On 12/30/2009 08:30 PM, Andrzej Adam Filip wrote:
aja-lists@tni.org wrote:
I'd like to make a filtering threshold for users to let them deal with spamassassin spam-level starred< 8 themselves, but spam-level starred higher than 8 should be discarded --- cut --- Try the following:
if header :contains "X-Spam-Level" "*****" {
Thanks, I've just tested that with the gtube spam test, ( which gives this amount of stars : X-Spam-Level: ************************************************** )
and the email ends up in the inbox.
And i've tried this also : http://pastebin.ca/1732221
Which fails with : Error: sieve: main_script: line 18: cannot nest tests deeper than 32 level
And I read about the spam-test option, but Dovecot Sieve does not support that yet (http://pigeonhole.dovecot.org).
Regards, Adrian
- aja-lists@tni.org [2009-12-30 21:41]:
On 12/30/2009 08:30 PM, Andrzej Adam Filip wrote:
aja-lists@tni.org wrote:
I'd like to make a filtering threshold for users to let them deal with spamassassin spam-level starred< 8 themselves, but spam-level starred higher than 8 should be discarded --- cut --- Try the following:
if header :contains "X-Spam-Level" "*****" {
Thanks, I've just tested that with the gtube spam test, ( which gives this amount of stars : X-Spam-Level: ************************************************** )
and the email ends up in the inbox.
Don't use
fileinto "discard";
to discard message. You're telling the server to deliver message to folder "discard" and it can't find it and so deliveres to INBOX.
Try
if header :contains "X-Spam-Level" "*****" { discard; stop; }
-- Kirill
"Kirill Miazine" km@krot.org wrote:
- aja-lists@tni.org [2009-12-30 21:41]:
On 12/30/2009 08:30 PM, Andrzej Adam Filip wrote:
aja-lists@tni.org wrote:
I'd like to make a filtering threshold for users to let them deal with spamassassin spam-level starred< 8 themselves, but spam-level starred higher than 8 should be discarded --- cut --- Try the following:
if header :contains "X-Spam-Level" "*****" {
Thanks, I've just tested that with the gtube spam test, ( which gives this amount of stars : X-Spam-Level: ************************************************** )
and the email ends up in the inbox.
Don't use
fileinto "discard";
to discard message. You're telling the server to deliver message to folder "discard" and it can't find it and so deliveres to INBOX.
Try
if header :contains "X-Spam-Level" "*****" { discard; stop; }
I would recommend delivery to special folder with auto-expire (e.g. set to 2-4 weeks) instead of discarding.
I have seen (a few years ago) VERY HIGH SpamAssassin scores when on some mailing list there had been discussion about DNS domain used by "ugly spammer".
http://en.wikipedia.org/wiki/Shit_happens
--
[pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu
If you hype something and it succeeds, you're a genius -- it wasn't a hype.
If you hype it and it fails, then it was just a hype.
-- Neil Bogart
aja-lists@tni.org wrote:
On 12/30/2009 08:30 PM, Andrzej Adam Filip wrote:
aja-lists@tni.org wrote:
I'd like to make a filtering threshold for users to let them deal with spamassassin spam-level starred< 8 themselves, but spam-level starred higher than 8 should be discarded --- cut --- Try the following:
if header :contains "X-Spam-Level" "*****" {
Thanks, I've just tested that with the gtube spam test, ( which gives this amount of stars : X-Spam-Level: ************************************************** )
and the email ends up in the inbox.
And i've tried this also : http://pastebin.ca/1732221
Which fails with : Error: sieve: main_script: line 18: cannot nest tests deeper than 32 level
As I understand ":contains" tests presence of the sub-string => so you can test 'at least n consecutive stars present' ('n stars OR more') => there is no need for complicated anyof like in http://pastebin.ca/1732221
And I read about the spam-test option, but Dovecot Sieve does not support that yet (http://pigeonhole.dovecot.org).
-- [pl>en: Andrew] Andrzej Adam Filip : anfi@onet.eu break; /* don't do magic till later */ -- Larry Wall in stab.c from the perl source code
On 12/30/2009 10:08 PM, Andrzej Adam Filip wrote:
As I understand ":contains" tests presence of the sub-string => so you can test 'at least n consecutive stars present' ('n stars OR more')
Okay, thanks, after some searching I see that quite some people apparently have this working successully, for example here :
http://www.cs.uchicago.edu/info/services/imap/sieve
But this (2nd line is one long line) :
require ["fileinto"]; if header :contains "X-Spam-Level" "**************************************************" { fileinto "Junk"; }
simply delivers the gtube test email in the Inbox instead of the Junk folder :( No errors in the dovecot-deliver log, what am I missing ?
All other Sieve scripts via AvelSieve work without problems (Although for some reason AvelSieve repeatedly sees all mail-folders except "Junk" folder), except this X-Spam-Level one doesn't.
=> there is no need for complicated anyof like in http://pastebin.ca/1732221
Hmm, yes, indeed, I misunderstood some remark in some other posting I found. :)
On 12/30/2009 11:56 PM, aja-lists@tni.org wrote:
But this (2nd line is one long line) :
require ["fileinto"]; if header :contains "X-Spam-Level" "**************************************************" { fileinto "Junk"; }
simply delivers the gtube test email in the Inbox instead of the Junk folder :( No errors in the dovecot-deliver log, what am I missing ?
My mistake, my personal .dovecot.sieve was in the way. (Found this : http://workaround.org/ispmail/lenny/server-side-sieve-filtering#comment-215 )
Problem solved :) Thanks for your help everyone!
aja-lists@tni.org wrote:
On 12/30/2009 11:56 PM, aja-lists@tni.org wrote:
But this (2nd line is one long line) :
require ["fileinto"]; if header :contains "X-Spam-Level" "**************************************************" { fileinto "Junk"; }
simply delivers the gtube test email in the Inbox instead of the Junk folder :( No errors in the dovecot-deliver log, what am I missing ?
My mistake, my personal .dovecot.sieve was in the way. (Found this : http://workaround.org/ispmail/lenny/server-side-sieve-filtering#comment-215 )
You can enforce global Sieve rules using the sieve_before setting:
http://wiki.dovecot.org/LDA/Sieve/Dovecot#multiscript
Regards,
Stephan.
On 12/30/2009 5:56 PM, aja-lists@tni.org wrote:
On 12/30/2009 10:08 PM, Andrzej Adam Filip wrote:
As I understand ":contains" tests presence of the sub-string => so you can test 'at least n consecutive stars present' ('n stars OR more')
Okay, thanks, after some searching I see that quite some people apparently have this working successully, for example here :
http://www.cs.uchicago.edu/info/services/imap/sieve
But this (2nd line is one long line) :
require ["fileinto"]; if header :contains "X-Spam-Level" "**************************************************" { fileinto "Junk"; }
simply delivers the gtube test email in the Inbox instead of the Junk folder :( No errors in the dovecot-deliver log, what am I missing ?
Probably because after filing it into the Junk folder you also want to issue a "stop;" statement to prevent further down rules from firing.
{ fileinto "Junk"; stop; }
(It's a common error that I make all the time.)
aja-lists@tni.org wrote:
And i've tried this also : http://pastebin.ca/1732221
Gross! :)
Which fails with : Error: sieve: main_script: line 18: cannot nest tests deeper than 32 level
Tests should be delimited with a ',' within an anyof() expression if you really want to write something like this. Now you are nesting tests, making the parser complain about the nesting level even before the compiler notices that the comma's are missing. Maybe I should make the compiler give a hint about such situations... or not.. :)
And I read about the spam-test option, but Dovecot Sieve does not support that yet (http://pigeonhole.dovecot.org).
It's in the works, but not quite finished yet. Particularly the configuration of this beast and including that configuration in the test suite is not trivial.
Regards,
-- Stephan Bosch stephan@rename-it.nl
On 12/30/2009 2:21 PM, aja-lists@tni.org wrote:
Hi,
I'd like to make a filtering threshold for users to let them deal with spamassassin spam-level starred < 8 themselves, but spam-level starred higher than 8 should be discarded
In general, it's better to quarantine high-scoring spam (we shove it in a server-side Junk folder) then to simply discard. (The old adage of mail delivery is that once you accept delivery of mail into your system you should never silently drop it on the floor.)
require ["comparator-i;ascii-numeric","fileinto","relational"]; # Definite spam gets shoved into the "Junk" folder in IMAP # Currently defined as a Spam Assassin score of 8.0 or higher if allof ( header :contains "X-Spam-Flag" "YES", header :value "ge" :comparator "i;ascii-numeric" ["X-Spam-Score"] ["8"] ) { fileinto "Junk"; stop; }
You need to check both that the spam flag is set to YES in addition to doing a comparison on the value of the spam score header. Otherwise you'll find that spams with negative scores can confuse the comparison rule.
This script is in a central sieve file that we include from the individual user's home folders. We always make sure that it's the *first* include in the user's file (after the "require" lines) so that we get a chance to stop processing on spam messages before processing things like vacation responses.
Basically, we score and tag at 5.0 - putting "[SPAM]" into the subject line, and leave the message in the Inbox. But for stuff over 8.0, we move it server-side to the Junk folder. This gives the users a lot of flexibility. If they don't trust our filter, then can look at the "maybe" spam messages in their Inbox and also look in the Junk folder. If they're not worried about false-positives in the 5.0-7.9 range, then they can setup a client side rule to simply move the messages from the Inbox to the Junk folder, or delete them.
We also have a server-side cron script that runs daily and removes any files in Junk that are older then 90 days.
participants (5)
-
aja-lists＠tni.org
-
Andrzej Adam Filip
-
Kirill Miazine
-
Stephan Bosch
-
Thomas Harold