Re: [Dovecot] Dovecot-antispam does not work
Hi Marcin and everyone,
On Tue, Feb 3, 2009 at 7:08 PM, Marcin Rzepecki m.rzepecki@iem.pw.edu.plwrote:
Hi,
first of all, thanks for the answer. "(Un)fortunately", I don't have these error messages. For information, I am using dovecot 1.1.7r1, compiled from the Gentoo package repository and
only enabled debug I set was "mail_debug = yes". The file where I see all debug messages is /var/log/messages, which is
one where all messages from dovecot and postfix are stored under Gentoo apparently.
I am recompiling the package with "debug" to see if I see any more debug messages. I will make more tests and try to understand this all better, because currently it is pretty messy in my head as well about how all
Tue, Feb 03, 2009 at 06:14:08PM +0100, Jehan Pag?s wrote: the the this
work together (postfix, dovecot, dspam, antispam plugin, etc.). I will keep you up to date, but if ever you have any idea, don't hesitate...
Hi,
please, don't forget to CC: your replies to dovecot@dovecot.org, maybe solution would be helpfull for someone else :)
yes sorry for this. I though I was replying to the whole ml. Apparently the ml is not very well configured because a reply does not reply to the whole ml, but to the single sender. I will take care now.
Errors which I wrote are sent to IMAP client, so are you getting getting eg. some pop-up window in your client with error message while trying to move message?
I was using the RoundCube web client on my server and had no such message. It was just telling me that it could not move the message. I configured a Thunderbird then and it displayed indeed this message 'Failed to call dspam'. So I made your trick, I compiled myself the dovecot-antispam library and now I can move messages... But are you sure that it is the right method and that the dspam training is indeed called? Because if I read well this piece of code, it looks like it was checking if the dspam training was successful. We commented out this check, so now maybe it fails and we copy the messages anyway to Junk/ even though no training really occurred. And when I check the logs or the dspam_stats, I don't see any change, even though I guess this retraining should be seen as a change in the stats, no?
Antispam plugin is easy to understand: when you configure antispam_spam = SPAM it will set SPAM folder as his special folder to observe. When You get e-mail that was classified incorrect by DSPAM, you can move it to SPAM folder. Antispam plugin will trigger dspam command to mark this message as retrained-spam. You can do this of course in reverse order - moving message _from_ SPAM folder to another will run dspam and mark message as innocent. Message you are moving must have X-DSPAM-Signature header (you can check in msg source to see if dspam adds it correctly)
I have the header on every emails. By the way, when one has the item "X-DSPAM-Result: Innocent" in the header, it means the email is not a spam, does'nt it (right now I am focusing on having this plugin working for training purpose, but currently not much is working perfectly as expected in my email spam system)?
You can provide some info from your log files, i'll try to help. Debug flag for plugin should effect with some info in debug-logs: Dec 15 23:54:18 freebsd imap: antispam: plugin initialising (1.1-notgit) Dec 15 23:54:18 freebsd imap: antispam: no trash folders Dec 15 23:54:18 freebsd imap: antispam: "SPAM" is spam folder Dec 15 23:54:18 freebsd imap: antispam: no unsure folders Dec 15 23:54:18 freebsd imap: antispam: dspam binary set to /usr/local/bin/dspam Dec 15 23:54:18 freebsd imap: antispam: dspam extra arg --user=dump-capability Dec 15 23:54:18 freebsd imap: antispam: signature header line is "X-DSPAM-Signature" Dec 15 23:54:18 freebsd imap: antispam: will silently move mails with missing signature [...]
Me I have: Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): antispam: plugin initialising (unknown) Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): antispam: "trash" is trash folder Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): antispam: "Trash" is trash folder Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): antispam: "Deleted Items" is trash folder Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): antispam: "SPAM" is spam folder Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): antispam: "Junk" is spam folder Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): antispam: no unsure folders Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): antispam: dspam binary set to /usr/bin/dspam Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): Growing pool 'antispam-pool' with: 2048
Cheers,
-- Marcin Rzepecki m.rzepecki(at)iem.pw.edu.pl
Thanks.
Jehan
Jehan wrote:
... Apparently the ml is not very well configured because a reply does not reply to the whole ml, but to the single sender. I will take care now.
I bet there are a *lot* of people (me included) who think the ml is very well configured precisely because it does not set Reply-to: the list.
H
On 2/3/2009, Harlan Stenn (Harlan.Stenn@pfcs.com) wrote:
I bet there are a *lot* of people (me included) who think the ml is very well configured precisely because it does not set Reply-to: the list.
Please lets not go there (I'm on the other side, since most people don't use mutt (a mail client with a working 'Reply to list' function)...
--
Best regards,
Charles
On Tue, 2009-02-03 at 16:59 -0500, Charles Marcus wrote:
On 2/3/2009, Harlan Stenn (Harlan.Stenn@pfcs.com) wrote:
I bet there are a *lot* of people (me included) who think the ml is very well configured precisely because it does not set Reply-to: the list.
Please lets not go there (I'm on the other side, since most people don't use mutt (a mail client with a working 'Reply to list' function)...
..And my opinion is that "reply to all" is the correct behavior most of the time. I just hate it when I'm sending questions to other lists which I don't actively follow and people reply to me a few days later but don't Cc: me. I can easily lose the reply.
On 2/3/2009, Timo Sirainen (tss@iki.fi) wrote:
..And my opinion is that "reply to all" is the correct behavior most of the time. I just hate it when I'm sending questions to other lists which I don't actively follow and people reply to me a few days later but don't Cc: me. I can easily lose the reply.
I know... this is why I said 'lets don't go there'... because it is a personal preference thing.
I have been asked many times to STOP CC'ing someone who also is on the list, because they then get two copies, so your preferred method would not make some others happy...
There is absolutely no way to please everyone on this issue (that I can see), unless there was some way for a user to configure the 'Reply To' at the list level the way they wanted it - ie, three choices:
Reply to List
Reply to original sender
Reply to List, CC original sender...
At least there is a way to choose 1 or 2 for the dovecot list...
Tue, Feb 03, 2009 at 10:36:12PM +0100, Jehan Pag?s wrote:
and that the dspam training is indeed called? Because if I read well this piece of code, it looks like it was checking if the dspam training was successful. We commented out this check, so now maybe it fails and we copy the messages anyway to Junk/ even though no training really occurred. And when I check the logs or the dspam_stats, I don't see any change, even though I guess this retraining should be seen as a change in the stats, no?
Yes, it checks if dspam was called correctly, but it didn't worked with dovecot >=1.1 I've comment out checking call_dspam exit status, but:
call_dspam(item->sig, item->wanted); item = item->next;
it calls dspam anyway and it works for me fine. I've patched dspam a little bit to make it log better, so when I move message to SPAM folder I'm getting:
dspam_stats mr: mr TP: 0 TN: 340 FP: 21 FN: 33 SC: 0 NC: 4
dspam.log: Feb 3 21:20:31 freebsd dspam[549]: Message from <None Specified> marked as SPAM (signature=4988a7065441870592463, user=mr, status=Retrained, time=0.016614s) and so on.
dspam_stats mr: mr TP: 0 TN: 339 FP: 21 FN: 34 SC: 0 NC: 4 So, you should see changes.
I have the header on every emails. By the way, when one has the item "X-DSPAM-Result: Innocent" in the header, it means the email is not a spam, does'nt it (right now I am focusing on having this plugin working for training purpose, but currently not much is working perfectly as expected in my email spam system)?
Yes. It was classified as not spam. You can get X-DSPAM-Result: Whitelisted, when dspam auto-whitelists some sender
Me I have: Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): antispam: plugin initialising (unknown) [...] Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan@zemarmot.net): antispam: dspam binary set to /usr/bin/dspam
Seems to be well configured. Check if messages are going to correct dspam user. You are using jehan@zemarmot.net as imap user, so dspam user have to be exactly the same. (i'm using hash driver, so all is in $DSPAM_HOME/data directory).
-- Marcin Rzepecki m.rzepecki(at)iem.pw.edu.pl
Hi,
On Wed, Feb 4, 2009 at 12:45 AM, Marcin Rzepecki m.rzepecki@iem.pw.edu.plwrote:
Yes, it checks if dspam was called correctly, but it didn't worked with dovecot >=1.1
I've comment out checking call_dspam exit status, but:
call_dspam(item->sig, item->wanted); item = item->next;
it calls dspam anyway and it works for me fine. I've patched dspam a little bit to make it log better, so when I move message to SPAM folder I'm getting:
dspam_stats mr: mr TP: 0 TN: 340 FP: 21 FN: 33 SC: 0 NC: 4
dspam.log: Feb 3 21:20:31 freebsd dspam[549]: Message from <None Specified> marked as SPAM (signature=4988a7065441870592463, user=mr, status=Retrained, time=0.016614s) and so on.
dspam_stats mr: mr TP: 0 TN: 339 FP: 21 FN: 34 SC: 0 NC: 4 So, you should see changes.
I saw this piece of code and did the same as you in my change of course. Unfortunately, it didn't work (no logged "marked as SPAM", nor change in my stats. I have dovecot 1.1.7-r1 though, so it is probably the reason.
Me I have: Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan at mydomain.tld): antispam: plugin initialising (unknown) [...] Feb 3 22:10:39 ks34186 dovecot: IMAP(jehan at mydomain.tld): antispam: dspam binary set to /usr/bin/dspam
Seems to be well configured. Check if messages are going to correct dspam user. You are using jehan at mydomain.tld as imap user, so dspam user have to be exactly the same. (i'm using hash driver, so all is in $DSPAM_HOME/data directory).
Ok I will have a look at this dspam user stuffs. I may have misconfigured some parts... (I forgot to change my real email in my previous message but it looks that the ml's web archive changes it to some non "easily" readable by spam bots...).
Thanks.
Jehan
Wed, Feb 04, 2009 at 12:12:44PM +0100, Jehan Pag?s wrote:
I saw this piece of code and did the same as you in my change of course. Unfortunately, it didn't work (no logged "marked as SPAM", nor change in my stats. I have dovecot 1.1.7-r1 though, so it is probably the reason.
Logging '...markes as SPAM..' lines to syslog is my idea, I can send some dspam patches, to make it work. It's usefull for me. Are you using mbox or maildir? If you are using mbox, try to upgrade to Dovecot >=1.1.8 - it contains several mbox bug fixes (e.g 'Next message unexpectedly lost') which are needed to make antispam plugin work.
-- Marcin Rzepecki m.rzepecki(at)iem.pw.edu.pl
participants (5)
-
Charles Marcus
-
Harlan Stenn
-
Jehan Pagès
-
Marcin Rzepecki
-
Timo Sirainen