[Dovecot] Using IMAP keywords instead of mailboxes for spam with dovecot-antispam
Hi,
Some email clients, like Evolution, use IMAP keywords to flag spam instead of moving mail to dedicated spam mailbox. This is very useful, as false positives can be easily "moved" to original mailbox simply by removing $Junk keyword.
I couldn't find a way how to integrate antispam software, like dspam, with dovecot, so that antispam automatically learns ham/spam.
I have found Johannes's dovecot-antispam plugin. Unfortunatelly, it doesn't have IMAP keywords support, autolearn is based on folders.
Is there any solution for this use case?
Thanks,
Jan
On 3/20/2008 7:23 AM, Jan Oravec wrote:
Some email clients, like Evolution, use IMAP keywords to flag spam instead of moving mail to dedicated spam mailbox. This is very useful, as false positives can be easily "moved" to original mailbox simply by removing $Junk keyword.
Thunderbird does this too...
I couldn't find a way how to integrate antispam software, like dspam, with dovecot, so that antispam automatically learns ham/spam.
I have found Johannes's dovecot-antispam plugin. Unfortunatelly, it doesn't have IMAP keywords support, autolearn is based on folders.
Is there any solution for this use case?
Support for this in the plug-in would be 'very cool'...
--
Best regards,
Charles
Hi Jan,
Some email clients, like Evolution, use IMAP keywords to flag spam instead of moving mail to dedicated spam mailbox. This is very useful, as false positives can be easily "moved" to original mailbox simply by removing $Junk keyword.
Interesting idea, but the keyword my evolution uses seems to be "Junk", not "$Junk".
I couldn't find a way how to integrate antispam software, like dspam, with dovecot, so that antispam automatically learns ham/spam.
I have found Johannes's dovecot-antispam plugin. Unfortunatelly, it doesn't have IMAP keywords support, autolearn is based on folders.
Is there any solution for this use case?
There isn't really, without some work. I have just committed this patch: http://git.sipsolutions.net/?p=dovecot-antispam.git;a=commitdiff;h=a5941271d... which will doesn't work yet but illustrates the concept (by debugging stuff to the syslog rather than calling the backend.) The new setting "antispam_spam_keywords" isn't documented yet but needs to be set for the keyword code to be invoked at all.
One problem with this is that evolution doesn't sync the keywords to the server right away, only when you change folders. Also, it's not entirely obvious what the code should do when the backend fails since there doesn't seem to be a way to roll back the keyword change easily. Any comments on the code would be appreciated, and you can probably easily make it invoke the backend with what's there to actually test it.
johannes
Hi Johannes,
Interesting idea, but the keyword my evolution uses seems to be "Junk", not "$Junk".
You are right, I didn't checked it and copied "$Junk" from http://wiki.dovecot.org/MailboxFormat/Maildir
There isn't really, without some work. I have just committed this patch: http://git.sipsolutions.net/?p=dovecot-antispam.git;a=commitdiff;h=a5941271d... which will doesn't work yet but illustrates the concept (by debugging stuff to the syslog rather than calling the backend.) The new setting "antispam_spam_keywords" isn't documented yet but needs to be set for the keyword code to be invoked at all.
Great!
I guess that in need_keyword_hook case save_init/save_finish hooks will have to check email for spams and keyword it. It is not clear how to mark mail as spam if both need_folder_hook and need_keyword_hook are true.
One problem with this is that evolution doesn't sync the keywords to the server right away, only when you change folders. Also, it's not entirely obvious what the code should do when the backend fails since there doesn't seem to be a way to roll back the keyword change easily. Any comments on the code would be appreciated, and you can probably easily make it invoke the backend with what's there to actually test it.
If I understand how it works with folders -- if message is moved from/to spam folder and spam learning fails, dovecot-antispam rolls back the IMAP transaction and dovecot reports error to client? (sorry if I am wrong, didn't read code nor IMAP specification, just assuming).
Then I think it would be correct to do exactly same with keywords and let client handle error. I am not sure if Evolution would handle that...
Jan
Hi,
I guess that in need_keyword_hook case save_init/save_finish hooks will have to check email for spams and keyword it. It is not clear how to mark mail as spam if both need_folder_hook and need_keyword_hook are true.
Well, the need_keyword_hook variable is just internal. The point is that save_{init,finish} are not invoked when just the keywords change, so I override the struct mail's update_keywords function.
See http://git.sipsolutions.net/?p=dovecot-antispam.git;a=blob;f=antispam-storag...
One problem with this is that evolution doesn't sync the keywords to the server right away, only when you change folders. Also, it's not entirely obvious what the code should do when the backend fails since there doesn't seem to be a way to roll back the keyword change easily. Any comments on the code would be appreciated, and you can probably easily make it invoke the backend with what's there to actually test it.
If I understand how it works with folders -- if message is moved from/to spam folder and spam learning fails, dovecot-antispam rolls back the IMAP transaction and dovecot reports error to client? (sorry if I am wrong, didn't read code nor IMAP specification, just assuming).
Yes.
Then I think it would be correct to do exactly same with keywords and let client handle error. I am not sure if Evolution would handle that...
The thing is that I don't know what context the update_keywords function runs in and whether it has a transaction at all, so I don't know how to roll back. I suppose it would be possible to grab the original keyword list before invoking super.update_keywords() and then resetting to that if the backend fails...
johannes
On Thu, 2008-03-20 at 15:16 +0100, Johannes Berg wrote:
The thing is that I don't know what context the update_keywords function runs in and whether it has a transaction at all, so I don't know how to roll back. I suppose it would be possible to grab the original keyword list before invoking super.update_keywords() and then resetting to that if the backend fails...
I haven't looked at your code, but struct mail always has a transaction (mail->transaction). You could fail the transaction by hooking into transaction_commit and calling rollback if your internal failure-flag was set.
But that sounds a bit extreme. Aren't you just adding/removing a single keyword? If so, you can use mail_update_keywords() to just add/remove that without reading/touching other keywords.
I haven't looked at your code, but struct mail always has a transaction (mail->transaction). You could fail the transaction by hooking into transaction_commit and calling rollback if your internal failure-flag was set.
But that sounds a bit extreme.
Indeed. But I suppose if I always have a transaction I'll already be hooked into it, I'll check that out.
Aren't you just adding/removing a single keyword? If so, you can use mail_update_keywords() to just add/remove that without reading/touching other keywords.
Evolution for example always replaces the complete keyword list, so that doesn't really work easily.
johannes
But that sounds a bit extreme. Aren't you just adding/removing a single keyword? If so, you can use mail_update_keywords() to just add/remove that without reading/touching other keywords.
Wouldn't it make a difference to mail client (silent fail instead of reported fail)?
I guess that in need_keyword_hook case save_init/save_finish hooks will have to check email for spams and keyword it. It is not clear how to mark mail as spam if both need_folder_hook and need_keyword_hook are true.
Well, the need_keyword_hook variable is just internal. The point is that save_{init,finish} are not invoked when just the keywords change, so I override the struct mail's update_keywords function.
See http://git.sipsolutions.net/?p=dovecot-antispam.git;a=blob;f=antispam-storag...
I thought about new mail case here, sorry for confusion.
Jan Oravec wrote:
Some email clients, like Evolution, use IMAP keywords to flag spam instead of moving mail to dedicated spam mailbox. This is very useful, as false positives can be easily "moved" to original mailbox simply by removing $Junk keyword.
I couldn't find a way how to integrate antispam software, like dspam, with dovecot, so that antispam automatically learns ham/spam.
I'm using a sieve script to add a "Junk" keyword based on spamassassin/amavis headers. Very easy and flexible. The antispam software only needs to add a special header to make it work.
participants (5)
-
Charles Marcus
-
Jan Oravec
-
Jef Driesen
-
Johannes Berg
-
Timo Sirainen