[Dovecot] Reviewing end-user ham/spam submissions before feeding them to sa-learn via Dovecot Antispam plug-in

[Ben Johnson]

On 1/17/2013 4:31 AM, Steffen Kaiser wrote:
> On Wed, 16 Jan 2013, Ben Johnson wrote:
> 
>> Currently, I'm using the Dovecot Antispam plug-in with great
>> success. Everything works as expected.
> 
>> However, I would like to change the plug-in's behavior such that
>> it simply sends a copy of a message that is moved from Inbox ->
>> Junk (or Junk -> Inbox) to an administrator, instead of calling
>> sa-learn (I'm using SpamAssassin) automatically.
> 
>> Basically, I would like to be able to review messages that
>> end-users submit for training before they are actually fed to
>> sa-learn.
> 
> Hmm, if you use Maildir:
> 
> plugin { antispam_backend = spool2dir antispam_allow_append_to_spam
> = true [snip] antispam_spool2dir_spam = 
> /path/to/admin/Maildir/.TrainingReview.spam/new/%%020lu-%%05lu 
> antispam_spool2dir_notspam  = 
> /path/to/admin/Maildir/.TrainingReview.not_spam/new/%%020lu-%%05lu 
> }
> 
> any of your mail users need write permission those directories, the
> admin needs read permissions for the spooled files, you need some
> method to pass the reviewed messages to sa-learn.
> 
> IMHO, for 3rd step: I would either flag messages to be learned or
> move them into another folder. And a cron job feeds them to
> sa-learn, so they do not get into the message queue again and are
> probably re-filterred or modified before sa-learn picks them up.
> 
> If you do not have Maildir to drop the files to or get problems,
> because the messages appear in "new" before they are spooled on
> disk and hence they may get indexed wrongly, because their content
> changes (which is forbidden in IMAP), you could spool them into
> 
> antispam_spool2dir_spam = /tmp/spamspool/spam/%%020lu-%%05lu-%u 
> antispam_spool2dir_notspam  = /tmp/spamspool/ham/%%020lu-%%05lu-%u 
> -or- antispam_spool2dir_spam = /tmp/spamspool/%%020lu-%%05lu-%u-S 
> antispam_spool2dir_notspam  = /tmp/spamspool/%%020lu-%%05lu-%u-H
> 
> and have another cron job or inotify-wrapper move the messages to 
> the reviewer's mailbox. In this case, you can make use of the "%u" 
> component, which is expanded by the username, who spooled the
> message. So you could ignore messages of some users and/or ... .
> 
> Kind regards,
> 
> -- Steffen Kaiser
> 

Steffen, thank you very much for the thorough reply. I apologize for
taking so long to digest it and respond.

I am using the Maildir format indeed.

>> any of your mail users need write permission those directories, 
>> the admin needs read permissions for the spooled files,

By "mail users", do you mean, e.g., the "vmail" user account (I'm on
Debian/Ubuntu)? My understanding is that the "vmail" user account
handles all IMAP transactions; if this is true, then are you saying
that the only requisite to your suggestions is that the "vmail" user
has read/write access to the following two directories?

/path/to/admin/Maildir/.TrainingReview.spam/new/

and

/path/to/admin/Maildir/.TrainingReview.not_spam/new/

>> you need some method to pass the reviewed messages to sa-learn.

In the past, I have simply sorted the messages into "Ham" and "Spam"
sub-folders of the admin's training Inbox, and called sa-learn, with
the appropriate --ham/--spam switch on each, using a cron job. It
sounds as though this is what you are suggesting, and I can continue
with this approach.

I went ahead and tried to reconfigure Dovecot's Antispam plug-in to
use the spool2dir backend, but I'm receiving a less-than-helpful
message from the plug-in when I try to move a message from Inbox to
Junk or vice versa: "CANNOT: antispam plugin not configured".

Initially, I was using paths to the admin's mailbox, as demonstrated
in your initial response, but I simplified the spool location in order
to eliminate permission problems.

Please note that I am using Dovecot 1.2.9 in Ubuntu 10.04 LTS. By
extension, I am using the Antispam plug-in for Dovecot 1 (not 2), the
manpage for which is at
http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html
. So, the configuration option names and expected values differ
slightly from those in your example.

These are the only configuration directives that I am using:

-------------------------------------
plugin {
  # Unrelated plugin options here [...]

  antispam_spam_pattern_ignorecase = SPAM;JUNK
  # Is this next directive actually required for this approach?
  antispam_allow_append_to_spam = yes
  antispam_spool2dir_spam = /tmp/spamspool/%%020lu-%u-%%05lus
  antispam_spool2dir_notspam  = /tmp/spamspool/%%020lu-%u-%%05luh
}
-------------------------------------

The spool directory that is specified exists and, for testing
purposes, has 0777 permissions:

# ls -lah /tmp | grep "spamspool"
drwxrwxrwx  2 root     root     4.0K Jan 31 14:22 spamspool

Based on the above-cited manpage, these are be the only options that
are required (perhaps with the exception of
antispam_allow_append_to_spam).

What have I overlooked here?

Thanks again for all your help!

-Ben