[Dovecot] Trouble implementing Antispam plug-in for Dovecot

[Ben Johnson]

On 9/20/2012 11:53 AM, Ben Johnson wrote:
> Thanks again, Tom. After you set me straight on the various versions of
> the plug-in, I was able to synthesize the relevant documentation and
> make it all work on Ubuntu 12.04 LTS with Dovecot 2.
> 
> The key was to understand the following:
> 
> 1.) The Ubuntu repositories use the Johannes versions of the plug-in
> (for both Dovecot 1 and 2).
> 
> Ubuntu 10.04 LTS (Dovecot 1)
> http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html
> 
> Ubuntu 12.04 LTS (Dovecot 2)
> http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html
> 
> 2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to
> the Eugene version, and that version's configuration directives differ
> from the Johannes version 2 directives. So, anyone attempting to follow
> the Wiki article on Ubuntu will need to adapt the directives accordingly
> (see documentation links above for equivalent directive names).
> 
> 3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki
> instructions lacks a shebang; until a proper shebang is added to the
> shell script, script execution will fail with "status 1". I added the
> standard shebang:
> 
> #!/bin/sh
> 
> Here are the configuration directives that work for me:
> 
> plugin {
> antispam_debug_target = syslog
> antispam_backend = pipe
> antispam_signature = X-DSPAM-Signature
> antispam_signature_missing = error
> antispam_spam_pattern_ignorecase = SPAM;JUNK
> antispam_pipe_tmpdir = /tmp
> antispam_pipe_program = /usr/bin/sa-learn-pipe.sh
> antispam_pipe_program_spam_arg = --spam
> antispam_pipe_program_notspam_arg = --ham
> }
> 
> My final goal is to get the version 1 plug-in working with Dovecot 1.2.9.
> 
> I'm using the following directives, but when I move mail from Inbox to
> Junk, for example, I see no evidence that antispam is taking action.
> 
> plugin {
> antispam_signature = X-DSPAM-Signature
> antispam_signature_missing = error
> antispam_spam_pattern_ignorecase = SPAM;JUNK
> antispam_mail_tmpdir = /tmp
> antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh
> antispam_mail_spam = --spam
> antispam_mail_notspam = --ham
> }
> 
> Dovecot writes nothing at all to syslog. No errors, no reference to
> antispam. And the message is moved successfully (no failure/error notice
> in IMAP client). The back-end script, "/usr/bin/sa-learn-pipe.sh",
> attempts to create a log file in /tmp, which works with Antispam 2. But
> no such luck here; nothing is ever written to /tmp.
> 
> Version 2 of the plug-in includes a verbose debugging mode, but it seems
> that no such feature is present in version 1.
> 
> I noticed also that version 1 seems not to require a back-end to be
> specified explicitly (as in version 2). Is the back-end implied via the
> configuration directives that are present?
> 
> Respectfully,
> 
> -Ben

Just a follow-up to report that I finally have the Antispam plug-in (the
Johannes version) -- with the mailtrain/pipe back-end -- working as
expected in both of the following environments:

- Ubuntu 10.04 with Dovecot 1.2.9
- Ubuntu 12.04 with Dovecot 2.0.19

In addition to the points that I mentioned in my previous note, a
crucial step in the debugging process was to modify the
"sa-learn-pipe.sh" script to produce more verbose output. Unlike
Antispam 2, Antispam 1 lacks a verbose debugging mode, so when the
plug-in does not work as expected, tracking-down the root cause can be
much more difficult.

Here's the shell script from http://wiki2.dovecot.org/Plugins/Antispam ,
with my modifications:

http://pastebin.com/igTXCYQF

After making these modifications to the shell script, I was able to
determine the cause of the failure in Dovecot 1.2.9. In fact, I was
incorrect earlier when I stated that version 2 was working; the same
problem applied there, too.

The "vmail" user, as whom the script is executed, lacks the permissions
required to change the Bayes token database (this is a "good thing").

An excerpt from the custom log file:

29722-start (--spam)
Calling (as user vmail) '/usr/bin/sa-learn --spam
/tmp/sendmail-msg-29722.txt'
bayes: expire_old_tokens: locker: safe_lock: cannot create tmp lockfile
/var/lib/amavis/.spamassassin/bayes.lock.example.com.29725 for
/var/lib/amavis/.spamassassin/bayes.lock: Permission denied
plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create
tmp lockfile /var/lib/amavis/.spamassassin/bayes.lock.example.com.29725
for /var/lib/amavis/.spamassassin/bayes.lock: Permission denied
ERROR: the Bayes learn function returned an error, please re-run with -D
for more information at /usr/bin/sa-learn line 493.
Learned tokens from 0 message(s) (1 message(s) examined)
29722-end

The only means by which I was able to work around this issue -- without
changing filesystem permissions, users, groups, or otherwise
compromising system security -- was to migrate from the standard
flat-file Bayes database format to MySQL. (The guide at
http://www.starbridge.org/spip/spip.php?article15 had everything I
needed, and it only took me a few minutes to implement.)

After making that change, everything began working as expected; here's a
sample custom log entry:

30565-start (--ham)
Calling (as user vmail) '/usr/bin/sa-learn --ham
/tmp/sendmail-msg-30565.txt'
Learned tokens from 1 message(s) (1 message(s) examined)
30565-end

In any event, this concludes the saga, and I must say, I'm thrilled to
have it all working.

Thanks again to everyone who contributed valuable input and pointed me
in the right direction.

And thanks to Johannes and Eugene (and everyone else who contributed to
Antispam) for solving a major hurdle in Bayes DB training.

-Ben