[Dovecot] Trouble implementing Antispam plug-in for Dovecot

[Ben Johnson]

On 9/19/2012 7:17 PM, Ben Johnson wrote:
> 
> 
> On 9/19/2012 6:19 PM, Tom Hendrikx wrote:
>> On 19/09/12 23:41, Ben Johnson wrote:
>>>
>>>
>>> On 9/11/2012 10:40 AM, Ben Johnson wrote:
>>>> You nailed it, Ben. Thanks!
>>>>
>>>> I downloaded/installed the dovecot-dev package from the same maintainer
>>>> and was able to build Antispam without issue.
>>>>
>>>> The hang-up now seems to be with my configuration.
>>>>
>>>> I'm seeing the following error in my email client whenever I attempt to
>>>> move an email message from, say, Inbox to Junk folder:
>>>>
>>>> [SERVERBUG] BUG Unknown internal error.
>>>>
>>>> Nothing at all appears in syslog when this happens.
>>>>
>>>> I followed the instructions at http://wiki2.dovecot.org/Plugins/Antispam
>>>> , to the letter, I believe.
>>>>
>>>> The relevant portions of my configuration are as follows:
>>>>
>>>> ---------------------
>>>> protocol imap {
>>>>   mail_plugins = quota imap_quota antispam
>>>> }
>>>>
>>>> plugin {
>>>>   quota = dict:user::file:/var/vmail/%d/%n/.quotausage
>>>>   sieve = /var/vmail/%d/%n/.sieve
>>>>
>>>>   antispam_backend = MAILTRAIN
>>>>   antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh
>>>>   antispam_mail_spam = --spam
>>>>   antispam_mail_notspam = --ham
>>>>   antispam_spam_pattern_ignorecase = spam;junk
>>>> }
>>>> ---------------------
>>>>
>>>> The permissions and execute bit are set in accordance with other
>>>> executables in the same directory:
>>>>
>>>> # ls -lah /usr/bin/ | grep -i "sa-learn-pipe.sh"
>>>> -rwxr-xr-x  1 root root     297 Sep  5 11:17 sa-learn-pipe.sh
>>>>
>>>> I would provide more log content, but the debug options seem to be gone
>>>> in the forked, >= 2.0 version of this plug-in.
>>>>
>>>> Any ideas are very much appreciated.
>>>>
>>>> Thanks,
>>>>
>>>> -Ben
>>>
>>> Given that nobody shared any insight here, I decided to stand-up a
>>> completely new VM with Ubuntu 12.04 LTS and see if the same error occurs.
>>>
>>> While the error is different this time around, I am still unable to get
>>> any variant of the dovecot-antispam plug-in working.
>>>
>>> When I attempt to use the current packages from Ubuntu 12.04.1 LTS, and
>>> I use the above configuration, my syslog fills-up with the following:
>>>
>>> dovecot: imap-login: Login: user=<user at example.com>, method=PLAIN,
>>> rip=10.0.1.57, lip=10.0.1.8, mpid=2879, TLS
>>> dovecot: master: Error: service(imap): child 2879 returned error 3
>>>
>>> Surprisingly, I can find only two other mentions of "error 3" anywhere
>>> on the Internet. Those mentions were from another user to this mailing list:
>>>
>>> http://www.dovecot.org/list/dovecot/2012-August/067787.html
>>> http://www.dovecot.org/list/dovecot/2012-August/067814.html
>>>
>>> The lack of "error 3" ubiquity leads me to believe that I must be doing
>>> something stupid.
>>>
>>> There's no question that this behavior is related to the
>>> antispam-plugin, because when I commented-out the relevant lines in
>>> Dovecot's configuration file, the error does not occur.
>>>
>>> I have followed the instructions at
>>> http://wiki2.dovecot.org/Plugins/Antispam to the letter, and this is the
>>> result.
>>>
>>> The only instance in which I diverged from the Wiki article is that I
>>> used the Ubuntu-provided dovecot-antispam package (instead of compiling
>>> the plug-in from source).
>>>
>>> I am trying to use the "mailtrain" back-end, per the above-cited Wiki
>>> article, to call SpamAssassin's "sa-learn" utility for real-time learning.
>>>
>>> I am not using the SpamAssassin daemon (spamd).
>>>
>>> I'm about to throw in the towel on this plug-in, so any help would be
>>> greatly appreciated.
>>>
>>> Thank you,
>>>
>>> -Ben
>>
>> There are 2 versions of the dovecot-antispam package. The original
>> version was created by Johannes Berg, then died almost a silent death,
>> and was forked by Eugene Paskevitch. Later Johannes revived his original
>> version somewhat.
>>
>> The ubuntu package is based on the 'revived Johannes fork' [1], while
>> the documentation on the dovecot wiki describes the 'Eugene fork'. [2]
>>
>> [1] https://launchpad.net/ubuntu/precise/+source/dovecot-antispam/+copyright
>> [2] http://wiki2.dovecot.org/Plugins/Antispam
>>
>> Thus you'll need to choose which plugin you want, then follow the
>> complete procedure for that plugin only. They both can do what you want,
>> but building and configuration are slightly different.
>>
>> It would be nice if Johannes and Eugene would either agree on a single
>> version of the plugin, or rename one of the forks so they don't get
>> mixed up that easily. I'm not sure if that has been discussed (on- or
>> off-list) in the past.
>>
>> FWIW: I used the 'original Johannes fork' with dovecot 1.x, and the
>> 'Eugene fork' with 2.0.x, both with delight. Didn't get myself to
>> upgrade to 2.1 yet, so no experience there.
>>
>> --
>> Tom
>>
> 
> Thank you for your timely response, Tom. Much appreciated.
> 
> Your remarks clear-up a lot of the confusion for me. I didn't realize
> that the "revived Johannes fork" made a comeback for Dovecot 2. Nor did
> I realize that this is the version that is packaged with Ubuntu 12.04.
> 
> Given that I'm more-or-less forced to work with Ubuntu LTS, I would
> prefer to use the OS's package management system wherever possible. I
> suppose, by extension, that I would prefer to use the "revived Johannes
> fork".
> 
> I've tried the Johannes versions (for both Dovecot 1 and Dovecot 2), but
> am unable to get either to work the way I want (mailtrain back-end with
> real-time calls to sa-learn, as described in the Dovecot 2/Eugene Wiki
> article). Shouldn't the basic tenets of each version of the plug-in apply?
> 
> Where should I start, in the way of seeking support? Basically, I intend
> to use the Johannes version, but I want to configure it in the manner
> described in the Eugene version Wiki article.
> 
> I've already described (rather exhaustively) what happens when I attempt
> to use the Johannes version and adapt the Eugene version instructions.
> 
> I'm curious to know what method you've used over the years with such
> glowing success. Which back-end did you use with Dovecot 1/Johannes? How
> about with Dovecot 2/Eugene?
> 
> Thanks again,
> 
> -Ben
> 

Thanks again, Tom. After you set me straight on the various versions of
the plug-in, I was able to synthesize the relevant documentation and
make it all work on Ubuntu 12.04 LTS with Dovecot 2.

The key was to understand the following:

1.) The Ubuntu repositories use the Johannes versions of the plug-in
(for both Dovecot 1 and 2).

Ubuntu 10.04 LTS (Dovecot 1)
http://manpages.ubuntu.com/manpages/lucid/man7/dovecot-antispam.7.html

Ubuntu 12.04 LTS (Dovecot 2)
http://manpages.ubuntu.com/manpages/precise/man7/dovecot-antispam.7.html

2.) The Dovecot 2 Wiki instructions for the Antispam plug-in refer to
the Eugene version, and that version's configuration directives differ
from the Johannes version 2 directives. So, anyone attempting to follow
the Wiki article on Ubuntu will need to adapt the directives accordingly
(see documentation links above for equivalent directive names).

3.) The sample "/usr/bin/sa-learn-pipe.sh" script in the Dovecot 2 Wiki
instructions lacks a shebang; until a proper shebang is added to the
shell script, script execution will fail with "status 1". I added the
standard shebang:

#!/bin/sh

Here are the configuration directives that work for me:

plugin {
antispam_debug_target = syslog
antispam_backend = pipe
antispam_signature = X-DSPAM-Signature
antispam_signature_missing = error
antispam_spam_pattern_ignorecase = SPAM;JUNK
antispam_pipe_tmpdir = /tmp
antispam_pipe_program = /usr/bin/sa-learn-pipe.sh
antispam_pipe_program_spam_arg = --spam
antispam_pipe_program_notspam_arg = --ham
}

My final goal is to get the version 1 plug-in working with Dovecot 1.2.9.

I'm using the following directives, but when I move mail from Inbox to
Junk, for example, I see no evidence that antispam is taking action.

plugin {
antispam_signature = X-DSPAM-Signature
antispam_signature_missing = error
antispam_spam_pattern_ignorecase = SPAM;JUNK
antispam_mail_tmpdir = /tmp
antispam_mail_sendmail = /usr/bin/sa-learn-pipe.sh
antispam_mail_spam = --spam
antispam_mail_notspam = --ham
}

Dovecot writes nothing at all to syslog. No errors, no reference to
antispam. And the message is moved successfully (no failure/error notice
in IMAP client). The back-end script, "/usr/bin/sa-learn-pipe.sh",
attempts to create a log file in /tmp, which works with Antispam 2. But
no such luck here; nothing is ever written to /tmp.

Version 2 of the plug-in includes a verbose debugging mode, but it seems
that no such feature is present in version 1.

I noticed also that version 1 seems not to require a back-end to be
specified explicitly (as in version 2). Is the back-end implied via the
configuration directives that are present?

Respectfully,

-Ben