antispam plugin, pipe backend, how to make it work?

[Johannes Rohr]

Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:
> On 13-04-16 08:27, Johannes Rohr wrote:
>> Hi List,
>>
>> does the antispam plugin any way of actually seeing how it is
>> calling the pipe program and how it is failing? I can't reproduce the
>> failure when I call sa-learn manually. Can someone tell me, with
>> which UID the call happens, is it the UID the current dovecot process
>> is running under (which would be vmail, as it is a virtual user
>> setup) Cheers,
>>
> Hi Johannes,
> http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting
> This is exactly what the wrapper script is supposed to log for you, or
> which is trivially to add to its output. The wrapper script is not
> actually useful for production, but it is for debugging. That is why I
> asked for its logging, and any differences between manual run and a
> dovecot run.

When I used the wrapper script recommended at 
http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID 
and parameters (--username=some at na.me --ham). The wrapper script noted 
in the dovecot wiki seems broken. Note the commented out line:

#echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt

It seems like the second half of the line "cat<&0 >> 
/tmp/sendmail-msg-$$.txt" was originally a separate line, catting the 
message, which has been passed by the plugin via STDOUT to a file. 
Without that command, the call of sa-learn in the next line has no file 
to read from:

/usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f 
/tmp/sendmail-msg-$$.txt &

So I moved the cat command to a separate uncommented line and to be sure 
I also commented out the

rm -f /tmp/sendmail-msg-$$.txt &

to verify that the file actually contains something, (why is there an 
"&" terminating the line, which would have the rm command executed in 
the background?)

But it seems the

/tmp/sendmail-msg-$$.txt &

was empty and the whole operation continued to fail. Could it be that 
the plugin fails to pass the actual message?

Cheers,

Johannes

>
> Regards,
> Tom
>
>
>> Johannes
>>
>> Am 12.04.2016 um 23:09 schrieb Johannes Rohr:
>>> Hi Tom,
>>>
>>> Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:
>>>> On 12-04-16 20:14, Johannes Rohr wrote:
>>>>> Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise
>>>>> with the antispam plugin in version  2.0+20120225-2 and
>>>>> spamassassin at version 3.2.2
>>>>>
>>>>> I have been trying and failed to get the pipe backend of the
>>>>> antispam plugin to work. Spamassin by itself works, a manual
>>>>> call of sa-learn works fine. Bayes data is stored in a mysql
>>>>> DB.
>>>>>
>>>>> I have the following configuration in
>>>>> /etc/dovecot/conf.d/90-plugin.conf
>>>>>
>>>>> plugin { #setting_name = value sieve=~/.dovecot.sieve
>>>>> sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam
>>>>> antispam_pipe_program_notspam_arg  = --ham
>>>>> antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
>>>>> antispam_pipe_program_args = --username=%u # % expansion done
>>>>> by dovecot
>>>> You need to specify the argument list as a list seperated by
>>>> semicolons, per example on the wiki:
>>>>
>>>> antispam_pipe_program_args = --username;%u
>>> Not sure about that, because sa-learn expects the parameter
>>> --username=user at na.me, this is one parameter, not two, and it
>>> works for me locally, where I also set up dovecot to test. The
>>> main differences are that the versions I have here are newer, I'm
>>> running Debian sid, the server is on Ubuntu Precise, and second,
>>> the server is handling virtual users, locally I have only real Unix
>>> users.
>>>
>>> Also, dovecot doesn't complain about configuration errors regard
>>> --username=%u. What other way would be there to get the parameter
>>> passed to sa-learn the way I want?
>>>
>>> btw, I tried this, just to be sure, but the result was the same.
>>>
>>>> The verbose debug should give you some logging to work with.
>>> Yes, and I pasted the output. I can of course repeat:
>>>
>>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr
>>> 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): 0 Apr 12
>>> 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12
>>> 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash:
>>> 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0
>>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr
>>> 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12
>>> 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1,
>>> src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running
>>> mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10
>>> imap: antispam: running mailtrain backend program
>>> /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running
>>> mailtrain backend program parameter 1 --username=johannes at rohr.org
>>> Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend
>>> program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run
>>> program failed with exit code -1
>>>
>>>
>>>
>>>
>>>> Also, what does the wrapper script log when dovecot runs it?
>>> I have actually ditched  the wrapper script because it works
>>> without locally, I have configured the plugin to call sa-learn
>>> directly.
>>>
>>>> How does that differ from a manual run?
>>> sa-learn works just fine with a manual run. It seems as if the
>>> plugin is passing the parameters wrongly, but the debugging output
>>> doesn't reveal the actual command, so it's just guesswork.
>>>
>>> Cheers,
>>>
>>> Johannes
>>>
>>>>
>>>>> antispam_trash = trash;Trash;Deleted Items;Deleted Messages
>>>>> antispam_spam = SPAM;Junk antispam_backend = pipe
>>>>> antispam_verbose_debug = 1 antispam_debug_target = syslog
>>>>> antispam_pipe_tmpdir = /tmp }
>>>>>
>>>>> I have tried the following wrapper script: #!/bin/bash # set
>>>>> -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >>
>>>>> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >>
>>>>> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D
>>>>> /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f
>>>>> /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0
>>>>>
>>>>> It somehow isn't working. I can call the wrapper script
>>>>> manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh
>>>>> --username=some at mail.user --ham \ < someham
>>>>>
>>>>> But it doesn't work from dovecot. It seems the script is
>>>>> called. but no new patterns show up in the bayes DB.
>>>>> Unfortunately, the plugin seems to be documented poorly. How
>>>>> does it call the programme it pipes to? Is it something like
>>>>> $programme --par1 --par2 < message ? Why can't / shouldn't I
>>>>> call sa-learn directly? All the instructions I found on the in
>>>>> internet contained a wrapper script, but none bothered to
>>>>> explain why you would actually need it.
>>>>>
>>>>> I would be very grateful, if someone could enlighten me on
>>>>> that...
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Johannes