antispam plugin, pipe backend, how to make it work?

Wed Apr 13 08:35:59 UTC 2016

tweaked the wrapper script to make it work. It seems like the sa-learn 
call silently fails:

"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben
root at vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log
starting log
25200-start (--username=johannes at rohr.org --spam)
libgcc_s.so.1 must be installed for pthread_cancel to work
25200-end

Does the above error message have anything to do with it? Running the 
script & sa-learn manually works fine. do virtual users change something 
how dovecot calls programs, such as chrooting them?

Cheers,

Johannes

Am 13.04.2016 um 09:45 schrieb Johannes Rohr:
> Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:
>> On 13-04-16 08:27, Johannes Rohr wrote:
>>> Hi List,
>>>
>>> does the antispam plugin any way of actually seeing how it is
>>> calling the pipe program and how it is failing? I can't reproduce the
>>> failure when I call sa-learn manually. Can someone tell me, with
>>> which UID the call happens, is it the UID the current dovecot process
>>> is running under (which would be vmail, as it is a virtual user
>>> setup) Cheers,
>>>
>> Hi Johannes,
>> http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting
>> This is exactly what the wrapper script is supposed to log for you, or
>> which is trivially to add to its output. The wrapper script is not
>> actually useful for production, but it is for debugging. That is why I
>> asked for its logging, and any differences between manual run and a
>> dovecot run.
>
> When I used the wrapper script recommended at 
> http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID 
> and parameters (--username=some at na.me --ham). The wrapper script noted 
> in the dovecot wiki seems broken. Note the commented out line:
>
> #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt
>
> It seems like the second half of the line "cat<&0 >> 
> /tmp/sendmail-msg-$$.txt" was originally a separate line, catting the 
> message, which has been passed by the plugin via STDOUT to a file. 
> Without that command, the call of sa-learn in the next line has no 
> file to read from:
>
> /usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f 
> /tmp/sendmail-msg-$$.txt &
>
> So I moved the cat command to a separate uncommented line and to be 
> sure I also commented out the
>
> rm -f /tmp/sendmail-msg-$$.txt &
>
> to verify that the file actually contains something, (why is there an 
> "&" terminating the line, which would have the rm command executed in 
> the background?)
>
> But it seems the
>
> /tmp/sendmail-msg-$$.txt &
>
> was empty and the whole operation continued to fail. Could it be that 
> the plugin fails to pass the actual message?
>
> Cheers,
>
> Johannes
>
>>
>> Regards,
>> Tom
>>
>>
>>> Johannes
>>>
>>> Am 12.04.2016 um 23:09 schrieb Johannes Rohr:
>>>> Hi Tom,
>>>>
>>>> Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:
>>>>> On 12-04-16 20:14, Johannes Rohr wrote:
>>>>>> Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise
>>>>>> with the antispam plugin in version  2.0+20120225-2 and
>>>>>> spamassassin at version 3.2.2
>>>>>>
>>>>>> I have been trying and failed to get the pipe backend of the
>>>>>> antispam plugin to work. Spamassin by itself works, a manual
>>>>>> call of sa-learn works fine. Bayes data is stored in a mysql
>>>>>> DB.
>>>>>>
>>>>>> I have the following configuration in
>>>>>> /etc/dovecot/conf.d/90-plugin.conf
>>>>>>
>>>>>> plugin { #setting_name = value sieve=~/.dovecot.sieve
>>>>>> sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam
>>>>>> antispam_pipe_program_notspam_arg  = --ham
>>>>>> antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
>>>>>> antispam_pipe_program_args = --username=%u # % expansion done
>>>>>> by dovecot
>>>>> You need to specify the argument list as a list seperated by
>>>>> semicolons, per example on the wiki:
>>>>>
>>>>> antispam_pipe_program_args = --username;%u
>>>> Not sure about that, because sa-learn expects the parameter
>>>> --username=user at na.me, this is one parameter, not two, and it
>>>> works for me locally, where I also set up dovecot to test. The
>>>> main differences are that the versions I have here are newer, I'm
>>>> running Debian sid, the server is on Ubuntu Precise, and second,
>>>> the server is handling virtual users, locally I have only real Unix
>>>> users.
>>>>
>>>> Also, dovecot doesn't complain about configuration errors regard
>>>> --username=%u. What other way would be there to get the parameter
>>>> passed to sa-learn the way I want?
>>>>
>>>> btw, I tried this, just to be sure, but the result was the same.
>>>>
>>>>> The verbose debug should give you some logging to work with.
>>>> Yes, and I pasted the output. I can of course repeat:
>>>>
>>>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr
>>>> 12 22:53:02 vm10 imap: antispam: 
>>>> mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): 
>>>> 0 Apr 12
>>>> 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12
>>>> 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash:
>>>> 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0
>>>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr
>>>> 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12
>>>> 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1,
>>>> src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running
>>>> mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10
>>>> imap: antispam: running mailtrain backend program
>>>> /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running
>>>> mailtrain backend program parameter 1 --username=johannes at rohr.org
>>>> Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend
>>>> program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run
>>>> program failed with exit code -1
>>>>
>>>>
>>>>
>>>>
>>>>> Also, what does the wrapper script log when dovecot runs it?
>>>> I have actually ditched  the wrapper script because it works
>>>> without locally, I have configured the plugin to call sa-learn
>>>> directly.
>>>>
>>>>> How does that differ from a manual run?
>>>> sa-learn works just fine with a manual run. It seems as if the
>>>> plugin is passing the parameters wrongly, but the debugging output
>>>> doesn't reveal the actual command, so it's just guesswork.
>>>>
>>>> Cheers,
>>>>
>>>> Johannes
>>>>
>>>>>
>>>>>> antispam_trash = trash;Trash;Deleted Items;Deleted Messages
>>>>>> antispam_spam = SPAM;Junk antispam_backend = pipe
>>>>>> antispam_verbose_debug = 1 antispam_debug_target = syslog
>>>>>> antispam_pipe_tmpdir = /tmp }
>>>>>>
>>>>>> I have tried the following wrapper script: #!/bin/bash # set
>>>>>> -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >>
>>>>>> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >>
>>>>>> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D
>>>>>> /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f
>>>>>> /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0
>>>>>>
>>>>>> It somehow isn't working. I can call the wrapper script
>>>>>> manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh
>>>>>> --username=some at mail.user --ham \ < someham
>>>>>>
>>>>>> But it doesn't work from dovecot. It seems the script is
>>>>>> called. but no new patterns show up in the bayes DB.
>>>>>> Unfortunately, the plugin seems to be documented poorly. How
>>>>>> does it call the programme it pipes to? Is it something like
>>>>>> $programme --par1 --par2 < message ? Why can't / shouldn't I
>>>>>> call sa-learn directly? All the instructions I found on the in
>>>>>> internet contained a wrapper script, but none bothered to
>>>>>> explain why you would actually need it.
>>>>>>
>>>>>> I would be very grateful, if someone could enlighten me on
>>>>>> that...
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Johannes