antispam plugin, pipe backend, how to make it work?

[Steffen Kaiser]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 13 Apr 2016, Johannes Rohr wrote:

> tweaked the wrapper script to make it work. It seems like the sa-learn call

See the page again.

> silently fails:
>
> "/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben
> root at vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log
> starting log
> 25200-start (--username=johannes at rohr.org --spam)
> libgcc_s.so.1 must be installed for pthread_cancel to work
> 25200-end
>
> Does the above error message have anything to do with it? Running the script 
> & sa-learn manually works fine. do virtual users change something how dovecot 
> calls programs, such as chrooting them?

Maybe, the script uses different search paths? Uncomment "# env" to 
verify.

>
> Cheers,
>
> Johannes
>
>
> Am 13.04.2016 um 09:45 schrieb Johannes Rohr:
>> Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:
>>> On 13-04-16 08:27, Johannes Rohr wrote:
>>>> Hi List,
>>>> 
>>>> does the antispam plugin any way of actually seeing how it is
>>>> calling the pipe program and how it is failing? I can't reproduce the
>>>> failure when I call sa-learn manually. Can someone tell me, with
>>>> which UID the call happens, is it the UID the current dovecot process
>>>> is running under (which would be vmail, as it is a virtual user
>>>> setup) Cheers,
>>>> 
>>> Hi Johannes,
>>> http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting
>>> This is exactly what the wrapper script is supposed to log for you, or
>>> which is trivially to add to its output. The wrapper script is not
>>> actually useful for production, but it is for debugging. That is why I
>>> asked for its logging, and any differences between manual run and a
>>> dovecot run.
>> 
>> When I used the wrapper script recommended at 
>> http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID and 
>> parameters (--username=some at na.me --ham). The wrapper script noted in the 
>> dovecot wiki seems broken. Note the commented out line:
>> 
>> #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt
>> 
>> It seems like the second half of the line "cat<&0 >> 
>> /tmp/sendmail-msg-$$.txt" was originally a separate line, catting the 
>> message, which has been passed by the plugin via STDOUT to a file. Without 
>> that command, the call of sa-learn in the next line has no file to read 
>> from:
>> 
>> /usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f 
>> /tmp/sendmail-msg-$$.txt &
>> 
>> So I moved the cat command to a separate uncommented line and to be sure I 
>> also commented out the
>> 
>> rm -f /tmp/sendmail-msg-$$.txt &
>> 
>> to verify that the file actually contains something, (why is there an "&" 
>> terminating the line, which would have the rm command executed in the 
>> background?)
>> 
>> But it seems the
>> 
>> /tmp/sendmail-msg-$$.txt &
>> 
>> was empty and the whole operation continued to fail. Could it be that the 
>> plugin fails to pass the actual message?
>> 
>> Cheers,
>> 
>> Johannes
>> 
>>> 
>>> Regards,
>>> Tom
>>> 
>>> 
>>>> Johannes
>>>> 
>>>> Am 12.04.2016 um 23:09 schrieb Johannes Rohr:
>>>>> Hi Tom,
>>>>> 
>>>>> Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:
>>>>>> On 12-04-16 20:14, Johannes Rohr wrote:
>>>>>>> Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise
>>>>>>> with the antispam plugin in version  2.0+20120225-2 and
>>>>>>> spamassassin at version 3.2.2
>>>>>>> 
>>>>>>> I have been trying and failed to get the pipe backend of the
>>>>>>> antispam plugin to work. Spamassin by itself works, a manual
>>>>>>> call of sa-learn works fine. Bayes data is stored in a mysql
>>>>>>> DB.
>>>>>>> 
>>>>>>> I have the following configuration in
>>>>>>> /etc/dovecot/conf.d/90-plugin.conf
>>>>>>> 
>>>>>>> plugin { #setting_name = value sieve=~/.dovecot.sieve
>>>>>>> sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam
>>>>>>> antispam_pipe_program_notspam_arg  = --ham
>>>>>>> antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
>>>>>>> antispam_pipe_program_args = --username=%u # % expansion done
>>>>>>> by dovecot
>>>>>> You need to specify the argument list as a list seperated by
>>>>>> semicolons, per example on the wiki:
>>>>>> 
>>>>>> antispam_pipe_program_args = --username;%u
>>>>> Not sure about that, because sa-learn expects the parameter
>>>>> --username=user at na.me, this is one parameter, not two, and it
>>>>> works for me locally, where I also set up dovecot to test. The
>>>>> main differences are that the versions I have here are newer, I'm
>>>>> running Debian sid, the server is on Ubuntu Precise, and second,
>>>>> the server is handling virtual users, locally I have only real Unix
>>>>> users.
>>>>> 
>>>>> Also, dovecot doesn't complain about configuration errors regard
>>>>> --username=%u. What other way would be there to get the parameter
>>>>> passed to sa-learn the way I want?
>>>>> 
>>>>> btw, I tried this, just to be sure, but the result was the same.
>>>>> 
>>>>>> The verbose debug should give you some logging to work with.
>>>>> Yes, and I pasted the output. I can of course repeat:
>>>>> 
>>>>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr
>>>>> 12 22:53:02 vm10 imap: antispam: 
>>>>> mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): 
>>>>> 0 Apr 12
>>>>> 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12
>>>>> 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash:
>>>>> 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0
>>>>> Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr
>>>>> 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12
>>>>> 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1,
>>>>> src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running
>>>>> mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10
>>>>> imap: antispam: running mailtrain backend program
>>>>> /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running
>>>>> mailtrain backend program parameter 1 --username=johannes at rohr.org
>>>>> Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend
>>>>> program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run
>>>>> program failed with exit code -1
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> Also, what does the wrapper script log when dovecot runs it?
>>>>> I have actually ditched  the wrapper script because it works
>>>>> without locally, I have configured the plugin to call sa-learn
>>>>> directly.
>>>>> 
>>>>>> How does that differ from a manual run?
>>>>> sa-learn works just fine with a manual run. It seems as if the
>>>>> plugin is passing the parameters wrongly, but the debugging output
>>>>> doesn't reveal the actual command, so it's just guesswork.
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Johannes
>>>>> 
>>>>>> 
>>>>>>> antispam_trash = trash;Trash;Deleted Items;Deleted Messages
>>>>>>> antispam_spam = SPAM;Junk antispam_backend = pipe
>>>>>>> antispam_verbose_debug = 1 antispam_debug_target = syslog
>>>>>>> antispam_pipe_tmpdir = /tmp }
>>>>>>> 
>>>>>>> I have tried the following wrapper script: #!/bin/bash # set
>>>>>>> -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >>
>>>>>>> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >>
>>>>>>> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D
>>>>>>> /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f
>>>>>>> /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0
>>>>>>> 
>>>>>>> It somehow isn't working. I can call the wrapper script
>>>>>>> manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh
>>>>>>> --username=some at mail.user --ham \ < someham
>>>>>>> 
>>>>>>> But it doesn't work from dovecot. It seems the script is
>>>>>>> called. but no new patterns show up in the bayes DB.
>>>>>>> Unfortunately, the plugin seems to be documented poorly. How
>>>>>>> does it call the programme it pipes to? Is it something like
>>>>>>> $programme --par1 --par2 < message ? Why can't / shouldn't I
>>>>>>> call sa-learn directly? All the instructions I found on the in
>>>>>>> internet contained a wrapper script, but none bothered to
>>>>>>> explain why you would actually need it.
>>>>>>> 
>>>>>>> I would be very grateful, if someone could enlighten me on
>>>>>>> that...
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> 
>>>>>>> Johannes
>

- -- 
Steffen Kaiser
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEVAwUBVw4R8Hz1H7kL/d9rAQKs7gf9F0QTDOugkaoIFCbzPlFohhhXGOKt6awv
AWVCqJdZgwmnYqH2WTn+xyueIG+HDZUrHLQB7CirjOIB2J8YWyprcsaBkcnZwxBW
kzklgjp9zK9ttHD7HHhkw7o4LwVjkOGtXJR49vz9bWL0P0N05dtdpI+HZuLS/Inx
qz+F7FLd7th2SFrFSRSKTlaVRyv/Ta6fh50tra+DQe5f4NblUwHe7dOUmXKYxs+U
Ievm4eKTxQA0GuQwyQSgKVqIKWZdAmA1bbmhbREK+7AEROCDt8XBYK1EJsRsG7LH
fGh6+kLi5QHihbumJtv14ba6P137UjnCwQ+6y8X38tDY50hOnVXASw==
=bmyk
-----END PGP SIGNATURE-----