antispam plugin, pipe backend, how to make it work?
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with the antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp }
I have tried the following wrapper script: #!/bin/bash
set -x
LOGFILE=/tmp/sa-learn-pipe.log
echo "$$-start ($*)" >> $LOGFILE
#echo $* > /tmp/sendmail-parms.txt
cat<&0 >> /tmp/sendmail-msg-$$.txt
/usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 &&
rm -f /tmp/sendmail-msg-$$.txt
echo "$$-end" >> $LOGFILE
exit 0
It somehow isn't working. I can call the wrapper script manually e.g.
sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user
--ham
< someham
But it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
Cheers,
Johannes
Hi,
here (Centos 7) its already integrated...
Just add something like this:
plugin {
antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_sendmail_args = -u amavis antispam_mail_spam = --spam antispam_mail_notspam = --ham }
to dovecot.conf
Cheers
Jürgen
Am 12.04.2016 um 20:14 schrieb Johannes Rohr:
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise withthe antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp }
I have tried the following wrapper script: #!/bin/bash
set -x
LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 &&
rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0It somehow isn't working. I can call the wrapper script manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user --ham
< somehamBut it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
Cheers,
Johannes
dovecot 2.2.10
Am 12.04.2016 um 20:18 schrieb J. Echter:
Hi,
here (Centos 7) its already integrated...
Just add something like this:
plugin {
antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_sendmail_args = -u amavis antispam_mail_spam = --spam antispam_mail_notspam = --ham }
to dovecot.conf
Cheers
Jürgen
Am 12.04.2016 um 20:14 schrieb Johannes Rohr:
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise withthe antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp }
I have tried the following wrapper script: #!/bin/bash
set -x
LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 &&
rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0It somehow isn't working. I can call the wrapper script manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user --ham
< somehamBut it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
Cheers,
Johannes
Hi Jürgen,
what do you mean by "It's already integrated"? Of course I added the configuration, see my initial mail below. I still am struggling to understand what the reason for the wrapper script is and how the plugin is supposed to call he helper programme, is the message passed to it via stdout?
This is what I find in the syslog: Apr 12 21:54:32 vm10 imap: antispam: plugin initialising (2.0-notgit) Apr 12 21:54:32 vm10 imap: antispam: verbose debug enabled Apr 12 21:54:32 vm10 imap: antispam: "SPAM" is exact match spam folder Apr 12 21:54:32 vm10 imap: antispam: "Junk" is exact match spam folder Apr 12 21:54:32 vm10 imap: antispam: no unsure folders Apr 12 21:54:32 vm10 imap: antispam: "trash" is exact match trash folder Apr 12 21:54:32 vm10 imap: antispam: "Trash" is exact match trash folder Apr 12 21:54:32 vm10 imap: antispam: "Deleted Items" is exact match trash folder Apr 12 21:54:32 vm10 imap: antispam: "Deleted Messages" is exact match trash folder Apr 12 21:54:32 vm10 imap: antispam: pipe backend spam argument = --spam Apr 12 21:54:32 vm10 imap: antispam: pipe backend not-spam argument = --ham Apr 12 21:54:32 vm10 imap: antispam: pipe backend program = /usr/bin/sa-learn Apr 12 21:54:32 vm10 imap: antispam: pipe backend program arg[0] = --username=[blanked out]@rohr.org Apr 12 21:54:32 vm10 imap: antispam: pipe backend tmpdir /tmp
It doesn't hint at any particular error, however, when I tried to move a message to junk, nothing happens, the message just stays at its original place and I see that the contents of the bayes db is unchanged, so somehow nothing happens. When I disable the plugin, moving messages to Junk works again.
I wonder whether this is a bug in a particular version, it is quite ancient software, the server is running on Ubuntu precise.
Cheers,
Johannes
Am 12.04.2016 um 20:18 schrieb J. Echter:
Hi,
here (Centos 7) its already integrated...
Just add something like this:
plugin {
antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_sendmail_args = -u amavis antispam_mail_spam = --spam antispam_mail_notspam = --ham }
to dovecot.conf
Cheers
Jürgen
Am 12.04.2016 um 20:14 schrieb Johannes Rohr:
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise withthe antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp }
I have tried the following wrapper script: #!/bin/bash
set -x
LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 &&
rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0It somehow isn't working. I can call the wrapper script manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user --ham
< somehamBut it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
Cheers,
Johannes
On 12-04-16 20:14, Johannes Rohr wrote:
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise withthe antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot
You need to specify the argument list as a list seperated by semicolons, per example on the wiki:
antispam_pipe_program_args = --username;%u
The verbose debug should give you some logging to work with. Also, what does the wrapper script log when dovecot runs it? How does that differ from a manual run?
antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp}
I have tried the following wrapper script: #!/bin/bash
set -x
LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 &&
rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0It somehow isn't working. I can call the wrapper script manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user --ham
< somehamBut it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
Cheers,
Johannes
Hi Tom,
Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:
On 12-04-16 20:14, Johannes Rohr wrote:
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise withthe antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot
You need to specify the argument list as a list seperated by semicolons, per example on the wiki:
antispam_pipe_program_args = --username;%u
Not sure about that, because sa-learn expects the parameter --username=user@na.me, this is one parameter, not two, and it works for me locally, where I also set up dovecot to test. The main differences are that the versions I have here are newer, I'm running Debian sid, the server is on Ubuntu Precise, and second, the server is handling virtual users, locally I have only real Unix users.
Also, dovecot doesn't complain about configuration errors regard --username=%u. What other way would be there to get the parameter passed to sa-learn the way I want?
btw, I tried this, just to be sure, but the result was the same.
The verbose debug should give you some logging to work with.
Yes, and I pasted the output. I can of course repeat:
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 1 --username=johannes@rohr.org Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run program failed with exit code -1
Also, what does the wrapper script log when dovecot runs it?
I have actually ditched the wrapper script because it works without locally, I have configured the plugin to call sa-learn directly.
How does that differ from a manual run? sa-learn works just fine with a manual run. It seems as if the plugin is passing the parameters wrongly, but the debugging output doesn't reveal the actual command, so it's just guesswork.
Cheers,
Johannes
antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp}
I have tried the following wrapper script: #!/bin/bash
set -x
LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 &&
rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0It somehow isn't working. I can call the wrapper script manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user --ham
< somehamBut it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
Cheers,
Johannes
Hi List,
does the antispam plugin any way of actually seeing how it is calling the pipe program and how it is failing? I can't reproduce the failure when I call sa-learn manually. Can someone tell me, with which UID the call happens, is it the UID the current dovecot process is running under (which would be vmail, as it is a virtual user setup) Cheers,
Johannes
Am 12.04.2016 um 23:09 schrieb Johannes Rohr:
Hi Tom,
Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:
On 12-04-16 20:14, Johannes Rohr wrote:
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precisewith the antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot
You need to specify the argument list as a list seperated by semicolons, per example on the wiki:
antispam_pipe_program_args = --username;%u
Not sure about that, because sa-learn expects the parameter --username=user@na.me, this is one parameter, not two, and it works for me locally, where I also set up dovecot to test. The main differences are that the versions I have here are newer, I'm running Debian sid, the server is on Ubuntu Precise, and second, the server is handling virtual users, locally I have only real Unix users.
Also, dovecot doesn't complain about configuration errors regard --username=%u. What other way would be there to get the parameter passed to sa-learn the way I want?
btw, I tried this, just to be sure, but the result was the same.
The verbose debug should give you some logging to work with.
Yes, and I pasted the output. I can of course repeat:
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 1 --username=johannes@rohr.org Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run program failed with exit code -1
Also, what does the wrapper script log when dovecot runs it?
I have actually ditched the wrapper script because it works without locally, I have configured the plugin to call sa-learn directly.
How does that differ from a manual run? sa-learn works just fine with a manual run. It seems as if the plugin is passing the parameters wrongly, but the debugging output doesn't reveal the actual command, so it's just guesswork.
Cheers,
Johannes
antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp}
I have tried the following wrapper script: #!/bin/bash
set -x
LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 &&
rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0It somehow isn't working. I can call the wrapper script manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user --ham
< somehamBut it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
Cheers,
Johannes
On 13-04-16 08:27, Johannes Rohr wrote:
Hi List,
does the antispam plugin any way of actually seeing how it is calling the pipe program and how it is failing? I can't reproduce the failure when I call sa-learn manually. Can someone tell me, with which UID the call happens, is it the UID the current dovecot process is running under (which would be vmail, as it is a virtual user setup) Cheers,
Hi Johannes,
This is exactly what the wrapper script is supposed to log for you, or which is trivially to add to its output. The wrapper script is not actually useful for production, but it is for debugging. That is why I asked for its logging, and any differences between manual run and a dovecot run.
Regards, Tom
Johannes
Am 12.04.2016 um 23:09 schrieb Johannes Rohr:
Hi Tom,
Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:
On 12-04-16 20:14, Johannes Rohr wrote:
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with the antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot You need to specify the argument list as a list seperated by semicolons, per example on the wiki:
antispam_pipe_program_args = --username;%u
Not sure about that, because sa-learn expects the parameter --username=user@na.me, this is one parameter, not two, and it works for me locally, where I also set up dovecot to test. The main differences are that the versions I have here are newer, I'm running Debian sid, the server is on Ubuntu Precise, and second, the server is handling virtual users, locally I have only real Unix users.
Also, dovecot doesn't complain about configuration errors regard --username=%u. What other way would be there to get the parameter passed to sa-learn the way I want?
btw, I tried this, just to be sure, but the result was the same.
The verbose debug should give you some logging to work with.
Yes, and I pasted the output. I can of course repeat:
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 1 --username=johannes@rohr.org Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run program failed with exit code -1
Also, what does the wrapper script log when dovecot runs it?
I have actually ditched the wrapper script because it works without locally, I have configured the plugin to call sa-learn directly.
How does that differ from a manual run? sa-learn works just fine with a manual run. It seems as if the plugin is passing the parameters wrongly, but the debugging output doesn't reveal the actual command, so it's just guesswork.
Cheers,
Johannes
antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp }
I have tried the following wrapper script: #!/bin/bash # set -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0
It somehow isn't working. I can call the wrapper script manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user --ham \ < someham
But it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
Cheers,
Johannes
Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:
On 13-04-16 08:27, Johannes Rohr wrote:
Hi List,
does the antispam plugin any way of actually seeing how it is calling the pipe program and how it is failing? I can't reproduce the failure when I call sa-learn manually. Can someone tell me, with which UID the call happens, is it the UID the current dovecot process is running under (which would be vmail, as it is a virtual user setup) Cheers,
Hi Johannes, http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting This is exactly what the wrapper script is supposed to log for you, or which is trivially to add to its output. The wrapper script is not actually useful for production, but it is for debugging. That is why I asked for its logging, and any differences between manual run and a dovecot run.
When I used the wrapper script recommended at http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID and parameters (--username=some@na.me --ham). The wrapper script noted in the dovecot wiki seems broken. Note the commented out line:
#echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt
It seems like the second half of the line "cat<&0 >> /tmp/sendmail-msg-$$.txt" was originally a separate line, catting the message, which has been passed by the plugin via STDOUT to a file. Without that command, the call of sa-learn in the next line has no file to read from:
/usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f /tmp/sendmail-msg-$$.txt &
So I moved the cat command to a separate uncommented line and to be sure I also commented out the
rm -f /tmp/sendmail-msg-$$.txt &
to verify that the file actually contains something, (why is there an "&" terminating the line, which would have the rm command executed in the background?)
But it seems the
/tmp/sendmail-msg-$$.txt &
was empty and the whole operation continued to fail. Could it be that the plugin fails to pass the actual message?
Cheers,
Johannes
Regards, Tom
Johannes
Am 12.04.2016 um 23:09 schrieb Johannes Rohr:
Hi Tom,
Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:
On 12-04-16 20:14, Johannes Rohr wrote:
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with the antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot You need to specify the argument list as a list seperated by semicolons, per example on the wiki:
antispam_pipe_program_args = --username;%u Not sure about that, because sa-learn expects the parameter --username=user@na.me, this is one parameter, not two, and it works for me locally, where I also set up dovecot to test. The main differences are that the versions I have here are newer, I'm running Debian sid, the server is on Ubuntu Precise, and second, the server is handling virtual users, locally I have only real Unix users.
Also, dovecot doesn't complain about configuration errors regard --username=%u. What other way would be there to get the parameter passed to sa-learn the way I want?
btw, I tried this, just to be sure, but the result was the same.
The verbose debug should give you some logging to work with. Yes, and I pasted the output. I can of course repeat:
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 1 --username=johannes@rohr.org Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run program failed with exit code -1
Also, what does the wrapper script log when dovecot runs it? I have actually ditched the wrapper script because it works without locally, I have configured the plugin to call sa-learn directly.
How does that differ from a manual run? sa-learn works just fine with a manual run. It seems as if the plugin is passing the parameters wrongly, but the debugging output doesn't reveal the actual command, so it's just guesswork.
Cheers,
Johannes
antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp }
I have tried the following wrapper script: #!/bin/bash # set -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0
It somehow isn't working. I can call the wrapper script manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user --ham \ < someham
But it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
Cheers,
Johannes
tweaked the wrapper script to make it work. It seems like the sa-learn call silently fails:
"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log starting log 25200-start (--username=johannes@rohr.org --spam) libgcc_s.so.1 must be installed for pthread_cancel to work 25200-end
Does the above error message have anything to do with it? Running the script & sa-learn manually works fine. do virtual users change something how dovecot calls programs, such as chrooting them?
Cheers,
Johannes
Am 13.04.2016 um 09:45 schrieb Johannes Rohr:
Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:
On 13-04-16 08:27, Johannes Rohr wrote:
Hi List,
does the antispam plugin any way of actually seeing how it is calling the pipe program and how it is failing? I can't reproduce the failure when I call sa-learn manually. Can someone tell me, with which UID the call happens, is it the UID the current dovecot process is running under (which would be vmail, as it is a virtual user setup) Cheers,
Hi Johannes, http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting This is exactly what the wrapper script is supposed to log for you, or which is trivially to add to its output. The wrapper script is not actually useful for production, but it is for debugging. That is why I asked for its logging, and any differences between manual run and a dovecot run.
When I used the wrapper script recommended at http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID and parameters (--username=some@na.me --ham). The wrapper script noted in the dovecot wiki seems broken. Note the commented out line:
#echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt
It seems like the second half of the line "cat<&0 >> /tmp/sendmail-msg-$$.txt" was originally a separate line, catting the message, which has been passed by the plugin via STDOUT to a file. Without that command, the call of sa-learn in the next line has no file to read from:
/usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f /tmp/sendmail-msg-$$.txt &
So I moved the cat command to a separate uncommented line and to be sure I also commented out the
rm -f /tmp/sendmail-msg-$$.txt &
to verify that the file actually contains something, (why is there an "&" terminating the line, which would have the rm command executed in the background?)
But it seems the
/tmp/sendmail-msg-$$.txt &
was empty and the whole operation continued to fail. Could it be that the plugin fails to pass the actual message?
Cheers,
Johannes
Regards, Tom
Johannes
Am 12.04.2016 um 23:09 schrieb Johannes Rohr:
Hi Tom,
Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:
On 12-04-16 20:14, Johannes Rohr wrote:
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with the antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot You need to specify the argument list as a list seperated by semicolons, per example on the wiki:
antispam_pipe_program_args = --username;%u Not sure about that, because sa-learn expects the parameter --username=user@na.me, this is one parameter, not two, and it works for me locally, where I also set up dovecot to test. The main differences are that the versions I have here are newer, I'm running Debian sid, the server is on Ubuntu Precise, and second, the server is handling virtual users, locally I have only real Unix users.
Also, dovecot doesn't complain about configuration errors regard --username=%u. What other way would be there to get the parameter passed to sa-learn the way I want?
btw, I tried this, just to be sure, but the result was the same.
The verbose debug should give you some logging to work with. Yes, and I pasted the output. I can of course repeat:
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 1 --username=johannes@rohr.org Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run program failed with exit code -1
Also, what does the wrapper script log when dovecot runs it? I have actually ditched the wrapper script because it works without locally, I have configured the plugin to call sa-learn directly.
How does that differ from a manual run? sa-learn works just fine with a manual run. It seems as if the plugin is passing the parameters wrongly, but the debugging output doesn't reveal the actual command, so it's just guesswork.
Cheers,
Johannes
antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp }
I have tried the following wrapper script: #!/bin/bash # set -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0
It somehow isn't working. I can call the wrapper script manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user --ham \ < someham
But it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
Cheers,
Johannes
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, 13 Apr 2016, Johannes Rohr wrote:
tweaked the wrapper script to make it work. It seems like the sa-learn call
See the page again.
silently fails:
"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log starting log 25200-start (--username=johannes@rohr.org --spam) libgcc_s.so.1 must be installed for pthread_cancel to work 25200-end
Does the above error message have anything to do with it? Running the script & sa-learn manually works fine. do virtual users change something how dovecot calls programs, such as chrooting them?
Maybe, the script uses different search paths? Uncomment "# env" to verify.
Cheers,
Johannes
Am 13.04.2016 um 09:45 schrieb Johannes Rohr:
Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:
On 13-04-16 08:27, Johannes Rohr wrote:
Hi List,
does the antispam plugin any way of actually seeing how it is calling the pipe program and how it is failing? I can't reproduce the failure when I call sa-learn manually. Can someone tell me, with which UID the call happens, is it the UID the current dovecot process is running under (which would be vmail, as it is a virtual user setup) Cheers,
Hi Johannes, http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting This is exactly what the wrapper script is supposed to log for you, or which is trivially to add to its output. The wrapper script is not actually useful for production, but it is for debugging. That is why I asked for its logging, and any differences between manual run and a dovecot run.
When I used the wrapper script recommended at http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID and parameters (--username=some@na.me --ham). The wrapper script noted in the dovecot wiki seems broken. Note the commented out line:
#echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt
It seems like the second half of the line "cat<&0 >> /tmp/sendmail-msg-$$.txt" was originally a separate line, catting the message, which has been passed by the plugin via STDOUT to a file. Without that command, the call of sa-learn in the next line has no file to read from:
/usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f /tmp/sendmail-msg-$$.txt &
So I moved the cat command to a separate uncommented line and to be sure I also commented out the
rm -f /tmp/sendmail-msg-$$.txt &
to verify that the file actually contains something, (why is there an "&" terminating the line, which would have the rm command executed in the background?)
But it seems the
/tmp/sendmail-msg-$$.txt &
was empty and the whole operation continued to fail. Could it be that the plugin fails to pass the actual message?
Cheers,
Johannes
Regards, Tom
Johannes
Am 12.04.2016 um 23:09 schrieb Johannes Rohr:
Hi Tom,
Am 12.04.2016 um 22:26 schrieb Tom Hendrikx:
On 12-04-16 20:14, Johannes Rohr wrote: > Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise > with the antispam plugin in version 2.0+20120225-2 and > spamassassin at version 3.2.2 > > I have been trying and failed to get the pipe backend of the > antispam plugin to work. Spamassin by itself works, a manual > call of sa-learn works fine. Bayes data is stored in a mysql > DB. > > I have the following configuration in > /etc/dovecot/conf.d/90-plugin.conf > > plugin { #setting_name = value sieve=~/.dovecot.sieve > sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam > antispam_pipe_program_notspam_arg = --ham > antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh > antispam_pipe_program_args = --username=%u # % expansion done > by dovecot You need to specify the argument list as a list seperated by semicolons, per example on the wiki:
antispam_pipe_program_args = --username;%u Not sure about that, because sa-learn expects the parameter --username=user@na.me, this is one parameter, not two, and it works for me locally, where I also set up dovecot to test. The main differences are that the versions I have here are newer, I'm running Debian sid, the server is on Ubuntu Precise, and second, the server is handling virtual users, locally I have only real Unix users.
Also, dovecot doesn't complain about configuration errors regard --username=%u. What other way would be there to get the parameter passed to sa-learn the way I want?
btw, I tried this, just to be sure, but the result was the same.
The verbose debug should give you some logging to work with. Yes, and I pasted the output. I can of course repeat:
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 1 --username=johannes@rohr.org Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run program failed with exit code -1
Also, what does the wrapper script log when dovecot runs it? I have actually ditched the wrapper script because it works without locally, I have configured the plugin to call sa-learn directly.
How does that differ from a manual run? sa-learn works just fine with a manual run. It seems as if the plugin is passing the parameters wrongly, but the debugging output doesn't reveal the actual command, so it's just guesswork.
Cheers,
Johannes
> antispam_trash = trash;Trash;Deleted Items;Deleted Messages > antispam_spam = SPAM;Junk antispam_backend = pipe > antispam_verbose_debug = 1 antispam_debug_target = syslog > antispam_pipe_tmpdir = /tmp } > > I have tried the following wrapper script: #!/bin/bash # set > -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> > $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> > /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D > /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f > /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0 > > It somehow isn't working. I can call the wrapper script > manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh > --username=some@mail.user --ham \ < someham > > But it doesn't work from dovecot. It seems the script is > called. but no new patterns show up in the bayes DB. > Unfortunately, the plugin seems to be documented poorly. How > does it call the programme it pipes to? Is it something like > $programme --par1 --par2 < message ? Why can't / shouldn't I > call sa-learn directly? All the instructions I found on the in > internet contained a wrapper script, but none bothered to > explain why you would actually need it. > > I would be very grateful, if someone could enlighten me on > that... > > Cheers, > > Johannes
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1
iQEVAwUBVw4R8Hz1H7kL/d9rAQKs7gf9F0QTDOugkaoIFCbzPlFohhhXGOKt6awv AWVCqJdZgwmnYqH2WTn+xyueIG+HDZUrHLQB7CirjOIB2J8YWyprcsaBkcnZwxBW kzklgjp9zK9ttHD7HHhkw7o4LwVjkOGtXJR49vz9bWL0P0N05dtdpI+HZuLS/Inx qz+F7FLd7th2SFrFSRSKTlaVRyv/Ta6fh50tra+DQe5f4NblUwHe7dOUmXKYxs+U Ievm4eKTxQA0GuQwyQSgKVqIKWZdAmA1bbmhbREK+7AEROCDt8XBYK1EJsRsG7LH fGh6+kLi5QHihbumJtv14ba6P137UjnCwQ+6y8X38tDY50hOnVXASw== =bmyk -----END PGP SIGNATURE-----
Am 13.04.2016 um 11:31 schrieb Steffen Kaiser:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, 13 Apr 2016, Johannes Rohr wrote:
tweaked the wrapper script to make it work. It seems like the sa-learn call
See the page again.
Can you kindly be more precise? The wrapper script at the page is broken, it cannot work, as it fails to write the message passed from the plugin to /tmp/sendmail-msg-$$.txt. So the input passed to sa-learn is missing, which I fixed by inserting "cat > /tmp/sendmail-msg-$$.txt" at the beginning of the script. However, this is not the actual issue, as sa-learn even fails even when the script is fixed.
silently fails:
"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log starting log 25200-start (--username=johannes@rohr.org --spam) libgcc_s.so.1 must be installed for pthread_cancel to work 25200-end
Does the above error message have anything to do with it? Running the script & sa-learn manually works fine. do virtual users change something how dovecot calls programs, such as chrooting them?
Maybe, the script uses different search paths? Uncomment "# env" to verify.
I'll try later, although I doubt that environment variables are to blame. (For now we have a cronjob which is scanning the Junk directories periodically as a workaround.) I have to change back a whole bunch of config files to enable the plugin again. It seems to be a bug which hits daemons that run as a limited user, see https://bugs.launchpad.net/ubuntu/+source/gcc-3.3/+bug/40285/comments/30
I am a bit worried because the same error is reported from newer Ubuntu versions, so unless the spamassassin guys have fixed it their way, it might not go away by upgrading
Cheers,
Johannes
Cheers,
Johannes
Am 13.04.2016 um 09:45 schrieb Johannes Rohr:
Am 13.04.2016 um 09:17 schrieb Tom Hendrikx:
On 13-04-16 08:27, Johannes Rohr wrote:
Hi List,
does the antispam plugin any way of actually seeing how it is calling the pipe program and how it is failing? I can't reproduce the failure when I call sa-learn manually. Can someone tell me, with which UID the call happens, is it the UID the current dovecot process is running under (which would be vmail, as it is a virtual user setup) Cheers,
Hi Johannes, http://wiki2.dovecot.org/Pigeonhole/Sieve/Troubleshooting This is exactly what the wrapper script is supposed to log for you, or which is trivially to add to its output. The wrapper script is not actually useful for production, but it is for debugging. That is why I asked for its logging, and any differences between manual run and a dovecot run.
When I used the wrapper script recommended at http://wiki2.dovecot.org/Plugins/Antispam, it merely reported the PID and parameters (--username=some@na.me --ham). The wrapper script noted in the dovecot wiki seems broken. Note the commented out line:
#echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt
It seems like the second half of the line "cat<&0 >> /tmp/sendmail-msg-$$.txt" was originally a separate line, catting the message, which has been passed by the plugin via STDOUT to a file. Without that command, the call of sa-learn in the next line has no file to read from:
/usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt && rm -f /tmp/sendmail-msg-$$.txt &
So I moved the cat command to a separate uncommented line and to be sure I also commented out the
rm -f /tmp/sendmail-msg-$$.txt &
to verify that the file actually contains something, (why is there an "&" terminating the line, which would have the rm command executed in the background?)
But it seems the
/tmp/sendmail-msg-$$.txt &
was empty and the whole operation continued to fail. Could it be that the plugin fails to pass the actual message?
Cheers,
Johannes
Regards, Tom
Johannes
Am 12.04.2016 um 23:09 schrieb Johannes Rohr:
Hi Tom,
Am 12.04.2016 um 22:26 schrieb Tom Hendrikx: > On 12-04-16 20:14, Johannes Rohr wrote: >> Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise >> with the antispam plugin in version 2.0+20120225-2 and >> spamassassin at version 3.2.2 >> >> I have been trying and failed to get the pipe backend of the >> antispam plugin to work. Spamassin by itself works, a manual >> call of sa-learn works fine. Bayes data is stored in a mysql >> DB. >> >> I have the following configuration in >> /etc/dovecot/conf.d/90-plugin.conf >> >> plugin { #setting_name = value sieve=~/.dovecot.sieve >> sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam >> antispam_pipe_program_notspam_arg = --ham >> antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh >> antispam_pipe_program_args = --username=%u # % expansion done >> by dovecot > You need to specify the argument list as a list seperated by > semicolons, per example on the wiki: > > antispam_pipe_program_args = --username;%u Not sure about that, because sa-learn expects the parameter --username=user@na.me, this is one parameter, not two, and it works for me locally, where I also set up dovecot to test. The main differences are that the versions I have here are newer, I'm running Debian sid, the server is on Ubuntu Precise, and second, the server is handling virtual users, locally I have only real Unix users.
Also, dovecot doesn't complain about configuration errors regard --username=%u. What other way would be there to get the parameter passed to sa-learn the way I want?
btw, I tried this, just to be sure, but the result was the same.
> The verbose debug should give you some logging to work with. Yes, and I pasted the output. I can of course repeat:
Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(INBhttp://wiki2.dovecot.org/Pigeonhole/Sieve/TroubleshootingOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_trash(Junk): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: from trash: 0, to trash: 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_spam(Junk): 1 Apr 12 22:53:02 vm10 imap: antispam: mailbox_is_unsure(INBOX): 0 Apr 12 22:53:02 vm10 imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program /usr/bin/sa-learn Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 1 --username=johannes@rohr.org Apr 12 22:53:02 vm10 imap: antispam: running mailtrain backend program parameter 2 --spam Apr 12 22:53:03 vm10 imap: antispam: run program failed with exit code -1
> Also, what does the wrapper script log when dovecot runs it? I have actually ditched the wrapper script because it works without locally, I have configured the plugin to call sa-learn directly.
> How does that differ from a manual run? sa-learn works just fine with a manual run. It seems as if the plugin is passing the parameters wrongly, but the debugging output doesn't reveal the actual command, so it's just guesswork.
Cheers,
Johannes
> >> antispam_trash = trash;Trash;Deleted Items;Deleted Messages >> antispam_spam = SPAM;Junk antispam_backend = pipe >> antispam_verbose_debug = 1 antispam_debug_target = syslog >> antispam_pipe_tmpdir = /tmp } >> >> I have tried the following wrapper script: #!/bin/bash # set >> -x LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D >> /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 && \ rm -f >> /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0 >> >> It somehow isn't working. I can call the wrapper script >> manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh >> --username=some@mail.user --ham \ < someham >> >> But it doesn't work from dovecot. It seems the script is >> called. but no new patterns show up in the bayes DB. >> Unfortunately, the plugin seems to be documented poorly. How >> does it call the programme it pipes to? Is it something like >> $programme --par1 --par2 < message ? Why can't / shouldn't I >> call sa-learn directly? All the instructions I found on the in >> internet contained a wrapper script, but none bothered to >> explain why you would actually need it. >> >> I would be very grateful, if someone could enlighten me on >> that... >> >> Cheers, >> >> Johannes
- -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1
iQEVAwUBVw4R8Hz1H7kL/d9rAQKs7gf9F0QTDOugkaoIFCbzPlFohhhXGOKt6awv AWVCqJdZgwmnYqH2WTn+xyueIG+HDZUrHLQB7CirjOIB2J8YWyprcsaBkcnZwxBW kzklgjp9zK9ttHD7HHhkw7o4LwVjkOGtXJR49vz9bWL0P0N05dtdpI+HZuLS/Inx qz+F7FLd7th2SFrFSRSKTlaVRyv/Ta6fh50tra+DQe5f4NblUwHe7dOUmXKYxs+U Ievm4eKTxQA0GuQwyQSgKVqIKWZdAmA1bbmhbREK+7AEROCDt8XBYK1EJsRsG7LH fGh6+kLi5QHihbumJtv14ba6P137UjnCwQ+6y8X38tDY50hOnVXASw== =bmyk -----END PGP SIGNATURE-----
Am 13.04.2016 um 10:35 schrieb Johannes Rohr <johannes@rohr.org>:
tweaked the wrapper script to make it work. It seems like the sa-learn call silently fails:
"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log starting log 25200-start (--username=johannes@rohr.org --spam) libgcc_s.so.1 must be installed for pthread_cancel to work 25200-end
Does the above error message have anything to do with it? Running the script & sa-learn manually works fine. do virtual users change something how dovecot calls programs, such as chrooting them?
Here [1] is an old thread about that problem.In short: It seems to be a problem in sa-learn that is triggered by the privilege drop that dovecot does when starting.
I have the same problem and I have not found a solution yet.
[1] http://www.dovecot.org/list/dovecot/2013-November/093378.html
Cheers,
Johannes
Am 13.04.2016 um 11:54 schrieb Florian Mutter:
Am 13.04.2016 um 10:35 schrieb Johannes Rohr <johannes@rohr.org>:
tweaked the wrapper script to make it work. It seems like the sa-learn call silently fails:
"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log starting log 25200-start (--username=johannes@rohr.org --spam) libgcc_s.so.1 must be installed for pthread_cancel to work 25200-end
Does the above error message have anything to do with it? Running the script & sa-learn manually works fine. do virtual users change something how dovecot calls programs, such as chrooting them? Here [1] is an old thread about that problem.In short: It seems to be a problem in sa-learn that is triggered by the privilege drop that dovecot does when starting.
I have the same problem and I have not found a solution yet.
Yikes! Our server is running a fairly dated Ubuntu Precise, what's your version? Did an upgrade fail to fix it? I am not seeing this error on Debian Sid but I am not using virtual users over here, so that might be the cause. Did you report the bug to the Spamassassin bugzilla?
Cheers,
Johannes
[1] http://www.dovecot.org/list/dovecot/2013-November/093378.html
Cheers,
Johannes
Am 13.04.2016 um 12:14 schrieb Johannes Rohr <johannes@rohr.org>:
Yikes! Our server is running a fairly dated Ubuntu Precise, what's your version? Did an upgrade fail to fix it? I am not seeing this error on Debian Sid but I am not using virtual users over here, so that might be the cause. Did you report the bug to the Spamassassin bugzilla?
I’m running Debian 8 (Jessie). Dovecot is version 2.2.13, Dovecot Antispam is 2.0+20130912-2 and SpamAssassin is 3.4.0. I did not fill a bug but I see you already did it.
Am 13.04.2016 um 11:54 schrieb Florian Mutter:
Am 13.04.2016 um 10:35 schrieb Johannes Rohr <johannes@rohr.org>:
tweaked the wrapper script to make it work. It seems like the sa-learn call silently fails:
"/usr/local/bin/sa-learn-pipe.sh" 10L, 350C geschrieben root@vm10:/etc/dovecot/conf.d# cat /tmp/sa-learn-pipe.log starting log 25200-start (--username=johannes@rohr.org --spam) libgcc_s.so.1 must be installed for pthread_cancel to work 25200-end
Does the above error message have anything to do with it? Running the script & sa-learn manually works fine. do virtual users change something how dovecot calls programs, such as chrooting them? Here [1] is an old thread about that problem.In short: It seems to be a problem in sa-learn that is triggered by the privilege drop that dovecot does when starting.
I have the same problem and I have not found a solution yet.
I have just filed a bug with spamassassin, https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7309
I am not sure that they will act upon it, as it is an old version, and SA isn't the ultimate cause anyway.
Cheers,
Johannes
[1] http://www.dovecot.org/list/dovecot/2013-November/093378.html
Cheers,
Johannes
Johannes -
I'm running 2.2.9 under Ubuntu 14.04. I gave up on using the pipe backend, just could not get the damn thing to work. I would up using spool2dir and incron, which works perfectly. The issue was that sa-learn would cause a pthread_cancel error with libgcc_s.so.1
Below is an excerpt from my install script :
############################################
Enable antispam - Damn, not working right with pipe backend
now using spool2dir and incron
if [ -e /etc/spamassassin ]; then sed -i " s/^ #mail_plugins.*/ mail_plugins = \$mail_plugins antispam ${DOVENOTIFY}/ s/^ #mail_max_userip.*/mail_max_userip_connections = 20/ " /etc/dovecot/conf.d/20-imap.conf cat > /etc/dovecot/conf.d/99-Installerbox-antispam.conf << EOF
antispam configuration
plugin { antispam_debug_target = syslog
antispam_verbose_debug = 1
antispam_trash_pattern = Trash;Deleted * antispam_spam = Junk;Spam
antispam_backend = spool2dir
antispam_spool2dir_spam =
/var/cache/dovecot-antispam/spam/%%020lu-%u-%%05luS
antispam_spool2dir_notspam =
/var/cache/dovecot-antispam/ham/%%020lu-%u-%%05luH
pipe backend not working with sa-learn - causes pthread_cancel error
with libgcc_s.so.1
antispam_backend = pipe
antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
antispam_pipe_program_args = --for;%u
antispam_pipe_program_spam_arg = --spam
antispam_pipe_program_notspam_arg = --ham
antispam_pipe_tmpdir = /tmp
} EOF
# incron watches the spam/ham spool dirs, calls sa-learn-pipe.sh to handle echo "root" >> /etc/incron.allow mkdir -p /var/cache/dovecot-antispam/spam /var/cache/dovecot-antispam/ham chown -R ${VMAIL_ID}.dovecot /var/cache/dovecot-antispam/ cat > /var/spool/incron/root << "EOF" /var/cache/dovecot-antispam/spam IN_CLOSE_WRITE /usr/local/bin/sa-learn-pipe.sh --spam /var/cache/dovecot-antispam/spam/$# /var/cache/dovecot-antispam/ham IN_CLOSE_WRITE /usr/local/bin/sa-learn-pipe.sh --ham /var/cache/dovecot-antispam/ham/$# EOF chgrp incron /var/spool/incron/root chmod 600 /var/spool/incron/root
# inotify needs a little more room to breath - default of 128 too low cat > /etc/sysctl.d/60-inotify.conf << EOF
inotify changes for Dovecot
http://dovecot.org/list/dovecot/2011-March/058300.html
Defaults are
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 8192
fs.inotify.max_user_instances = 2048 EOF
# spamassassin learning script
cat > /usr/local/bin/sa-learn-pipe.sh << "EOFSPAM"#!/bin/bash
Pipe script to learn/unlearn single email file
Set to read from file or from stdin
From stdin to accomodate dovecot-antispam pipe backend (nor currently
working)
echo /usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt
FILE=echo $* | sed "s/^.* //"
echo "$$-start ($*)" >> /var/log/sa-learn-pipe.log
echo -n "$$ " >> /var/log/sa-learn-pipe.log
egrep --no-filename "^Subject: " /tmp/sendmail-msg-$$.txt ${FILE} | head
-1 >> /var/log/sa-learn-pipe.log
cat<&0 >> /tmp/sendmail-msg-$$.txt
/usr/bin/sa-learn --progress $* /tmp/sendmail-msg-$$.txt >>
/tmp/sa-learn-pipe.$$.log 2>&1
echo $$ sa-learn rc=$? id=$(id) HOME=$HOME >> /var/log/sa-learn-pipe.log
while read line; do echo $$-sa-learn "$line" >> /var/log/sa-learn-pipe.log done < /tmp/sa-learn-pipe.$$.log
rm -f /tmp/sendmail-msg-$$.txt /tmp/sa-learn-pipe.$$.log rm -f ${FILE} echo "$$-end" >> /var/log/sa-learn-pipe.log
exit 0 EOFSPAM chmod 755 /usr/local/bin/sa-learn-pipe.sh touch /var/log/sa-learn-pipe.log chown ${VMAIL_ID}.dovecot /var/log/sa-learn-pipe.log chmod 660 /var/log/sa-learn-pipe.log cat > /etc/logrotate.d/sa-learn-pipe.log << EOFLOG /var/log/sa-learn-pipe.log { daily missingok rotate 10 compress delaycompress notifempty create 660 ${VMAIL_ID} dovecot } EOFLOG fi # spamassassin
############################################
On 2016-04-12 14:14, Johannes Rohr wrote:
Hi, my setup is a dovecot 2.0.19 IMAP server on Ubuntu Precise with the antispam plugin in version 2.0+20120225-2 and spamassassin at version 3.2.2
I have been trying and failed to get the pipe backend of the antispam plugin to work. Spamassin by itself works, a manual call of sa-learn works fine. Bayes data is stored in a mysql DB.
I have the following configuration in /etc/dovecot/conf.d/90-plugin.conf
plugin { #setting_name = value sieve=~/.dovecot.sieve sieve_dir=~/sieve antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_args = --username=%u # % expansion done by dovecot antispam_trash = trash;Trash;Deleted Items;Deleted Messages antispam_spam = SPAM;Junk antispam_backend = pipe antispam_verbose_debug = 1 antispam_debug_target = syslog antispam_pipe_tmpdir = /tmp }
I have tried the following wrapper script: #!/bin/bash
set -x
LOGFILE=/tmp/sa-learn-pipe.log echo "$$-start ($*)" >> $LOGFILE #echo $* > /tmp/sendmail-parms.txt cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn $* -D /tmp/sendmail-msg-$$.txt >> $LOGFILE 2>&1 &&
rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> $LOGFILE exit 0It somehow isn't working. I can call the wrapper script manually e.g. sudo -u vmail /usr/local/bin/sa-learn-pipe.sh --username=some@mail.user --ham
< somehamBut it doesn't work from dovecot. It seems the script is called. but no new patterns show up in the bayes DB. Unfortunately, the plugin seems to be documented poorly. How does it call the programme it pipes to? Is it something like $programme --par1 --par2 < message ? Why can't / shouldn't I call sa-learn directly? All the instructions I found on the in internet contained a wrapper script, but none bothered to explain why you would actually need it.
I would be very grateful, if someone could enlighten me on that...
-- Dean Carpenter deano is at areyes dot com 203 six oh four 6644
participants (6)
-
deano-dovecot＠areyes.com
-
Florian Mutter
-
J. Echter
-
Johannes Rohr
-
Steffen Kaiser
-
Tom Hendrikx