This might be a fairly long message, but I wanted to be sure to include as much information as possible. I'm having an issue with the dovecot-antispam plugin in that it seems to be unable to successfully run anything from the pipe backend. To qualify that, they run, but they fail ...
Running /usr/bin/sa-learn directly always returns with an error code of 1, and the bayes DB isn't actually updated. Running the /usr/local/bin/sa-learn-pipe.sh script from the example will run sa-learn successfully, but sa-learn fails partway through, right on a sql DB access. I have both bayes and FuzzyOcr data stored in mysql, and the error occurs on accessing either one. That is, I've tested with FuzzyOcr enabled, and it fails on the db access to the FuzzyOcr DB, and I've tried it with FuzzyOcr disabled. In that case, it fails on the access to the bayes DB.
The error line right there is :
libgcc_s.so.1 must be installed for pthread_cancel to workSince sa-learn terminated prematurely, the bayes DB isn't updated at all.
Running /usr/bin/sa-learn or the script /usr/local/bin/sa-learn-pipe.sh directly from cmdline works fine though - sa-learn completes correctly and updates the bayes DB properly, I can see the new tokens.
$ sa-learn-pipe.sh --spam </usr/share/doc/spamassassin/examples/sample-spam.txt $ sa-learn --spam /usr/share/doc/spamassassin/examples/sample-spam.txt
I think it would be great to have a dedicated logfile for antispam, one that would capture both stdout as well as stderr.
These tests are being done on a clean Ubuntu 13.04 system. I'm rebuilding the VM repeatedly as I configure the various pieces, and can test any changes or whatever you might suggest very easily. I've fully scripted the install and configuration of Exim4, Dovecot, Spamassassin, Clamav, Percona/Mysql, Roundcube, and can build a full VM under Virtualbox or any VPS in less than 10 minutes.
Ubuntu 13.04 Dovecot 2.1.7 dovecot-antispam 2.0+20120225-3
Anyone else using the antispam plugin with spamassassin 3.3.2 ? Anything I can check to see what's causing this problem ?
$ cat /usr/local/bin/sa-learn-pipe.sh #!/bin/bash
echo /usr/bin/sa-learn $* /tmp/sendmail-msg-$$.txt echo "$$-start ($*)" >> /var/log/sa-learn-pipe.log cat<&0 >> /tmp/sendmail-msg-$$.txt /usr/bin/sa-learn -D $* /tmp/sendmail-msg-$$.txt >> /tmp/sa-learn-pipe.$$.log 2>&1 echo $$ sa-learn rc=$? id=$(id) HOME=$HOME >> /var/log/sa-learn-pipe.log
while read line; do echo $$-sa-learn "$line" >> /var/log/sa-learn-pipe.log done < /tmp/sa-learn-pipe.$$.log
rm -f /tmp/sendmail-msg-$$.txt echo "$$-end" >> /var/log/sa-learn-pipe.log exit 0
With FuzzyOcr DISabled, fails right after Bayes DB access
$ cat /var/log/sa-learn-pipe.log 4505-start (--spam) 4505 sa-learn rc=134 id=uid=108(Debian-exim) gid=113(Debian-exim) groups=113(Debian-exim) HOME= 4505-sa-learn Nov 8 09:05:26.134 [4507] dbg: logger: adding facilities: all 4505-sa-learn Nov 8 09:05:26.134 [4507] dbg: logger: logging level is DBG 4505-sa-learn Nov 8 09:05:26.134 [4507] dbg: generic: SpamAssassin version 3.3.2 4505-sa-learn Nov 8 09:05:26.134 [4507] dbg: generic: Perl 5.014002, PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/spamassassin, LOCAL_STATE_DIR=/var/lib/spamassassin : : 4505-sa-learn Nov 8 09:05:27.131 [4507] dbg: replacetags: done replacing tags 4505-sa-learn Nov 8 09:05:27.131 [4507] dbg: FreeMail: loaded freemail_domains entries: 2470 normal, 29 wildcard 4505-sa-learn Nov 8 09:05:27.132 [4507] dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x3879e88) implements 'learner_new', priority 0 4505-sa-learn Nov 8 09:05:27.132 [4507] dbg: bayes: learner_new self=Mail::SpamAssassin::Plugin::Bayes=HASH(0x3879e88), bayes_store_module=Mail::SpamAssassin::BayesStore::MySQL 4505-sa-learn Nov 8 09:05:27.151 [4507] dbg: bayes: using username: debian-spamd 4505-sa-learn Nov 8 09:05:27.151 [4507] dbg: bayes: learner_new: got store=Mail::SpamAssassin::BayesStore::MySQL=HASH(0x3ec0388) 4505-sa-learn Nov 8 09:05:27.152 [4507] dbg: plugin: Mail::SpamAssassin::Plugin::Bayes=HASH(0x3879e88) implements 'learner_is_scan_available', priority 0 4505-sa-learn libgcc_s.so.1 must be installed for pthread_cancel to work 4505-end 7235-start ()
With FuzzyOcr ENabled, fails right after FuzzyOcr DB access
$ cat /var/log/sa-learn-pipe.log : : 7804-sa-learn Nov 8 09:37:09.612 [7806] info: FuzzyOcr: Using scan ocrad-invert: /usr/bin/ocrad -s5 -i $input 7804-sa-learn Nov 8 09:37:09.612 [7806] info: FuzzyOcr: Using scan ocrad-decolorize-invert: /usr/bin/ocrad -s5 -i $input 7804-sa-learn Nov 8 09:37:09.612 [7806] info: FuzzyOcr: Using scan ocrad-decolorize: /usr/bin/ocrad -s5 $input 7804-sa-learn Nov 8 09:37:09.612 [7806] info: FuzzyOcr: Using scan gocr: /usr/bin/gocr -i $input 7804-sa-learn Nov 8 09:37:09.612 [7806] info: FuzzyOcr: Using scan gocr-180: /usr/bin/gocr -l 180 -d 2 -i $input 7804-sa-learn Nov 8 09:37:09.612 [7806] info: FuzzyOcr: Using scan tesseract: /usr/bin/tesseract $input $output 7804-sa-learn Nov 8 09:37:09.612 [7806] info: FuzzyOcr: Importing for MLDBM databases not available (dependencies missing) 7804-sa-learn Nov 8 09:37:09.612 [7806] dbg: FuzzyOcr: Connecting to: dbi:mysql:database=FuzzyOcr;mysql_socket=/var/run/mysqld/mysqld.sock 7804-sa-learn libgcc_s.so.1 must be installed for pthread_cancel to work 7804-end
syslog entries from running with sa-learn-pipe.sh script antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh
Nov 8 09:36:41 newgalah1 imap: antispam: plugin initialising (2.0-notgit)
Nov 8 09:36:41 newgalah1 imap: antispam: verbose debug enabled
Nov 8 09:36:41 newgalah1 imap: antispam: "Junk" is exact match spam folder
Nov 8 09:36:41 newgalah1 imap: antispam: "Spam" is exact match spam folder
Nov 8 09:36:41 newgalah1 imap: antispam: no unsure folders
Nov 8 09:36:41 newgalah1 imap: antispam: "Trash" is wildcard match
trash folder
Nov 8 09:36:41 newgalah1 imap: antispam: "Deleted *" is wildcard match
trash folder
Nov 8 09:36:41 newgalah1 imap: antispam: pipe backend spam argument =
--spam
Nov 8 09:36:41 newgalah1 imap: antispam: pipe backend not-spam argument
= --ham
Nov 8 09:36:41 newgalah1 imap: antispam: pipe backend program =
/usr/local/bin/sa-learn-pipe.sh
Nov 8 09:36:41 newgalah1 imap: antispam: pipe backend tmpdir /tmpNov 8
09:37:08 newgalah1 imap: antispam: mailbox_is_unsure(Junk): 0
Nov 8 09:37:08 newgalah1 imap: antispam: mailbox_is_trash(INBOX): 0
Nov 8 09:37:08 newgalah1 imap: antispam: mailbox_is_trash(Junk): 0
Nov 8 09:37:08 newgalah1 imap: antispam: mail copy: from trash: 0, to
trash: 0
Nov 8 09:37:08 newgalah1 imap: antispam: mailbox_is_spam(INBOX): 0
Nov 8 09:37:08 newgalah1 imap: antispam: mailbox_is_spam(Junk): 1
Nov 8 09:37:08 newgalah1 imap: antispam: mailbox_is_unsure(INBOX): 0
Nov 8 09:37:08 newgalah1 imap: antispam: mail copy: src spam: 0, dst
spam: 1, src unsure: 0
Nov 8 09:37:08 newgalah1 imap: antispam: running mailtrain backend
program /usr/local/bin/sa-learn-pipe.sh
Nov 8 09:37:08 newgalah1 imap: antispam: running mailtrain backend
program /usr/local/bin/sa-learn-pipe.sh
Nov 8 09:37:08 newgalah1 imap: antispam: running mailtrain backend
program parameter 1 --spam
syslog entries from running with sa-learn directly antispam_pipe_program = /usr/bin/sa-learn
Nov 8 09:44:57 newgalah1 imap: antispam: plugin initialising (2.0-notgit)
Nov 8 09:44:57 newgalah1 imap: antispam: verbose debug
enabled
Nov 8 09:44:57 newgalah1 imap: antispam: "Junk" is exact match spam
folder
Nov 8 09:44:57 newgalah1 imap: antispam: "Spam" is exact match spam
folder
Nov 8 09:44:57 newgalah1 imap: antispam: no unsure
folders
Nov 8 09:44:57 newgalah1 imap: antispam: "Trash" is wildcard match
trash folder
Nov 8 09:44:57 newgalah1 imap: antispam: "Deleted *" is wildcard match
trash folder
Nov 8 09:44:57 newgalah1 imap: antispam: pipe backend spam argument =
--spam
Nov 8 09:44:57 newgalah1 imap: antispam: pipe backend not-spam argument
= --ham
Nov 8 09:44:57 newgalah1 imap: antispam: pipe backend program =
/usr/bin/sa-learn
Nov 8 09:44:57 newgalah1 imap: antispam: pipe backend tmpdir
/tmp
Nov 8 09:44:57 newgalah1 imap: antispam: mailbox_is_unsure(Junk):
0
Nov 8 09:44:57 newgalah1 imap: antispam: mailbox_is_trash(INBOX):
0
Nov 8 09:44:57 newgalah1 imap: antispam: mailbox_is_trash(Junk):
0
Nov 8 09:44:57 newgalah1 imap: antispam: mail copy: from trash: 0, to
trash: 0
Nov 8 09:44:57 newgalah1 imap: antispam: mailbox_is_spam(INBOX):
0
Nov 8 09:44:57 newgalah1 imap: antispam: mailbox_is_spam(Junk):
1
Nov 8 09:44:57 newgalah1 imap: antispam: mailbox_is_unsure(INBOX):
0
Nov 8 09:44:57 newgalah1 imap: antispam: mail copy: src spam: 0, dst
spam: 1, src unsure: 0
Nov 8 09:44:57 newgalah1 imap: antispam: running mailtrain backend
program /usr/bin/sa-learn
Nov 8 09:44:57 newgalah1 imap: antispam: running mailtrain backend
program /usr/bin/sa-learn
Nov 8 09:44:57 newgalah1 imap: antispam: running mailtrain backend
program parameter 1 --spam
Nov 8 09:44:58 newgalah1 imap: antispam: run program failed with exit
code -1
$ cat 90-plugin.conf plugin { antispam_debug_target = syslog antispam_verbose_debug = 1 antispam_backend = pipe antispam_trash_pattern = Trash;Deleted * antispam_spam = Junk;Spam antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh # antispam_pipe_program_args = --progress antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_tmpdir = /tmp }
$ dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.8.0-32-generic x86_64 Ubuntu 13.04 auth_mechanisms = cram-md5 login plain first_valid_uid = 108 mail_location = maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;as cii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = scheme=plain username_format=%u /var/mail/%d/PASSWD driver = passwd-file } plugin { antispam_backend = pipe antispam_debug_target = syslog antispam_pipe_program = /usr/local/bin/sa-learn-pipe.sh antispam_pipe_program_notspam_arg = --ham antispam_pipe_program_spam_arg = --spam antispam_pipe_tmpdir = /tmp antispam_spam = Junk;Spam antispam_trash_pattern = Trash;Deleted * antispam_verbose_debug = 1 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap lmtp sieve" service auth { unix_listener auth-client { mode = 0666 } } ssl = required ssl_cert =