[Dovecot] antispam plugin, amavis and sa-learn
Hello,
I would like to use the Dovecot antispam plugin to train SpamAssassin via sa-learn. I compiled the plugin and got it to work with the sendmail backend, which I tested by having it send a message to me.
As I'm calling SpamAssassin with amavisd-new, I have a system-wide Bayes db, maintained by the amavis user. As expected, the antispam plugin calls the configured binary as the mail user (I'm running deliver as suid, with mixed virtual and system accounts).
My question is: How can I get the plugin to call sa-learn as user amavis?
I have tried the following things: (1) making the amavis user's .spamassassin directory to world-writeable and handing the path to sa-learn with the --dbpath parameter. That failed
- the IMAP client says "Move not allowed" when I try to move a message to or from the Spam IMAP folder.
(2) making a suid copy of sa-learn with owner amavis. This failed because of an sperl error.
(3) editing sudoers to allow 'sudo -u amavis sa-learn' without a password. This works fine on the command line, but fails with the same error as (1) when used with the plugin.
Although I compiled the plugin from git with debugging to syslog activated, I do not get any error messages in mail.log where all messages from dovecot are logged.
Now, I know that the methods above aren't exactly secure, so if anybody has a different way of training SpamAssassin with the antispam plugin in a amavis setup, I would appreciate it.
Thank you, Johannes
On Tue, 2008-07-08 at 10:53 +0200, Johannes Bauer wrote:
Hello,
I would like to use the Dovecot antispam plugin to train SpamAssassin via sa-learn. I compiled the plugin and got it to work with the sendmail backend, which I tested by having it send a message to me.
As I'm calling SpamAssassin with amavisd-new, I have a system-wide Bayes db, maintained by the amavis user. As expected, the antispam plugin calls the configured binary as the mail user (I'm running deliver as suid, with mixed virtual and system accounts).
My question is: How can I get the plugin to call sa-learn as user amavis?
I have tried the following things: (1) making the amavis user's .spamassassin directory to world-writeable and handing the path to sa-learn with the --dbpath parameter. That failed
- the IMAP client says "Move not allowed" when I try to move a message to or from the Spam IMAP folder.
(2) making a suid copy of sa-learn with owner amavis. This failed because of an sperl error.
(3) editing sudoers to allow 'sudo -u amavis sa-learn' without a password. This works fine on the command line, but fails with the same error as (1) when used with the plugin.
I have no idea. Try maybe writing a small C wrapper that checks the arguments. Also check what user your imap is running under, normally it would be the user who logged in unless those are virtual.
johannes
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tue, 8 Jul 2008, Johannes Bauer wrote:
(1) making the amavis user's .spamassassin directory to world-writeable and handing the path to sa-learn with the --dbpath parameter. That failed
- the IMAP client says "Move not allowed" when I try to move a message to or from the Spam IMAP folder.
(3) editing sudoers to allow 'sudo -u amavis sa-learn' without a password. This works fine on the command line, but fails with the same error as (1) when used with the plugin.
Hmm, "Move not allowed" from the client?
What do you see from Dovecot? I mean when you speak IMAP directly or trace/sniff a non-SSL connection. I have the impression, that you hit a situation, like:
"Cannot APPEND to a SPAM folder."
However, in this case it should not work with the sendmail binary.
For (3): did you allow all users to call sudo without password? I mean: su - user sudo -u amavis sa-learn Also, did you noticed this paragraphe in man sudo? -H The -H (HOME) option sets the HOME environment variable to the homedir of the target user (root by default) as specified in passwd(5). By default, sudo does not modify HOME (see set_home and always_set_home in sudoers(5)).
If HOME is not amavis's HOME, sa-learn might be upset.
Wrap your program in order to trace the problems of sa-learn:
#!/bin/bash
( id -a [snip] sa-learn [snip] rc=$? echo "exit code $rc" exit $rc ) > /tmp/antispam.out.$$ 2> /tmp/antispam.err.$$
sa-learn needs to lock the database, maybe you get race problems? I used to call sa-learn via --no-sync and --sync'ed in regular intervals. Also, maybe you need a combination of -u/-C/-p.
Although I compiled the plugin from git with debugging to syslog activated, I do not get any error messages in mail.log where all messages from dovecot are logged.
Well, my antispam logs go to syslog, but Dovecot logs to a file.
Now, I know that the methods above aren't exactly secure, so if anybody
Dunno, but you want to train a site-wide database with information from the user. So what you consider unsecure in particular? If you are afraid of bugs in sa-learn, you should limit this ability to a certain group of users, because any user can push any "message" Dovecot accepts to sa-learn, regardless of its internal structure.
I have moved the Bayes DB to SQL to avoid the locking problems I had.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFIc0XIVJMDrex4hCIRAnN1AJwMeiSWiRl/qBbQwDNYIw6T+Zg6iwCeKNyA jcyx0LXE7EQ2oot6wrBp+lA= =NhLz -----END PGP SIGNATURE-----
Thank you for you quick answer and the suggestions, Steffen.
(3) editing sudoers to allow 'sudo -u amavis sa-learn' without a password. This works fine on the command line, but fails with the same error as (1) when used with the plugin.
Hmm, "Move not allowed" from the client?
What do you see from Dovecot? I mean when you speak IMAP directly or trace/sniff a non-SSL connection. I have the impression, that you hit a situation, like:
"Cannot APPEND to a SPAM folder."
The relevant lines in a sniffed IMAP connection:
cpy1 COPY 4 "Spam" cpy1 NO failed to send mail
At least that's what I gathered from a binary tcpdump, I haven't looked at it with a protocol analyzer.
However, in this case it should not work with the sendmail binary.
For (3): did you allow all users to call sudo without password? I mean: su - user sudo -u amavis sa-learn Also, did you noticed this paragraphe in man sudo? -H The -H (HOME) option sets the HOME environment variable to the homedir of the target user (root by default) as specified in passwd(5). By default, sudo does not modify HOME (see set_home and always_set_home in sudoers(5)).
If HOME is not amavis's HOME, sa-learn might be upset.
from sudoers:
ALL ALL= (amavis) NOPASSWD: /usr/bin/sa-learn
It works fine if I su to one of the local users and execute
sudo -H -u amavis /usr/bin/sa-learn -D [args]
I can access amavis' bayes db read and write.
Wrap your program in order to trace the problems of sa-learn:
#!/bin/bash
( id -a [snip] sa-learn [snip] rc=$? echo "exit code $rc" exit $rc ) > /tmp/antispam.out.$$ 2> /tmp/antispam.err.$$
sa-learn needs to lock the database, maybe you get race problems? I used to call sa-learn via --no-sync and --sync'ed in regular intervals. Also, maybe you need a combination of -u/-C/-p.
I inserted the above sudo line into the wrapper script; it worked fine when called from the command line and when called from the antispam plugin.
However, when I skipped the wrapper and called sudo sa-learn directly, it again failed with the above error.
But as it works with the wrapper script, that's ok for me. I trimmed down the logging and had sa-learn process the command line parameters given by the antispam. For reference, the command is now:
/usr/bin/sudo -H -u amavis /usr/bin/sa-learn $1 $2
Although I compiled the plugin from git with debugging to syslog activated, I do not get any error messages in mail.log where all messages from dovecot are logged.
Well, my antispam logs go to syslog, but Dovecot logs to a file.
Hm, yes, of course it is. Don't know why I was looking in mail.log when it says it logs to syslog. But the output did not contain any useful information in this case, anyway.
Now, I know that the methods above aren't exactly secure, so if anybody
Dunno, but you want to train a site-wide database with information from the user. So what you consider unsecure in particular? If you are afraid of bugs in sa-learn, you should limit this ability to a certain group of users, because any user can push any "message" Dovecot accepts to sa-learn, regardless of its internal structure.
In general, I try to avoid suid and word writeable system directories as much as possible. I can live with the sudo solution - after all, the worst a user could do is deleting the bayes db.
Again, thank you for your suggestions, it really helped getting this to work the way I wanted.
Bye, Johannes
Hi,
I would like to use the antispam plugin with dovecot on Solaris. I'm
mainly using blastwave packages and tried to compile the plugin
without success so far.
I use the following .config file;
BACKEND=mailtrain DOVECOT=/opt/csw/include/dovecot INSTALLDIR=/opt/csw/lib/dovecot/modules/imap USER=root GROUP=root DEBUG=syslog PLUGINNAME=antispam
and I modified this line in Makefile;
CFLAGS += -fPIC -shared -DPLUGINNAME=$(PLUGINNAME)
I use the Sun Studio Compiler (cc: Sun C 5.9 SunOS_i386 Patch
124868-04 2008/03/11) and gmake (also from blastwave). When I try to
compile I got these errors;
[munnik@stevie]{~/software/dovecot-antispam}>/opt/csw/bin/gmake
cc -DHAVE_CONFIG_H -I/opt/csw/include/dovecot/ -I/opt/csw/include/
dovecot/src/ -I/opt/csw/include/dovecot/src/lib/ -I/opt/csw/include/
dovecot/src/lib-storage/ -I/opt/csw/include/dovecot/src/lib-mail/ -I/
opt/csw/include/dovecot/src/lib-imap/ -I/opt/csw/include/dovecot/src/
lib-dict/ -I/opt/csw/include/dovecot/src/lib-index/ -I/opt/csw/include/
dovecot/src/imap/ -o dovecot-version dovecot-version.c
"/opt/csw/include/dovecot/config.h", line 525: warning: macro
redefined: _FILE_OFFSET_BITS
./dovecot-version > dovecot-version.h
cc -c -DCONFIG_DEBUG -DDEBUG_SYSLOG -fPIC -shared -
DPLUGINNAME=antispam -DHAVE_CONFIG_H -I/opt/csw/include/dovecot/ -I/
opt/csw/include/dovecot/src/ -I/opt/csw/include/dovecot/src/lib/ -I/
opt/csw/include/dovecot/src/lib-storage/ -I/opt/csw/include/dovecot/
src/lib-mail/ -I/opt/csw/include/dovecot/src/lib-imap/ -I/opt/csw/
include/dovecot/src/lib-dict/ -I/opt/csw/include/dovecot/src/lib-
index/ -I/opt/csw/include/dovecot/src/imap/ -o debug.o debug.c
"antispam-plugin.h", line 4: cannot find include file: "lib.h"
"antispam-plugin.h", line 5: cannot find include file: "str.h"
"antispam-plugin.h", line 6: cannot find include file: "client.h"
"antispam-plugin.h", line 7: cannot find include file: "ostream.h"
"antispam-plugin.h", line 8: cannot find include file: "imap-search.h"
"antispam-plugin.h", line 21: warning: no explicit type given
"antispam-plugin.h", line 21: syntax error before or at:
antispam_plugin_id
"antispam-plugin.h", line 21: warning: old-style declaration or
incorrect type for: antispam_plugin_id
"antispam-plugin.h", line 30: syntax error before or at: pool
"antispam-plugin.h", line 30: warning: undefined or missing type for:
pool
"antispam-plugin.h", line 39: warning: dubious tag declaration: struct
mailbox_transaction_context
"antispam-plugin.h", line 41: warning: dubious tag declaration: struct
mail
"antispam-plugin.h", line 42: warning: dubious tag declaration: struct
mailbox
"antispam-plugin.h", line 44: warning: dubious tag declaration: struct
mailbox_transaction_context
"antispam-plugin.h", line 66: warning: dubious tag declaration: struct
mail_storage
"antispam-plugin.h", line 67: warning: dubious tag declaration: struct
mail_storage
"antispam-plugin.h", line 68: warning: old-style declaration or
incorrect type for: bool
"antispam-plugin.h", line 68: syntax error before or at: mailbox_is_spam
"antispam-plugin.h", line 68: warning: dubious tag declaration: struct
mailbox
"antispam-plugin.h", line 68: warning: old-style declaration or
incorrect type for: mailbox_is_spam
"antispam-plugin.h", line 69: warning: old-style declaration or
incorrect type for: bool
"antispam-plugin.h", line 69: syntax error before or at:
mailbox_is_trash
"antispam-plugin.h", line 69: warning: dubious tag declaration: struct
mailbox
"antispam-plugin.h", line 69: warning: old-style declaration or
incorrect type for: mailbox_is_trash
"antispam-plugin.h", line 70: warning: old-style declaration or
incorrect type for: bool
"antispam-plugin.h", line 70: syntax error before or at:
mailbox_is_unsure
"antispam-plugin.h", line 70: warning: dubious tag declaration: struct
mailbox
"antispam-plugin.h", line 70: warning: old-style declaration or
incorrect type for: mailbox_is_unsure
"antispam-plugin.h", line 72: warning: old-style declaration or
incorrect type for: bool
"antispam-plugin.h", line 72: syntax error before or at:
antispam_can_append_to_spam
"antispam-plugin.h", line 72: warning: old-style declaration or
incorrect type for: antispam_can_append_to_spam
"antispam-plugin.h", line 73: warning: old-style declaration or
incorrect type for: bool
"antispam-plugin.h", line 73: syntax error before or at: keyword_is_spam
"antispam-plugin.h", line 73: warning: old-style declaration or
incorrect type for: keyword_is_spam
"antispam-plugin.h", line 75: warning: no explicit type given
"antispam-plugin.h", line 75: syntax error before or at:
need_keyword_hook
"antispam-plugin.h", line 75: warning: old-style declaration or
incorrect type for: need_keyword_hook
"antispam-plugin.h", line 76: warning: no explicit type given
"antispam-plugin.h", line 76: syntax error before or at:
need_folder_hook
"antispam-plugin.h", line 76: warning: old-style declaration or
incorrect type for: need_folder_hook
"antispam-plugin.h", line 116: warning: dubious tag declaration:
struct mail
"antispam-plugin.h", line 118: warning: implicit function declaration:
mail_get_stream
"antispam-plugin.h", line 118: warning: improper pointer/integer
combination: op "="
"antispam-plugin.h", line 122: warning: dubious tag declaration:
struct mail
"antispam-plugin.h", line 124: warning: implicit function declaration:
mail_get_headers
"antispam-plugin.h", line 124: warning: improper pointer/integer
combination: op "="
"antispam-plugin.h", line 128: syntax error before or at: pool_t
"antispam-plugin.h", line 128: warning: undefined or missing type for:
pool_t
"antispam-plugin.h", line 130: warning: implicit function declaration:
o_stream_create_file
"antispam-plugin.h", line 130: undefined symbol: pool
"antispam-plugin.h", line 130: undefined symbol: TRUE
"antispam-plugin.h", line 130: warning: improper pointer/integer
combination: op "="
"debug.c", line 11: warning: implicit function declaration: t_push
"debug.c", line 13: warning: implicit function declaration: t_strconcat
"debug.c", line 13: warning: improper pointer/integer combination: op
"="
"debug.c", line 23: warning: implicit function declaration: t_pop
cc: acomp failed for debug.c
gmake: *** [debug.o] Error 2
Martijn
DOVECOT=/opt/csw/include/dovecot
"antispam-plugin.h", line 4: cannot find include file: "lib.h" "antispam-plugin.h", line 5: cannot find include file: "str.h" "antispam-plugin.h", line 6: cannot find include file: "client.h" "antispam-plugin.h", line 7: cannot find include file: "ostream.h" "antispam-plugin.h", line 8: cannot find include file: "imap-search.h"
clearly, it's not finding the header files. Try pointing DOVECOT to the actual sources.
johannes
On Jul 11, 2008, at 9:10 AM, Johannes Berg wrote:
DOVECOT=/opt/csw/include/dovecot
"antispam-plugin.h", line 4: cannot find include file: "lib.h" "antispam-plugin.h", line 5: cannot find include file: "str.h" "antispam-plugin.h", line 6: cannot find include file: "client.h" "antispam-plugin.h", line 7: cannot find include file: "ostream.h" "antispam-plugin.h", line 8: cannot find include file: "imap- search.h"
clearly, it's not finding the header files. Try pointing DOVECOT to
the actual sources.johannes
I fixed the problem with the header files and antispam compiles
succesful. But when I try to start dovecot with the antispam plugin I
get this error;
ILoading modules from directory: /opt/csw/lib/dovecot/imap
Edlopen(/opt/csw/lib/dovecot/imap/lib90_antispam_plugin.so) failed:
ld.so.1: imap: fatal: relocation error: file /opt/csw/lib/dovecot/imap/
lib90_antispam_plugin.so: symbol mkdtemp: referenced symbol not found
FCouldn't load required plugins
Error: imap dump-capability process returned 89
Fatal: Invalid configuration in /opt/csw/etc/dovecot.conf
The .config file I've used
BACKEND=mailtrain DOVECOT=../dovecot-1.0.13 INSTALLDIR=/opt/csw/lib/dovecot/imap USER=root GROUP=root PLUGINNAME=antispam
and I've changed this line in the Makefile
CFLAGS += -fPIC -shared -DPLUGINNAME=$(PLUGINNAME)
I'm using gmake and cc (Sun Studio)
Martijn
On Wed, 2008-07-16 at 17:45 +0200, Martijn de Munnik wrote:
I fixed the problem with the header files and antispam compiles
succesful. But when I try to start dovecot with the antispam plugin I
get this error;ILoading modules from directory: /opt/csw/lib/dovecot/imap Edlopen(/opt/csw/lib/dovecot/imap/lib90_antispam_plugin.so) failed:
ld.so.1: imap: fatal: relocation error: file /opt/csw/lib/dovecot/imap/ lib90_antispam_plugin.so: symbol mkdtemp: referenced symbol not found
Well, looks like you don't have mkdtemp in your libc. What alternative does Solaris offer?
johannes
On Jul 16, 2008, at 5:58 PM, Johannes Berg wrote:
On Wed, 2008-07-16 at 17:45 +0200, Martijn de Munnik wrote:
I fixed the problem with the header files and antispam compiles succesful. But when I try to start dovecot with the antispam plugin I get this error;
ILoading modules from directory: /opt/csw/lib/dovecot/imap Edlopen(/opt/csw/lib/dovecot/imap/lib90_antispam_plugin.so) failed: ld.so.1: imap: fatal: relocation error: file /opt/csw/lib/dovecot/ imap/ lib90_antispam_plugin.so: symbol mkdtemp: referenced symbol not found
Well, looks like you don't have mkdtemp in your libc. What alternative does Solaris offer?
johannes
I've found a mkdtemp.c in http://www.r-project.org/ which seems to do
the same as the one in libc. Is it possible to use that?
Martijn
[please trim your quotes]
On Wed, 2008-07-16 at 18:50 +0200, Martijn de Munnik wrote:
I've found a mkdtemp.c in http://www.r-project.org/ which seems to do
the same as the one in libc. Is it possible to use that?
probably. Just add a lines
objs += mdtemp.c
in the Makefile somewhere around "# main make rules"
johannes
On Jul 16, 2008, at 6:54 PM, Johannes Berg wrote:
[please trim your quotes]
On Wed, 2008-07-16 at 18:50 +0200, Martijn de Munnik wrote:
I've found a mkdtemp.c in http://www.r-project.org/ which seems to do the same as the one in libc. Is it possible to use that?
probably. Just add a lines
objs += mdtemp.c
in the Makefile somewhere around "# main make rules"
johannes It works!
I've used mkdtemp.c from r-project.org and added/modified these lines
in Makefile
INCS += -DHAVE_CONFIG_H -DHAVE_GETTIMEOFDAY -DHAVE_SYS_TIME_H objs += mkdtemp.c
thank you for your help,
Martijn
participants (5)
-
Johannes Bauer
-
Johannes Berg
-
Martijn de Munnik
-
Martijn de Munnik
-
Steffen Kaiser