antispam plugin pipe script seems not to be called when it "should be"
ben at indietorrent.org
ben at indietorrent.org
Fri Aug 19 16:17:24 UTC 2016
On 2016-08-18 22:58, Edgar Pettijohn wrote:
> On 16-08-18 22:43:32, Ben Johnson wrote:
>> On 8/18/2016 10:26 PM, Ben Johnson wrote:
>> > Hello,
>> >
>> > (Full "doveconf -n" output is at the end of this message.)
>> >
>> > I'm attempting to recreate a "recipe" I had used successfully for quite
>> > some time (in Ubuntu 12.04 LTS), which allowed me to train
>> > SpamAssassin's Bayes database when someone dragged an email message from
>> > one sub-folder of the account to another.
>> >
>> > I later tried to re-implement it (in Ubuntu 14.04 LTS), and core-dumps
>> > resulted. I posted to this list about it several times, but nobody was
>> > ever able to help me solve the problem.
>> >
>> > I just tried the same (in Ubuntu 16.04 LTS), hoping that the source had
>> > evolved since then, and while I don't see a core-dump, the pipe script
>> > doesn't seem to be called.
>> >
>> > I've booby-trapped the pipe script with quite a bit of debugging output,
>> > which should be written to a file, but I don't see any indication that
>> > the pipe script is ever executed.
>> >
>> > When logged-in as an IMAP user, and I drag a message from the user's
>> > Inbox into the "Spam" folder, I do see the following activity in the syslog:
>> >
>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_unsure(Junk): 0
>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_trash(INBOX): 0
>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_trash(Junk): 0
>> > Aug 18 22:12:15 example.com imap: antispam: mail copy: from trash: 0, to
>> > trash: 0
>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_spam(INBOX): 0
>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_spam(Junk): 1
>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_unsure(INBOX): 0
>> > Aug 18 22:12:15 example.com imap: antispam: mail copy: src spam: 0, dst
>> > spam: 1, src unsure: 0
>> > Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend
>> > program /bin/bash
>> > Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend
>> > program /bin/bash
>> > Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend
>> > program parameter 1 /usr/local/bin/sa-learn-pipe.sh
>> > Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend
>> > program parameter 2 --spam
>> >
>> > But the piped message is never delivered to the target mailbox.
>> > The mail log at /var/log/mail.log does not register any activity when I
>> > move messages (and maybe that's expected).
>> >
>> > Here are the contents of the /usr/local/bin/sa-learn-pipe.sh script:
>> >
>> > http://pastebin.com/7vm74jmk
>> >
>> > The permissions and mode on the script seem appropriate:
>> >
>> > $ stat /usr/local/bin/sa-learn-pipe.sh
>> > File: '/usr/local/bin/sa-learn-pipe.sh'
>> > Size: 1765 Blocks: 8 IO Block: 4096 regular file
>> > Device: fd01h/64769d Inode: 669345 Links: 1
>> > Access: (0755/-rwxr-xr-x) Uid: ( 5000/ vmail) Gid: ( 5000/ vmail)
>> > Access: 2016-08-18 19:09:47.307392919 -0400
>> > Modify: 2016-08-18 19:09:31.231391749 -0400
>> > Change: 2016-08-18 19:09:31.231391749 -0400
>> > Birth: -
>> >
>> > What might I be doing incorrectly here?
>> >
>> > Please let me know if I can provide any other information.
>> >
>> > Thank you in advance,
>> >
>> > -Ben
>> >
>> >
>> >
>> > Full "doveconf -n" output:
>> >
>> > $ doveconf -n
>> > # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
>> > # Pigeonhole version 0.4.13 (7b14904)
>> > # OS: Linux 4.4.0-34-generic x86_64 Ubuntu 16.04.1 LTS
>> > auth_mechanisms = plain login
>> > disable_plaintext_auth = no
>> > listen = *,[::]
>> > log_timestamp = "%Y-%m-%d %H:%M:%S "
>> > mail_max_userip_connections = 100
>> > mail_plugins = " quota"
>> > mail_privileged_group = vmail
>> > passdb {
>> > args = /etc/dovecot/dovecot-sql.conf
>> > driver = sql
>> > }
>> > plugin {
>> > antispam_backend = pipe
>> > antispam_debug_target = syslog
>> > antispam_pipe_program = /bin/bash
>> > antispam_pipe_program_args = /usr/local/bin/sa-learn-pipe.sh
>> > antispam_pipe_program_notspam_arg = --ham
>> > antispam_pipe_program_spam_arg = --spam
>> > antispam_pipe_tmpdir = /tmp
>> > antispam_spam_pattern_ignorecase = SPAM;JUNK
>> > antispam_trash_pattern_ignorecase = trash;Deleted *
>> > antispam_verbose_debug = 1
>> > quota = dict:user::file:/var/vmail/%d/%n/.quotausage
>> > quota_rule2 = Trash:storage=+100M
>> > quota_rule3 = Junk:ignore
>> > quota_rule4 = INBOX:storage=+100M
>> > quota_warning = storage=100%% quota-reached 100 %u %d
>> > quota_warning2 = storage=95%% quota-warning 95 %u %d
>> > quota_warning3 = storage=80%% quota-warning 80 %u %d
>> > quota_warning4 = -storage=100%% quota-below below %u %d
>> > sieve = /var/vmail/%d/%n/.sieve
>> > sieve_max_redirects = 25
>> > }
>> > postmaster_address = postmaster at example.com
>> > protocols = imap pop3
>> > service auth {
>> > unix_listener /var/spool/postfix/private/auth {
>> > group = postfix
>> > mode = 0660
>> > user = postfix
>> > }
>> > unix_listener auth-userdb {
>> > group = vmail
>> > mode = 0600
>> > user = vmail
>> > }
>> > user = root
>> > }
>> > service imap-login {
>> > client_limit = 1000
>> > process_limit = 512
>> > }
>> > service lmtp {
>> > unix_listener /var/spool/postfix/private/dovecot-lmtp {
>> > group = postfix
>> > mode = 0600
>> > user = postfix
>> > }
>> > }
>> > service quota-below {
>> > executable = script /usr/local/bin/quota-below.sh
>> > unix_listener quota-below {
>> > group = vmail
>> > mode = 0666
>> > user = vmail
>> > }
>> > user = vmail
>> > }
>> > service quota-reached {
>> > executable = script /usr/local/bin/quota-reached.sh
>> > unix_listener quota-reached {
>> > group = vmail
>> > mode = 0666
>> > user = vmail
>> > }
>> > user = vmail
>> > }
>> > service quota-warning {
>> > executable = script /usr/local/bin/quota-warning.sh
>> > unix_listener quota-warning {
>> > group = vmail
>> > mode = 0666
>> > user = vmail
>> > }
>> > user = vmail
>> > }
>> > ssl_cert = </etc/postfix/smtpd.cert
>> > ssl_key = </etc/postfix/smtpd.key
>> > ssl_protocols = !SSLv2 !SSLv3
>> > userdb {
>> > driver = prefetch
>> > }
>> > userdb {
>> > args = /etc/dovecot/dovecot-sql.conf
>> > driver = sql
>> > }
>> > protocol imap {
>> > mail_plugins = quota imap_quota antispam
>> > }
>> > protocol pop3 {
>> > mail_plugins = quota
>> > pop3_uidl_format = %08Xu%08Xv
>> > }
>> > protocol lda {
>> > mail_plugins = sieve quota
>> > }
>> > protocol lmtp {
>> > mail_plugins = quota sieve
>> > postmaster_address = webmaster at localhost
>> > }
>> >
>>
>> If it's helpful, I've tried replacing the pipe script's contents with
>> the following:
>>
>> #####################################################################
>> #!/bin/bash
>>
>> /usr/lib/dovecot/deliver -d "sa-training at example.com" -m
>> "Training.$mode" -p /var/vmail/gtube.txt
>>
>> # Exit with "success" status code.
>> exit 0
>> #####################################################################
>>
>> And still, the message never arrives when I drag a message from
>> "Inbox"
>> to "Spam" in the IMAP client, despite the messages logged to syslog,
>> which "all seem to be correct".
>>
>> Yet, if I do this using the above content for the script, the "gtube"
>> message is delivered successfully:
>>
>> # su vmail
>> $ /usr/local/bin/sa-learn-pipe-test.sh
>>
>> The message arrives in the Inbox, presumably because -m
>> "Training.$mode"
>> does not "resolve" to anything, and per the documentation, the
>> delivery
>> destination will default to the Inbox in such cases.
>>
>> What's wrong with my the custom pipe script to which I posted a link
>> in
>> my previous message, I wonder?
>>
>> Thanks again for any assistance,
>>
>> -Ben
>
> I don't know how to solve your problem, but I have an alternate method.
>
> Create a Junk folder and a ham folder then add the following cron jobs:
>
> 30 5 * * * /usr/local/bin/sa-learn --spam
> /var/vmail/Maildir/.Junk
> 30 6 * * 6 /usr/local/bin/sa-learn
> --backup > /etc/mail/spamassassin/spam.db
> 30 7 * * * /usr/local/bin/sa-learn --ham
> /var/vmail/Maildir/.Ham
Thank you for taking a look, Edgar!
I already have the very setup that you describe.
The challenge at hand, however, is with regard to *how* messages end-up
in the "ham" and "spam" folders in the first place.
This is a multi-user system and each user must be able to contribute to
ham/spam training. To make this as easy as possible, and transparent to
the end-users, I need for the simple act of dragging a message from
Inbox -> Spam (or vice versa) to submit the sample on the user's behalf,
automatically. This has all worked swimmingly in the past.
In any case, I've made some progress on this go-around.
I tested my backend pipe script by doing this, as root, and it works:
# /bin/bash /usr/local/bin/sa-learn-pipe.sh --spam <
/var/vmail/gtube.txt
The message is delivered to the appropriate mailbox and all is well in
the world.
Of course, in practice, Dovecot does not run as root. So, let's try the
same as the vmail user:
# su vmail
$ /bin/bash /usr/local/bin/sa-learn-pipe.sh --spam <
/var/vmail/gtube.txt
/usr/local/bin/sa-learn-pipe.sh: line 8: /tmp/sa-learn-pipe.log:
Permission denied
/usr/local/bin/sa-learn-pipe.sh: line 10: /tmp/sa-learn-pipe.log:
Permission denied
/usr/local/bin/sa-learn-pipe.sh: line 12: /tmp/sa-learn-pipe.log:
Permission denied
/usr/local/bin/sa-learn-pipe.sh: line 16: /tmp/sendmail-parms.txt:
Permission denied
/usr/local/bin/sa-learn-pipe.sh: line 22: /tmp/sa-learn-pipe.log:
Permission denied
/usr/local/bin/sa-learn-pipe.sh: line 37: /tmp/sa-learn-pipe.log:
Permission denied
/usr/local/bin/sa-learn-pipe.sh: line 40: /tmp/sa-learn-pipe.log:
Permission denied
/usr/local/bin/sa-learn-pipe.sh: line 43: /tmp/strace.txt: Permission
denied
/usr/local/bin/sa-learn-pipe.sh: line 47: /tmp/sa-learn-pipe.log:
Permission denied
/usr/local/bin/sa-learn-pipe.sh: line 54: /tmp/sa-learn-pipe.log:
Permission denied
Aha! Clearly, the vmail user cannot read from nor write to /tmp. (Why
that is, I have no idea, as the /tmp directory's permissions certainly
allow for both; maybe Dovecot implements this as a security measure.)
This prompted me to change all references to /tmp in the pipe script to
~/tmp, and create this directory:
$ whoami
vmail
$ mkdir ~/tmp && chmod 770 ~/tmp
$ /bin/bash /usr/local/bin/sa-learn-pipe.sh --ham < /var/vmail/gtube.txt
No errors this time (at least not on the console).
But I do get this in /var/log/mail.err:
Aug 19 12:04:24 example.com dovecot: lda(sa-training at example.com):
Fatal: Can't open delivery mail as raw: Permission denied
I'm not sure how to interpret this message. Where is permission being
denied? More importantly, what's the fix?
Thanks for any hints!
-Ben
More information about the dovecot
mailing list