antispam plugin pipe script seems not to be called when it "should be"
Edgar Pettijohn
edgar at pettijohn-web.com
Fri Aug 19 16:27:56 UTC 2016
Sent from my iPhone
> On Aug 19, 2016, at 11:17 AM, ben at indietorrent.org wrote:
>
>> On 2016-08-18 22:58, Edgar Pettijohn wrote:
>>> On 16-08-18 22:43:32, Ben Johnson wrote:
>>> On 8/18/2016 10:26 PM, Ben Johnson wrote:
>>> > Hello,
>>> >
>>> > (Full "doveconf -n" output is at the end of this message.)
>>> >
>>> > I'm attempting to recreate a "recipe" I had used successfully for quite
>>> > some time (in Ubuntu 12.04 LTS), which allowed me to train
>>> > SpamAssassin's Bayes database when someone dragged an email message from
>>> > one sub-folder of the account to another.
>>> >
>>> > I later tried to re-implement it (in Ubuntu 14.04 LTS), and core-dumps
>>> > resulted. I posted to this list about it several times, but nobody was
>>> > ever able to help me solve the problem.
>>> >
>>> > I just tried the same (in Ubuntu 16.04 LTS), hoping that the source had
>>> > evolved since then, and while I don't see a core-dump, the pipe script
>>> > doesn't seem to be called.
>>> >
>>> > I've booby-trapped the pipe script with quite a bit of debugging output,
>>> > which should be written to a file, but I don't see any indication that
>>> > the pipe script is ever executed.
>>> >
>>> > When logged-in as an IMAP user, and I drag a message from the user's
>>> > Inbox into the "Spam" folder, I do see the following activity in the syslog:
>>> >
>>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_unsure(Junk): 0
>>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_trash(INBOX): 0
>>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_trash(Junk): 0
>>> > Aug 18 22:12:15 example.com imap: antispam: mail copy: from trash: 0, to
>>> > trash: 0
>>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_spam(INBOX): 0
>>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_spam(Junk): 1
>>> > Aug 18 22:12:15 example.com imap: antispam: mailbox_is_unsure(INBOX): 0
>>> > Aug 18 22:12:15 example.com imap: antispam: mail copy: src spam: 0, dst
>>> > spam: 1, src unsure: 0
>>> > Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend
>>> > program /bin/bash
>>> > Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend
>>> > program /bin/bash
>>> > Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend
>>> > program parameter 1 /usr/local/bin/sa-learn-pipe.sh
>>> > Aug 18 22:12:15 example.com imap: antispam: running mailtrain backend
>>> > program parameter 2 --spam
>>> >
>>> > But the piped message is never delivered to the target mailbox.
>>> > The mail log at /var/log/mail.log does not register any activity when I
>>> > move messages (and maybe that's expected).
>>> >
>>> > Here are the contents of the /usr/local/bin/sa-learn-pipe.sh script:
>>> >
>>> > http://pastebin.com/7vm74jmk
>>> >
>>> > The permissions and mode on the script seem appropriate:
>>> >
>>> > $ stat /usr/local/bin/sa-learn-pipe.sh
>>> > File: '/usr/local/bin/sa-learn-pipe.sh'
>>> > Size: 1765 Blocks: 8 IO Block: 4096 regular file
>>> > Device: fd01h/64769d Inode: 669345 Links: 1
>>> > Access: (0755/-rwxr-xr-x) Uid: ( 5000/ vmail) Gid: ( 5000/ vmail)
>>> > Access: 2016-08-18 19:09:47.307392919 -0400
>>> > Modify: 2016-08-18 19:09:31.231391749 -0400
>>> > Change: 2016-08-18 19:09:31.231391749 -0400
>>> > Birth: -
>>> >
>>> > What might I be doing incorrectly here?
>>> >
>>> > Please let me know if I can provide any other information.
>>> >
>>> > Thank you in advance,
>>> >
>>> > -Ben
>>> >
>>> >
>>> >
>>> > Full "doveconf -n" output:
>>> >
>>> > $ doveconf -n
>>> > # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
>>> > # Pigeonhole version 0.4.13 (7b14904)
>>> > # OS: Linux 4.4.0-34-generic x86_64 Ubuntu 16.04.1 LTS
>>> > auth_mechanisms = plain login
>>> > disable_plaintext_auth = no
>>> > listen = *,[::]
>>> > log_timestamp = "%Y-%m-%d %H:%M:%S "
>>> > mail_max_userip_connections = 100
>>> > mail_plugins = " quota"
>>> > mail_privileged_group = vmail
>>> > passdb {
>>> > args = /etc/dovecot/dovecot-sql.conf
>>> > driver = sql
>>> > }
>>> > plugin {
>>> > antispam_backend = pipe
>>> > antispam_debug_target = syslog
>>> > antispam_pipe_program = /bin/bash
>>> > antispam_pipe_program_args = /usr/local/bin/sa-learn-pipe.sh
>>> > antispam_pipe_program_notspam_arg = --ham
>>> > antispam_pipe_program_spam_arg = --spam
>>> > antispam_pipe_tmpdir = /tmp
>>> > antispam_spam_pattern_ignorecase = SPAM;JUNK
>>> > antispam_trash_pattern_ignorecase = trash;Deleted *
>>> > antispam_verbose_debug = 1
>>> > quota = dict:user::file:/var/vmail/%d/%n/.quotausage
>>> > quota_rule2 = Trash:storage=+100M
>>> > quota_rule3 = Junk:ignore
>>> > quota_rule4 = INBOX:storage=+100M
>>> > quota_warning = storage=100%% quota-reached 100 %u %d
>>> > quota_warning2 = storage=95%% quota-warning 95 %u %d
>>> > quota_warning3 = storage=80%% quota-warning 80 %u %d
>>> > quota_warning4 = -storage=100%% quota-below below %u %d
>>> > sieve = /var/vmail/%d/%n/.sieve
>>> > sieve_max_redirects = 25
>>> > }
>>> > postmaster_address = postmaster at example.com
>>> > protocols = imap pop3
>>> > service auth {
>>> > unix_listener /var/spool/postfix/private/auth {
>>> > group = postfix
>>> > mode = 0660
>>> > user = postfix
>>> > }
>>> > unix_listener auth-userdb {
>>> > group = vmail
>>> > mode = 0600
>>> > user = vmail
>>> > }
>>> > user = root
>>> > }
>>> > service imap-login {
>>> > client_limit = 1000
>>> > process_limit = 512
>>> > }
>>> > service lmtp {
>>> > unix_listener /var/spool/postfix/private/dovecot-lmtp {
>>> > group = postfix
>>> > mode = 0600
>>> > user = postfix
>>> > }
>>> > }
>>> > service quota-below {
>>> > executable = script /usr/local/bin/quota-below.sh
>>> > unix_listener quota-below {
>>> > group = vmail
>>> > mode = 0666
>>> > user = vmail
>>> > }
>>> > user = vmail
>>> > }
>>> > service quota-reached {
>>> > executable = script /usr/local/bin/quota-reached.sh
>>> > unix_listener quota-reached {
>>> > group = vmail
>>> > mode = 0666
>>> > user = vmail
>>> > }
>>> > user = vmail
>>> > }
>>> > service quota-warning {
>>> > executable = script /usr/local/bin/quota-warning.sh
>>> > unix_listener quota-warning {
>>> > group = vmail
>>> > mode = 0666
>>> > user = vmail
>>> > }
>>> > user = vmail
>>> > }
>>> > ssl_cert = </etc/postfix/smtpd.cert
>>> > ssl_key = </etc/postfix/smtpd.key
>>> > ssl_protocols = !SSLv2 !SSLv3
>>> > userdb {
>>> > driver = prefetch
>>> > }
>>> > userdb {
>>> > args = /etc/dovecot/dovecot-sql.conf
>>> > driver = sql
>>> > }
>>> > protocol imap {
>>> > mail_plugins = quota imap_quota antispam
>>> > }
>>> > protocol pop3 {
>>> > mail_plugins = quota
>>> > pop3_uidl_format = %08Xu%08Xv
>>> > }
>>> > protocol lda {
>>> > mail_plugins = sieve quota
>>> > }
>>> > protocol lmtp {
>>> > mail_plugins = quota sieve
>>> > postmaster_address = webmaster at localhost
>>> > }
>>> >
>>> If it's helpful, I've tried replacing the pipe script's contents with
>>> the following:
>>> #####################################################################
>>> #!/bin/bash
>>> /usr/lib/dovecot/deliver -d "sa-training at example.com" -m
>>> "Training.$mode" -p /var/vmail/gtube.txt
>>> # Exit with "success" status code.
>>> exit 0
>>> #####################################################################
>>> And still, the message never arrives when I drag a message from "Inbox"
>>> to "Spam" in the IMAP client, despite the messages logged to syslog,
>>> which "all seem to be correct".
>>> Yet, if I do this using the above content for the script, the "gtube"
>>> message is delivered successfully:
>>> # su vmail
>>> $ /usr/local/bin/sa-learn-pipe-test.sh
>>> The message arrives in the Inbox, presumably because -m "Training.$mode"
>>> does not "resolve" to anything, and per the documentation, the delivery
>>> destination will default to the Inbox in such cases.
>>> What's wrong with my the custom pipe script to which I posted a link in
>>> my previous message, I wonder?
>>> Thanks again for any assistance,
>>> -Ben
>> I don't know how to solve your problem, but I have an alternate method.
>> Create a Junk folder and a ham folder then add the following cron jobs:
>> 30 5 * * * /usr/local/bin/sa-learn --spam
>> /var/vmail/Maildir/.Junk
>> 30 6 * * 6 /usr/local/bin/sa-learn
>> --backup > /etc/mail/spamassassin/spam.db
>> 30 7 * * * /usr/local/bin/sa-learn --ham
>> /var/vmail/Maildir/.Ham
>
> Thank you for taking a look, Edgar!
>
> I already have the very setup that you describe.
>
> The challenge at hand, however, is with regard to *how* messages end-up in the "ham" and "spam" folders in the first place.
>
> This is a multi-user system and each user must be able to contribute to ham/spam training. To make this as easy as possible, and transparent to the end-users, I need for the simple act of dragging a message from Inbox -> Spam (or vice versa) to submit the sample on the user's behalf, automatically. This has all worked swimmingly in the past.
>
> In any case, I've made some progress on this go-around.
>
> I tested my backend pipe script by doing this, as root, and it works:
>
> # /bin/bash /usr/local/bin/sa-learn-pipe.sh --spam < /var/vmail/gtube.txt
>
> The message is delivered to the appropriate mailbox and all is well in the world.
>
> Of course, in practice, Dovecot does not run as root. So, let's try the same as the vmail user:
>
> # su vmail
> $ /bin/bash /usr/local/bin/sa-learn-pipe.sh --spam < /var/vmail/gtube.txt
> /usr/local/bin/sa-learn-pipe.sh: line 8: /tmp/sa-learn-pipe.log: Permission denied
> /usr/local/bin/sa-learn-pipe.sh: line 10: /tmp/sa-learn-pipe.log: Permission denied
> /usr/local/bin/sa-learn-pipe.sh: line 12: /tmp/sa-learn-pipe.log: Permission denied
> /usr/local/bin/sa-learn-pipe.sh: line 16: /tmp/sendmail-parms.txt: Permission denied
> /usr/local/bin/sa-learn-pipe.sh: line 22: /tmp/sa-learn-pipe.log: Permission denied
> /usr/local/bin/sa-learn-pipe.sh: line 37: /tmp/sa-learn-pipe.log: Permission denied
> /usr/local/bin/sa-learn-pipe.sh: line 40: /tmp/sa-learn-pipe.log: Permission denied
> /usr/local/bin/sa-learn-pipe.sh: line 43: /tmp/strace.txt: Permission denied
> /usr/local/bin/sa-learn-pipe.sh: line 47: /tmp/sa-learn-pipe.log: Permission denied
> /usr/local/bin/sa-learn-pipe.sh: line 54: /tmp/sa-learn-pipe.log: Permission denied
>
> Aha! Clearly, the vmail user cannot read from nor write to /tmp. (Why that is, I have no idea, as the /tmp directory's permissions certainly allow for both; maybe Dovecot implements this as a security measure.)
>
If in a chroot /tmp may not exist. That's all I can think of there.
> This prompted me to change all references to /tmp in the pipe script to ~/tmp, and create this directory:
>
> $ whoami
> vmail
> $ mkdir ~/tmp && chmod 770 ~/tmp
> $ /bin/bash /usr/local/bin/sa-learn-pipe.sh --ham < /var/vmail/gtube.txt
>
> No errors this time (at least not on the console).
>
> But I do get this in /var/log/mail.err:
>
> Aug 19 12:04:24 example.com dovecot: lda(sa-training at example.com): Fatal: Can't open delivery mail as raw: Permission denied
>
That's a new one for me.
> I'm not sure how to interpret this message. Where is permission being denied? More importantly, what's the fix?
>
> Thanks for any hints!
>
> -Ben
More information about the dovecot
mailing list