Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11
Hello,
I am trying to get rspam to learn what is spam and what is ham.
After following the IMAPSieve guide in the dovecot wiki (https://wiki2.dovecot.org/HowTo/AntispamWithSieve), the mail.log shows the following, whenever I move a mail from the Spam folder to my inbox and vice versa:
sieve: Opening script 1 of 1 from /usr/lib/dovecot/sieve/report-ham.sieve' sieve: Loading script /usr/lib/dovecot/sieve/report-ham.sieve sieve: Script binary /usr/lib/dovecot/sieve/report-ham.svbin successfully loaded sieve: binary save: not saving binary /usr/lib/dovecot/sieve/report-ham.svbin, because it is already stored sieve: Executing script from /usr/lib/dovecot/sieve/report-ham.svbin'
sieve: action pipe: running program: sa-learn-ham.sh
Mailbox INBOX: Opened mail UID=2996 because: mail stream
Debug: waiting for program /usr/lib/dovecot/sieve/sa-learn-ham.sh' to finish after 0 msecs Error: program /usr/lib/dovecot/sieve/sa-learn-ham.sh' terminated
abnormally, signal 11
Error: sieve: pipe action: failed to pipe message to program
`sa-learn-ham.sh': refer to server log for more information. [2019-11-17
00:39:33]
Error: sieve: Execution of script
/usr/lib/dovecot/sieve/report-ham.sieve failed
My relevant dovecot configuration:
plugin { imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Spam imapsieve_mailbox2_before = file:/usr/lib/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Spam imapsieve_mailbox2_name = *
sieve = file:/var/vmail/sieve/%d/%n/scripts;active=/var/vmail/sieve/%d/%n/active-script.sieve sieve_before = /var/vmail/sieve/global/spam-global.sieve sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /usr/lib/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms }
$ dovecot --version 2.2.27 (c0f36b0)
$ ls -l /usr/lib/dovecot/sieve/ total 24 -rw-r--r-- 1 root root 314 Nov 17 00:25 report-ham.sieve -rw-r--r-- 1 root root 448 Nov 17 00:30 report-ham.svbin -rw-r--r-- 1 root root 199 Nov 17 00:25 report-spam.sieve -rw-r--r-- 1 root root 354 Nov 17 00:30 report-spam.svbin -rwxr-xr-x 1 root root 41 Nov 17 00:32 sa-learn-ham.sh -rwxr-xr-x 1 root root 42 Nov 17 00:32 sa-learn-spam.sh
$ cat sa-learn-ham.sh #!/bin/sh exec /usr/bin/rspamc learn_ham
When I execute the script manually, it seems to work: $ cat [some-mail] | /usr/lib/dovecot/sieve/sa-learn-ham.sh Results for file: stdin (0.011 seconds) success = true; filename = "stdin"; scan_time = 0.012000;
Again, with the same mail: $ cat [some-mail] | /usr/lib/dovecot/sieve/sa-learn-ham.sh Results for file: stdin (0.000 seconds) error = "<some-mail> has been already learned as ham, ignore it"; filename = "stdin"; scan_time = 0.0;
Any ideas what causes the abnormal temrination of the script?
Regards, Jens
Jens Rey via dovecot skrev den 2019-11-17 00:51:
$ cat sa-learn-ham.sh #!/bin/sh exec /usr/bin/rspamc learn_ham
rspamc is not spamc
is rspamd installed ?, is spamassassn installed ?
make sa-laern-ham.sh change name to rspamd-learn-ham.sh and addopt sieve to that name change
i cant se other error on what you have
make sa-laern-ham.sh change name to rspamd-learn-ham.sh and addopt sieve to that name change
I thought about the same thing while following the wiki guide. But in the end it's just the name of a bash script. Now I changed it as you proposed, since it just makes more sense, but the error remains:
Error: program /usr/lib/dovecot/sieve/rspamd-learn-ham.sh' terminated abnormally, signal 11 Error: sieve: pipe action: failed to pipe message to program rspamd-learn-ham.sh': refer to server log for more information.
[2019-11-17 08:35:58]
Error: sieve: Execution of script
/usr/lib/dovecot/sieve/report-ham.sieve failed
Since I want to avoid debugging old and already fixed bugs, I updated to
# dovecot --version 2.3.8 (9df20d2db)
Now the debug log is a bit more chatty, but the error remains:
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Created Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass environment: USER=user@domain.tld Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass environment: HOME=/var/vmail/mailboxes/domain.tld/user Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass environment: HOST=mail.domain.tld Debug: Mailbox INBOX: UID 22039: Opened mail because: mail stream Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Establishing connection Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Forked child process Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Connected to program Error: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): write(program stdin) failed: Broken pipe Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Failed to run program Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Disconnected Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Child process ended Error: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Terminated abnormally with signal 11 Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Destroy Error: sieve: pipe action: failed to pipe message to program `rspamd-learn-ham.sh': refer to server log for more information. [2019-11-17 19:32:40] sieve: left message in mailbox 'INBOX' Error: sieve: Execution of script /usr/lib/dovecot/sieve/report-ham.sieve failed
E-Mail: it@rey.nrw Tel.: +49 211 43636543 Mobil: +49 162 9706249 Anschrift: Adolf-Klarenbach-Straße 16, 40589 Düsseldorf
On 17/11/2019 19:40, Jens Rey via dovecot wrote:
Since I want to avoid debugging old and already fixed bugs, I updated to
# dovecot --version 2.3.8 (9df20d2db)
Now the debug log is a bit more chatty, but the error remains:
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Created Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass environment: USER=user@domain.tld Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass environment: HOME=/var/vmail/mailboxes/domain.tld/user Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass environment: HOST=mail.domain.tld Debug: Mailbox INBOX: UID 22039: Opened mail because: mail stream Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Establishing connection Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Forked child process Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Connected to program Error: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): write(program stdin) failed: Broken pipe Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Failed to run program Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Disconnected Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Child process ended Error: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Terminated abnormally with signal 11 Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): Destroy Error: sieve: pipe action: failed to pipe message to program `rspamd-learn-ham.sh': refer to server log for more information. [2019-11-17 19:32:40] sieve: left message in mailbox 'INBOX' Error: sieve: Execution of script /usr/lib/dovecot/sieve/report-ham.sieve failed
Dovecot forks a child process to execute the program. Subsequently, at some point, the child process fails with a segmentation fault. Granted, that could still be happening in Dovecot code, but since the child process quickly executes the actual program (the shell in this case), it is more likely that this problem emerges outside Dovecot.
You can attach GDB to imap with follow-fork-mode set to 'child', so you can maybe see what causes the segfault.
Regards,
Stephan.
Hi,
You can attach GDB to imap with follow-fork-mode set to 'child', so you can maybe see what causes the segfault.
now I am confused. Hopefully, I used GDB correctly, but as far as I can see, everything seemed to work.
The only difference I can see is that I executed the whole thing as root, but I would hope doveadm uses the correct user (vmail). At least on the file system all emails still belong to vmail, so I guess that works.
For a quick reference, here is what I executed:
root@mail:~# gdb --args doveadm exec imap -u user@domain ... (gdb) set follow-fork-mode child (gdb) r . SELECT Spam . MOVE 1 Inbox
And the (imho) relevant output:
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Establishing connection [New process 13335] imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Forked child process imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Connected to program [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". process 13335 is executing new program: /bin/dash process 13335 is executing new program: /usr/bin/rspamc [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7fffee7ff700 (LWP 13338)] [New Thread 0x7fffebffe700 (LWP 13339)] [New Thread 0x7fffe97fd700 (LWP 13340)] imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Finished streaming payload to program imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Finished input to program imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Disconnected imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Waiting for program to finish after 886 msecs [Thread 0x7fffe97fd700 (LWP 13340) exited] [Thread 0x7fffebffe700 (LWP 13339) exited] [Thread 0x7ffff7fd1140 (LWP 13335) exited] [Inferior 2 (process 13335) exited normally] imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Child process ended imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Destroy imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Info: sieve: pipe action: piped message to program `rspamd-learn-ham.sh'
There is more output, lots more, but we are talking about spam after all... If it is needed, please say so and I will provide it.
Regards, Jens
Op 20-11-2019 om 8:50 schreef Jens Rey via dovecot:
Hi,
You can attach GDB to imap with follow-fork-mode set to 'child', so you can maybe see what causes the segfault.
now I am confused. Hopefully, I used GDB correctly, but as far as I can see, everything seemed to work.
The only difference I can see is that I executed the whole thing as root, but I would hope doveadm uses the correct user (vmail). At least on the file system all emails still belong to vmail, so I guess that works.
For a quick reference, here is what I executed:
root@mail:~# gdb --args doveadm exec imap -u user@domain ... (gdb) set follow-fork-mode child (gdb) r . SELECT Spam . MOVE 1 Inbox
And the (imho) relevant output:
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Establishing connection [New process 13335] imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Forked child process imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Connected to program [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". process 13335 is executing new program: /bin/dash process 13335 is executing new program: /usr/bin/rspamc [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7fffee7ff700 (LWP 13338)] [New Thread 0x7fffebffe700 (LWP 13339)] [New Thread 0x7fffe97fd700 (LWP 13340)] imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Finished streaming payload to program imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Finished input to program imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Disconnected imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Waiting for program to finish after 886 msecs [Thread 0x7fffe97fd700 (LWP 13340) exited] [Thread 0x7fffebffe700 (LWP 13339) exited] [Thread 0x7ffff7fd1140 (LWP 13335) exited] [Inferior 2 (process 13335) exited normally] imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Child process ended imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Destroy imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Info: sieve: pipe action: piped message to program `rspamd-learn-ham.sh'
There is more output, lots more, but we are talking about spam after all... If it is needed, please say so and I will provide it.
Hmm, so now it suddenly works. So, why would it not fail when run in GDB?
Regards, Jens
*NOW* it actually works. I updated dovecot *and* rspamd, so not completely sure, what fixed it. But as far as I remember, I also tested without rspamd by piping messages to a shell script which did not do anything, resulting in the same error.
Am 20.11.19 um 16:32 schrieb Jens Rey via dovecot:
Hmm, so now it suddenly works. So, why would it not fail when run in GDB?
That would be my question as well. I have no idea except from having executed gdb as root and dovecot running as user vmail.
participants (3)
-
Benny Pedersen
-
Jens Rey
-
Stephan Bosch