Antispam plugin: insufficent error messages
Hi!
"Dovecot always logs a detailed error message if something goes wrong. If it doesn't, it's considered a bug and will be fixed." (http:// wiki2.dovecot.org/Logging)
I'm trying to set up a spam filter with dovecot-antispam and dspam as backend.
When I move a spam message from INBOX to Spam, I get this in syslog:
Nov 28 21:15:58 desktop imap: antispam: mailbox_is_unsure(Spam): 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_trash(INBOX): 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_trash(Spam): 0 Nov 28 21:15:58 desktop imap: antispam: mail copy: from trash: 0, to trash: 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_spam(INBOX): 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_spam(Spam): 1 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_unsure(INBOX): 0 Nov 28 21:15:58 desktop imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0
There are no log entries about how dspam is called, or what goes on. Indeed, dspam doesn't get called at all. I know, because I've examined the antispam- plugin source code. There would be a log message if dspam was called.
It's hard to study the source code, because there are hardly any comments.
I have been able to trace the problem to function signature_extract_to_list() in signature.c. There, -1 is returned:
int signature_extract_to_list(const struct signature_config *cfg, struct mailbox_transaction_context *t, struct mail *mail, struct siglist **list, enum classification wanted) { const char *const *signatures; struct siglist *item;
signatures = get_mail_headers(mail, cfg->signature_hdr);
if (!signatures || !signatures[0]) {
if (!cfg->signature_nosig_ignore) {
mail_storage_set_error(t->box->storage,
ME(NOTPOSSIBLE)
"antispam signature not found");
return -1; /* <-- HERE */
} else {
return 0;
}
}
while (signatures[1])
signatures++;
item = i_new(struct siglist, 1);
item->next = *list;
item->wanted = wanted;
item->sig = i_strdup(signatures[0]);
*list = item;
return 0;}
So, what's happening is "antispam signature not found". My question: What does this mean? What's going on?
My "dovecot -n" output is attached.
Bye Volker
One more point:
The spam message, which has been moved to the Spam folder for training dspam, doesn't get moved. When I refresh (in KMail), the message reappears in INBOX.
Bye V.W.
Op 28-11-2016 om 21:28 schreef Volker Wysk:
Hi!
"Dovecot always logs a detailed error message if something goes wrong. If it doesn't, it's considered a bug and will be fixed." (http:// wiki2.dovecot.org/Logging)
Yes, that is indeed what Dovecot and Pigeonhole do. However, AntiSpam is not part of the normal Dovecot distribution, nor is it a plugin created by Dovecot developers. From what I know, Dovecot has mirrored/hosted the AntiSpam Mercurial repository in the past and contributed a few small fixes, but nothing more. Also, it doesn't look like it is actively maintained at the moment.
Pigeonhole Sieve recently gained support for the IMAPSieve extension (http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve). When combined with the exprograms plugin (http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms), you should be able achieve something quite similar using a Sieve script, although I have never tried that myself.
Regards,
Stephan.
Am Dienstag, 29. November 2016, 13:59:52 CET schrieb Stephan Bosch:
However, AntiSpam is not part of the normal Dovecot distribution, nor is it a plugin created by Dovecot developers. From what I know, Dovecot has mirrored/hosted the AntiSpam Mercurial repository in the past and contributed a few small fixes, but nothing more. Also, it doesn't look like it is actively maintained at the moment.
So, if I would debug the antispam plugin, adding more log messages, finding the reason it doesn't work now, would my changes be included in the antispam distribution? Do you know any documentation about the Dovecot plugin API? I couldn't find any.
Pigeonhole Sieve recently gained support for the IMAPSieve extension (http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve). When combined with the exprograms plugin (http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extprograms), you should be able achieve something quite similar using a Sieve script, although I have never tried that myself.
Thanks for the hint. This looks interesing, but also quite complicated. I've read the two pages you specified. Is there any good documentation? I have Peer Heinlein's Dovecot book. But there isn't much in it about plugins.
Bye Volker
On 11/28/2016 3:28 PM, Volker Wysk wrote:
Hi!
"Dovecot always logs a detailed error message if something goes wrong. If it doesn't, it's considered a bug and will be fixed." (http:// wiki2.dovecot.org/Logging)
I'm trying to set up a spam filter with dovecot-antispam and dspam as backend.
When I move a spam message from INBOX to Spam, I get this in syslog:
Nov 28 21:15:58 desktop imap: antispam: mailbox_is_unsure(Spam): 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_trash(INBOX): 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_trash(Spam): 0 Nov 28 21:15:58 desktop imap: antispam: mail copy: from trash: 0, to trash: 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_spam(INBOX): 0 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_spam(Spam): 1 Nov 28 21:15:58 desktop imap: antispam: mailbox_is_unsure(INBOX): 0 Nov 28 21:15:58 desktop imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0
There are no log entries about how dspam is called, or what goes on. Indeed, dspam doesn't get called at all. I know, because I've examined the antispam- plugin source code. There would be a log message if dspam was called.
It's hard to study the source code, because there are hardly any comments.
I have been able to trace the problem to function signature_extract_to_list() in signature.c. There, -1 is returned:
int signature_extract_to_list(const struct signature_config *cfg, struct mailbox_transaction_context *t, struct mail *mail, struct siglist **list, enum classification wanted) { const char *const *signatures; struct siglist *item;
signatures = get_mail_headers(mail, cfg->signature_hdr); if (!signatures || !signatures[0]) { if (!cfg->signature_nosig_ignore) {
mail_storage_set_error(t->box->storage, ME(NOTPOSSIBLE) "antispam signature not found"); return -1; /* <-- HERE */ } else { return 0; }}
while (signatures[1]) signatures++;
item = i_new(struct siglist, 1); item->next = *list; item->wanted = wanted; item->sig = i_strdup(signatures[0]);
*list = item;
return 0; }
So, what's happening is "antispam signature not found". My question: What does this mean? What's going on?
I too encountered this same issue after upgrading to 2.2.27. I traced it to an apparent change in Dovecot's mail_get_headers() function, which now returns a non-zero value even when the header is found. This wasn't the behavior in 2.2.24, though I'm not sure if it was in .25 or .26. I haven't tracked down the details further.
Anyway, the antispam plugin uses this API to find the X-DSPAM-Signature header, and the change causes the plugin to abort. This happens in the plugin's antispam-plugin.h wrapper, which I modified as follows.
I'm confused why the dovecot API behavior changed, but I hope this helps!
Tom.
*** antispam-plugin.h 2016-12-17 17:25:48.571704442 -0500 --- antispam-plugin.h.orig 2016-12-17 17:24:55.010093416 -0500
*** 268,275 **** static inline const char *const * get_mail_headers(struct mail *mail, const char *hdr) { ! const char *const *ret = NULL; ! mail_get_headers(mail, hdr, &ret); return ret; }
--- 268,276 ---- static inline const char *const * get_mail_headers(struct mail *mail, const char *hdr) { ! const char *const *ret; ! if (mail_get_headers(mail, hdr, &ret)) ! return NULL; return ret; }
Am Samstag, 17. Dezember 2016, 17:57:54 CET schrieb Tom Talpey:
I too encountered this same issue after upgrading to 2.2.27. I traced it to an apparent change in Dovecot's mail_get_headers() function, which now returns a non-zero value even when the header is found. This wasn't the behavior in 2.2.24, though I'm not sure if it was in .25 or .26. I haven't tracked down the details further.
Anyway, the antispam plugin uses this API to find the X-DSPAM-Signature header, and the change causes the plugin to abort. This happens in the plugin's antispam-plugin.h wrapper, which I modified as follows.
I'm confused why the dovecot API behavior changed, but I hope this helps!
Thank you for your help, but I've already done it differently now. I'm using the spam filter capabilities of my mail reader (KMail). This uses an external spam filter (I've chosen CRM114). That's not optimal, because I have two spam filters now (on my desktop and my laptop), and no spam filter for my smartphone, but it works.
Bye Volker
On 18 Dec 2016, at 0.57, Tom Talpey tom@talpey.com wrote:
I too encountered this same issue after upgrading to 2.2.27. I traced it to an apparent change in Dovecot's mail_get_headers() function, which now returns a non-zero value even when the header is found. This wasn't the behavior in 2.2.24, though I'm not sure if it was in .25 or .26. I haven't tracked down the details further.
Anyway, the antispam plugin uses this API to find the X-DSPAM-Signature header, and the change causes the plugin to abort. This happens in the plugin's antispam-plugin.h wrapper, which I modified as follows.
I'm confused why the dovecot API behavior changed, but I hope this helps!
It was mostly an accident. There was a mismatch between mail_get_headers() and mail_get_headers_utf8() return values, which was fixed the wrong way. Also I think there might have been some code that assumed that mail_get_headers() would return > 0 on success. Anyway, too late to change back now...
! const char *const *ret = NULL; ! mail_get_headers(mail, hdr, &ret); return ret;
That works, but maybe a nicer way that works for both old and new API and checks also the return value:
if (mail_get_headers(mail, hdr, &ret) < 0) return NULL;
}
--- 268,276 ---- static inline const char *const * get_mail_headers(struct mail *mail, const char *hdr) { ! const char *const *ret; ! if (mail_get_headers(mail, hdr, &ret)) ! return NULL; return ret; }
participants (4)
-
Stephan Bosch
-
Timo Sirainen
-
Tom Talpey
-
Volker Wysk