Op 1/22/2018 om 8:58 PM schreef Stephan Bosch:
Op 1/21/2018 om 4:34 PM schreef Stephan Bosch:
Op 1/20/2018 om 11:01 PM schreef Adam Weinberger:
On 20 Jan, 2018, at 10:05, Adam Weinberger <adamw@adamw.org> wrote:
On 19 Jan, 2018, at 4:39, Aki Tuomi <aki.tuomi@dovecot.fi> wrote:
On 19.01.2018 04:35, Adam Weinberger wrote:
Since upgrading to 2.3.0 / 0.5.0.1, incoming PDFs are getting mangled. It seems to be happening when I use vnd.dovecot.filter. When I comment out the block, things come through fine.
My filter block looks like this: require "vnd.dovecot.filter"; filter "bogofilter_filter";
if header :contains "X-Bogosity" [ "Spam, tests=bogofilter, spamicity=1.00", "Spam, tests=bogofilter, spamicity=0.99" ] { fileinto "spam/totally"; stop; } elsif header :contains "X-Bogosity" "Spam," { fileinto "spam/probably"; stop; } elsif header :contains "X-Bogosity" "Unsure," { fileinto "spam/maybe"; stop; }
filter/bogofilter_filter looks like this: #!/bin/sh cat /dev/stdin | /usr/local/bin/bogofilter -p -e -c /path/to/bogofilter.cf
Images seem to come through fine. I didn't have this problem before the upgrade. It could easily be a problem with bogofilter, but I figured that I'd check here first to see if anybody else has an attachment-mangling issue with vnd.dovecot.filter.
# Adam
-- Adam Weinberger adamw@adamw.org http://www.adamw.org It might work better with
exec /usr/local/bin/bogofilter -p -e -c /path/to/bogofilter.cf
Aki After looking into this some more, what's happening is that messages are getting truncated. Regardless of whether it's an attachment or just a long message body, the message gets truncated.
To show this, I made a fileinto :copy rule to save a message copy before it gets sent off to bogofilter.
You can see from the below pastings that the message is definitely getting truncated. When I replace my filter script with "cat /dev/stdin", it works fine. When I send the message to bogofilter on the command-line, it works fine. Something about the combination of vnd.dovecot.filter + bogofilter is causing the messages to truncate, and I don't know whether it's from truncating on the send or receive of the filter.
Can you give me any advice?
$ jot 200000 1 > numbers.txt $ du -a . | grep numbers 1260 ./numbers-sent.txt 1248 ./numbers-received.txt
root@imap:~# ll test-* 125 -rw------- 1 root wheel 123K Jan 20 09:35 test-afterbogo.msg 149 -rw------- 1 root wheel 136K Jan 20 09:35 test-beforebogo.msg The more I look into it, the more it looks to me like pigeonhole is somehow losing the last 4-6K of messages over 100K.
When my filter script is: cat /dev/stdin | tee /tmp/input | bogofilter[...] | tee /tmp/output Then /tmp/output is the full message, but what pigeonhole gets back is truncated at by about 4K.
Same with [...] bogofilter[...] > /tmp/output; cat /tmp/output The message is truncated there too.
I've tried everything I can think of to make sure it's not a buffering problem: [...] bogofilter[...] | cat [...] bogofilter[...] | dd bs=1k [...] bogofilter[...] ; a=$?; sleep 2; exit $a
But everything I do comes up with the same problem.
But a simple "cat /dev/stdin" and the message comes through fine.
Your help is really appreciated. I'm in the process of buying a house, so the mangled attachments is not coming at a great time :-) We are looking into it. Problem found; working on a fix.
Fix pending:
https://github.com/stephanbosch/dovecot-core/commits/fix-program-client-big-...
This is a feature branch of 9 commits containing the fix and also cleanups and reliability improvements. This is of course based on master.
Regards,
Stephan.