[Dovecot] dovecot-1.2.2+sieve 0.1.9: sieve issues with (global?) includes
Hi,
After upgrading from dovecot 1.1 to dovecot 1.2.2 and the new sieve implementation (0.1.9), I'm having some strange problems. I'm including some general purpose scripts into my user script from sieve_global_dir.
After some tests it seems that ths scripts get executed at random: sometimes the mail gets stored in the correct dir, and sometimes it goes to inbox. This also happens while using the same compiled .svbin file.
Another indication that something is wrong with the compiled .svbin is output of sieved (see below). Running sieved on the compiled test{1,2,3}.svbin files gives regular output.
Any ideas what could be causing this?
user script:
require ["include"];
include :global "test1.sieve";
include :global "test2.sieve";
include :global "test3.sieve";
keep;
/etc/dovecot/sieve/test1.sieve:
require ["fileinto"];
if header :contains "Subject" "test1" { fileinto "sieve_test"; stop; }
/etc/dovecot/sieve/test2.sieve:
require ["fileinto"];
if header :contains "Subject" "test2" { fileinto "sieve_test"; stop; }
/etc/dovecot/sieve/test3.sieve:
require ["fileinto"];
if header :contains "Subject" "test3" { fileinto "sieve_test"; stop; }
Log output with mail_debug=yes:
# test mail for test1.sieve Jul 28 23:20:27 meredith-mail dovecot: deliver(tomhendr): Loading modules from directory: /usr/lib/dovecot/lda Jul 28 23:20:27 meredith-mail dovecot: deliver(tomhendr): Module loaded: /usr/lib/dovecot/lda/lib90_sieve_plugin.so Jul 28 23:20:27 meredith-mail dovecot: deliver(tomhendr): maildir: data=/var/spool/mail/tomhendr/Maildir Jul 28 23:20:27 meredith-mail dovecot: deliver(tomhendr): maildir++: root=/var/spool/mail/tomhendr/Maildir, index=, control=, inbox=/var/spool/mail/tomhendr/Maildir Jul 28 23:20:27 meredith-mail dovecot: deliver(tomhendr): sieve: using sieve path for user's script: /home/tomhendr/.default.sieve Jul 28 23:20:27 meredith-mail dovecot: deliver(tomhendr): sieve: opening script /home/tomhendr/.default.sieve Jul 28 23:20:27 meredith-mail dovecot: deliver(tomhendr): sieve: executing compiled script /home/tomhendr/.default.sieve Jul 28 23:20:27 meredith-mail dovecot: deliver(tomhendr): sieve: msgid=20090728212027.02EEBD002@smtp.whyscream.net: stored mail into mailbox 'sieve_test'
# test mail for test2.sieve Jul 28 23:20:35 meredith-mail dovecot: deliver(tomhendr): Loading modules from directory: /usr/lib/dovecot/lda Jul 28 23:20:35 meredith-mail dovecot: deliver(tomhendr): Module loaded: /usr/lib/dovecot/lda/lib90_sieve_plugin.so Jul 28 23:20:35 meredith-mail dovecot: deliver(tomhendr): maildir: data=/var/spool/mail/tomhendr/Maildir Jul 28 23:20:35 meredith-mail dovecot: deliver(tomhendr): maildir++: root=/var/spool/mail/tomhendr/Maildir, index=, control=, inbox=/var/spool/mail/tomhendr/Maildir Jul 28 23:20:35 meredith-mail dovecot: deliver(tomhendr): sieve: using sieve path for user's script: /home/tomhendr/.default.sieve Jul 28 23:20:35 meredith-mail dovecot: deliver(tomhendr): sieve: opening script /home/tomhendr/.default.sieve Jul 28 23:20:35 meredith-mail dovecot: deliver(tomhendr): sieve: executing compiled script /home/tomhendr/.default.sieve Jul 28 23:20:35 meredith-mail dovecot: deliver(tomhendr): Namespace : Using permissions from /var/spool/mail/tomhendr/Maildir: mode=0700 gid=-1 Jul 28 23:20:35 meredith-mail dovecot: deliver(tomhendr): sieve: msgid=20090728212035.5F2F8D003@smtp.whyscream.net: stored mail into mailbox 'INBOX'
# test with 2 mails for test2 in a row Jul 28 23:40:50 meredith-mail dovecot: deliver(tomhendr): Loading modules from directory: /usr/lib/dovecot/lda Jul 28 23:40:50 meredith-mail dovecot: deliver(tomhendr): Module loaded: /usr/lib/dovecot/lda/lib90_sieve_plugin.so Jul 28 23:40:50 meredith-mail dovecot: deliver(tomhendr): maildir: data=/var/spool/mail/tomhendr/Maildir Jul 28 23:40:50 meredith-mail dovecot: deliver(tomhendr): maildir++: root=/var/spool/mail/tomhendr/Maildir, index=, control=, inbox=/var/spool/mail/tomhendr/Maildir Jul 28 23:40:50 meredith-mail dovecot: deliver(tomhendr): sieve: using sieve path for user's script: /home/tomhendr/.default.sieve Jul 28 23:40:50 meredith-mail dovecot: deliver(tomhendr): sieve: opening script /home/tomhendr/.default.sieve Jul 28 23:40:50 meredith-mail dovecot: deliver(tomhendr): sieve: executing compiled script /home/tomhendr/.default.sieve Jul 28 23:40:51 meredith-mail dovecot: deliver(tomhendr): sieve: msgid=20090728214050.C2983D002@smtp.whyscream.net: stored mail into mailbox 'sieve_test' Jul 28 23:40:57 meredith-mail dovecot: deliver(tomhendr): Loading modules from directory: /usr/lib/dovecot/lda Jul 28 23:40:57 meredith-mail dovecot: deliver(tomhendr): Module loaded: /usr/lib/dovecot/lda/lib90_sieve_plugin.so Jul 28 23:40:57 meredith-mail dovecot: deliver(tomhendr): maildir: data=/var/spool/mail/tomhendr/Maildir Jul 28 23:40:57 meredith-mail dovecot: deliver(tomhendr): maildir++: root=/var/spool/mail/tomhendr/Maildir, index=, control=, inbox=/var/spool/mail/tomhendr/Maildir Jul 28 23:40:57 meredith-mail dovecot: deliver(tomhendr): sieve: using sieve path for user's script: /home/tomhendr/.default.sieve Jul 28 23:40:57 meredith-mail dovecot: deliver(tomhendr): sieve: opening script /home/tomhendr/.default.sieve Jul 28 23:40:57 meredith-mail dovecot: deliver(tomhendr): sieve: executing compiled script /home/tomhendr/.default.sieve Jul 28 23:40:57 meredith-mail dovecot: deliver(tomhendr): Namespace : Using permissions from /var/spool/mail/tomhendr/Maildir: mode=0700 gid=-1 Jul 28 23:40:57 meredith-mail dovecot: deliver(tomhendr): sieve: msgid=20090728214057.43048D002@smtp.whyscream.net: stored mail into mailbox 'INBOX'
Output of 'sieved /home/tomhendr/.default.svbin
Required extensions:
0: include (16) 1: fileinto (5) Panic: file ext-include-variables.c: line 133 (ext_include_variables_dump): assertion failed: (global_vars != NULL) Error: Raw backtrace: sieved [0xa7fbfe10] -> sieved(default_fatal_handler+0x57) [0xa7fbfe97] -> sieved [0xa7fc006e] -> sieved [0xa7f7384c] -> sieved(ext_include_binary_dump+0x48) [0xa7f531c8] -> sieved(sieve_binary_dumper_run+0x119) [0xa7f671a9] -> sieved(sieve_dump+0x42) [0xa7f4bf82] -> sieved(sieve_tool_dump_binary_to+0x6f) [0xa7f7b77f] -> sieved(main+0x174) [0xa7f42564] -> /lib/libc.so.6(__libc_start_main+0xe2) [0xa7dce712] -> sieved [0xa7f422b1] Aborted
-- Regards, Tom
Tom Hendrikx schreef:
Hi,
After upgrading from dovecot 1.1 to dovecot 1.2.2 and the new sieve implementation (0.1.9), I'm having some strange problems. I'm including some general purpose scripts into my user script from sieve_global_dir.
After some tests it seems that ths scripts get executed at random: sometimes the mail gets stored in the correct dir, and sometimes it goes to inbox. This also happens while using the same compiled .svbin file.
Another indication that something is wrong with the compiled .svbin is output of sieved (see below). Running sieved on the compiled test{1,2,3}.svbin files gives regular output.
Output of 'sieved /home/tomhendr/.default.svbin
Required extensions:
0: include (16) 1: fileinto (5) Panic: file ext-include-variables.c: line 133 (ext_include_variables_dump): assertion failed: (global_vars != NULL) Error: Raw backtrace: sieved [0xa7fbfe10] -> sieved(default_fatal_handler+0x57) [0xa7fbfe97] -> sieved [0xa7fc006e] -> sieved [0xa7f7384c] -> sieved(ext_include_binary_dump+0x48) [0xa7f531c8] -> sieved(sieve_binary_dumper_run+0x119) [0xa7f671a9] -> sieved(sieve_dump+0x42) [0xa7f4bf82] -> sieved(sieve_tool_dump_binary_to+0x6f) [0xa7f7b77f] -> sieved(main+0x174) [0xa7f42564] -> /lib/libc.so.6(__libc_start_main+0xe2) [0xa7dce712] -> sieved [0xa7f422b1] Aborted
Thanks for the thorough bug report. I could reproduce this instantly. I forgot to initialize the binary for use with the include extension in one particular situation: when the binary is loaded from disk. Directly after compilation, all works well. Fixed:
http://hg.rename-it.nl/dovecot-1.2-sieve/rev/d989537882d0
I've tested this fix with sieved and sieve-test and both show correct behaviour, so I am pretty confident that the problem is resolved for the actual Sieve plugin as well. Your dump should look as in the attached file after applying the fix. I'll make a new release soon, once I've thoroughly tested recent changes to the action execution code.
Sadly the binary load/save functionality is not yet part of the test suite, giving this bug the opportunity to slip through. Adding this to the testsuite should not be too much work, so I'll give that some more priority.
Regards,
-- Stephan Bosch stephan@rename-it.nl
Required extensions:
0: include (17) 1: fileinto (5)
Included global script 'test1.sieve' (block: 3):
00000000: EXTENSIONS [1]: 00000001: fileinto 00000002: HEADER 00000005: match type: contains 00000008: header names: STR[7] "Subject" 00000012: key list: STR[5] "test1" 0000001a: JMPFALSE 20 [0000002f] 0000001f: FILEINTO 00000020: (source line: 4) 00000021: folder: STR[10] "sieve_test" 0000002e: STOP 0000002f: [End of code]
- Included global script 'test2.sieve' (block: 4):
00000000: EXTENSIONS [1]: 00000001: fileinto 00000002: HEADER 00000005: match type: contains 00000008: header names: STR[7] "Subject" 00000012: key list: STR[5] "test2" 0000001a: JMPFALSE 20 [0000002f] 0000001f: FILEINTO 00000020: (source line: 4) 00000021: folder: STR[10] "sieve_test" 0000002e: STOP 0000002f: [End of code]
- Included global script 'test3.sieve' (block: 5):
00000000: EXTENSIONS [1]: 00000001: fileinto 00000002: HEADER 00000005: match type: contains 00000008: header names: STR[7] "Subject" 00000012: key list: STR[5] "test3" 0000001a: JMPFALSE 20 [0000002f] 0000001f: FILEINTO 00000020: (source line: 4) 00000021: folder: STR[10] "sieve_test" 0000002e: STOP 0000002f: [End of code]
- Main program (block: 1):
00000000: EXTENSIONS [1]: 00000001: include 00000002: INCLUDE: 00000004: script: test1.sieve [ID: 1, BLOCK: 3] 00000006: INCLUDE: 00000008: script: test2.sieve [ID: 2, BLOCK: 4] 0000000a: INCLUDE: 0000000c: script: test3.sieve [ID: 3, BLOCK: 5] 0000000e: KEEP 0000000f: (source line: 19) 00000010: [End of code]
Stephan Bosch schreef:
Tom Hendrikx schreef:
Hi,
After upgrading from dovecot 1.1 to dovecot 1.2.2 and the new sieve implementation (0.1.9), I'm having some strange problems. I'm including some general purpose scripts into my user script from sieve_global_dir.
After some tests it seems that ths scripts get executed at random: sometimes the mail gets stored in the correct dir, and sometimes it goes to inbox. This also happens while using the same compiled .svbin file.
Another indication that something is wrong with the compiled .svbin is output of sieved (see below). Running sieved on the compiled test{1,2,3}.svbin files gives regular output.
Output of 'sieved /home/tomhendr/.default.svbin
Required extensions:
0: include (16) 1: fileinto (5) Panic: file ext-include-variables.c: line 133 (ext_include_variables_dump): assertion failed: (global_vars != NULL) Error: Raw backtrace: sieved [0xa7fbfe10] -> sieved(default_fatal_handler+0x57) [0xa7fbfe97] -> sieved [0xa7fc006e] -> sieved [0xa7f7384c] -> sieved(ext_include_binary_dump+0x48) [0xa7f531c8] -> sieved(sieve_binary_dumper_run+0x119) [0xa7f671a9] -> sieved(sieve_dump+0x42) [0xa7f4bf82] -> sieved(sieve_tool_dump_binary_to+0x6f) [0xa7f7b77f] -> sieved(main+0x174) [0xa7f42564] -> /lib/libc.so.6(__libc_start_main+0xe2) [0xa7dce712] -> sieved [0xa7f422b1] Aborted
Thanks for the thorough bug report. I could reproduce this instantly. I forgot to initialize the binary for use with the include extension in one particular situation: when the binary is loaded from disk. Directly after compilation, all works well. Fixed:
Applied and tested. sieved and sieve-test now complain with an error message, but everything seems to function when run from within dovecot.
Error: sieve: include: sieve_dir and home not set (wanted script spam.sieve) Error: failed to load binary: .default.svbin
I'll wait for your new release, but of course I'm willing to test another patch beforehand :)
-- regards, Tom
Tom Hendrikx schreef:
Applied and tested. sieved and sieve-test now complain with an error message, but everything seems to function when run from within dovecot.
Error: sieve: include: sieve_dir and home not set (wanted script spam.sieve) Error: failed to load binary: .default.svbin
I'll wait for your new release, but of course I'm willing to test another patch beforehand :)
Oh yes, I should have mentioned that. The sieve tools don't retrieve settings from the dovecot config file, nor are they (yet) aware of the config settings extensions may rely on. So, the SIEVE_DIR and SIEVE_GLOBAL_DIR environments are not set. Setting these beforehand will fix the problem.
BTW, before I fixed your bug I had the same error for the above reason. But, as you may have noticed, that error is wrong, since you are using :global. I've committed a fix for that already:
http://hg.rename-it.nl/dovecot-1.2-sieve/rev/449d8ecb0f34
Executing something like this should work:
export SIEVE_GLOBAL_DIR=/var/lib/dovecot/sieve sieved ~/.dovecot.svbin
I will add the means to specify configuration items at the command line in the near future.
Regards,
-- Stephan Bosch stephan@rename-it.nl
Stephan Bosch wrote:
Tom Hendrikx schreef: BTW, before I fixed your bug I had the same error for the above reason. But, as you may have noticed, that error is wrong, since you are using :global. I've committed a fix for that already:
http://hg.rename-it.nl/dovecot-1.2-sieve/rev/449d8ecb0f34
Executing something like this should work:
Works like a charm ;)
export SIEVE_GLOBAL_DIR=/var/lib/dovecot/sieve sieved ~/.dovecot.svbin
I will add the means to specify configuration items at the command line in the near future.
Adding above method to the man page would already suffice, I guess
None the less, thanks for the quick reaction and all the hard work :)
-- Regards, Tom
participants (2)
-
Stephan Bosch
-
Tom Hendrikx