Sieve: deleteheader not working with duplicate filter for implicit keep

[Alec Moskvin]

Hi Stephan,

On Thursday 03 September 2020 22:28:20, Stephan Bosch wrote:
>
>
> On 03/09/2020 23:25, Alec Moskvin wrote:
> > Hi Stephan,
> >
> > On Wednesday 02 September 2020 19:59:57, Stephan Bosch wrote:
> >>
> >> On 29/08/2020 21:04, Alec Moskvin wrote:
> >>> Hello,
> >>>
> >>> I have a rule to always delete a header. If the message gets fileinto'd,
> >>> the header is gone, but if it's delivered into the INBOX through
> >>> implicit keep, the header does not get deleted.
> >>>
> >>> Interestingly, if I remove the "if duplicate" filter, the header does
> >>> get deleted as expected.
> >>>
> >>> I'm using the latest Dovecot 2.3.11.3 / Pigeonhole 0.5.11.
> >>>
> >>> Any idea why this is happening?
> >> Can you enable Sieve tracing? That should show you in detail what is
> >> happening during delivery and why.
> > It's saying "deleted 1 occurrences of header", but when it gets stored,
> > the header is still there.
>
> Hmm, did you try tracing with actual delivery using the sieve_trace*
> settings
> (https://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Trace_Debugging)?

Sure, same result:


alec at haswell ~ $ grep sieve_trace /etc/dovecot/conf.d/90-sieve.conf
  sieve_trace_dir = ~/sievetrace
  sieve_trace_level = matching
  sieve_trace_debug = yes
  sieve_trace_addresses = yes
alec at haswell ~ $ nl -ba .dovecot.sieve
     1	require ["fileinto", "editheader", "duplicate", "mailbox"];
     2
     3	deleteheader "X-Bad-Header";
     4
     5	if duplicate {
     6		fileinto :create "Trash";
     7		stop;
     8	}
alec at haswell ~ $ /usr/libexec/dovecot/deliver << -end-
To: me at example.com
From: someone at example.com
Subject: Test Message
Message-Id: <${RANDOM}@example.net>
Date: Thu, 20 Aug 2020 00:21:24 +0000 (UTC)
X-Bad-Header: DELETE ME

Hello world
-end-
alec at haswell ~ $ cat .maildir/new/*
To: me at example.com
From: someone at example.com
Subject: Test Message
Message-Id: <12963 at example.net>
Date: Thu, 20 Aug 2020 00:21:24 +0000 (UTC)
X-Bad-Header: DELETE ME

Hello world
alec at haswell ~ $ cat sievetrace/20200903-224142.61896.1.trace
Sieve trace log for message delivery:

  Username: alec
  Session ID: 6T4DNWapUV/I8QAAISDkhQ
  Sender: <>
  Final recipient: <alec at haswell>
  Default mailbox: INBOX


00000000:       ## Started executing script '.dovecot'
0000000a:    3: deleteheader command
0000000a:    3:   deleting header `X-Bad-Header'
0000000a:    3:   deleted 1 occurrences of header `X-Bad-Header'
0000001c:    5: duplicate test
0000001c:    5:   message is not a duplicate
0000001d:    5: jump if result is false
0000001d:    5:   jumping to line 7 [00000030]
00000030:       ## Finished executing script '.dovecot'

> >
> >
> > + nl -ba mini.sieve
> >       1	require ["fileinto", "editheader", "duplicate", "mailbox"];
> >       2
> >       3	deleteheader "X-Bad-Header";
> >       4
> >       5	if duplicate {
> >       6		fileinto :create "Trash";
> >       7		stop;
> >       8	}
> >
> > + tee test.msg
> > To: me at example.com
> > From: someone at example.com
> > Subject: Test Message
> > Message-Id: <8500 at example.net>
> > Date: Thu, 20 Aug 2020 00:21:24 +0000 (UTC)
> > X-Bad-Header: DELETE ME
> >
> > Hello world
> >
> > + sieve-test -CDe -t- -Tlevel=matching -Tdebug -l maildir:maildir mini.sieve test.msg
> > sieve-test(alec): Debug: sieve: Pigeonhole version 0.5.11 (d71e0372) initializing
> > sieve-test(alec): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts.
> > sieve-test(alec): Debug: sieve: file storage: Using Sieve script path: /home/alec/sievetest/header/mini.sieve
> > sieve-test(alec): Debug: sieve: file storage: script: Opened script `mini' from `/home/alec/sievetest/header/mini.sieve'
> > sieve-test(alec): Debug: sieve: file storage: script: Opened script for reading
> > sieve-test(alec): Debug: sieve: Script `mini' from /home/alec/sievetest/header/mini.sieve successfully compiled
> >
> >        ## Started executing script 'mini'
> > sieve-test(alec): Debug: sieve: Started running script `/home/alec/sievetest/header/mini.sieve'
> >     3: deleteheader command
> >     3:   deleting header `X-Bad-Header'
> >     3:   deleted 1 occurrences of header `X-Bad-Header'
> >     5: duplicate test
> > sieve-test(alec): Info: checked duplicate for user alec.
> >
> >     5:   message is not a duplicate
> >     5: jump if result is false
> >     5:   jumping to line 7
> > sieve-test(alec): Debug: sieve: Finished running script `/home/alec/sievetest/header/mini.sieve'
> >        ## Finished executing script 'mini'
> >
> > sieve-test(alec): Info: sieve: msgid=<8500 at example.net>: stored mail into mailbox 'INBOX'
> > info: msgid=<8500 at example.net>: stored mail into mailbox 'INBOX'.
> > sieve-test(alec): Info: marked duplicate for user alec.
> >
> > sieve-test(alec): Info: final result: success
> > sieve-test(alec): Debug: sieve: file storage: script: Closed script
> >
> > + cat maildir/new/1599167523.M233933P55196.haswell,S=179,W=187
> > To: me at example.com
> > From: someone at example.com
> > Subject: Test Message
> > Message-Id: <8500 at example.net>
> > Date: Thu, 20 Aug 2020 00:21:24 +0000 (UTC)
> > X-Bad-Header: DELETE ME
> >
> > Hello world
>