[Dovecot] duplicates with multiple To/CC and sieve redirect copy

[Stephan Bosch]

On 2/22/2012 12:15 AM, Adam Szpakowski wrote:
> On 22.02.2012 00:09, Timo Sirainen wrote:
>> Well, it would be possible to build a doveadm script that deletes the 
>> duplicates after delivery, but currently there's no implementation to 
>> avoid delivering duplicate Message-IDs in the first place.
>>
>> I don't really like such a Message-ID-based deduplication feature 
>> enabled by default, but something like this could be nice:
>>
>> fileinto :copy :x-deduplicate "boss";
>>
>> Anyway, probably not going to be implemented anytime soon.
> Maybe there is a way to use a procmail with something like this:
>
> :0 Wh: msgid.lock
> | formail -D 8192 .msgid.cache
>
> But is there a safe way to use it together with sieve? Using 
> Pigeonhole Sieve Pipe Plugin?
>

There are a few options:

* You can use Procmail as primary delivery agent and invoke 
dovecot-lda/sieve from within Procmail once Procmail has determined that 
it is not a duplicate.

* Invoke procmail from Sieve using the pipe extension (i.e. the other 
way around). This  has the disadvantage that Procmail will have to take 
care of final delivery, meaning the Dovecot indexes are not updated.

* For Pigeonhole v0.3 there is the possibility to "filter" the message 
through Procmail using the sieve_extprograms plugin, but I haven't 
actually tested something like that.

* I've just created an alternative that implements something similar to 
the Procmail code you posted above, but from within Sieve itself. It is 
a custom language extension called vnd.dovecot.duplicate and it adds the 
"duplicate" test. This test keeps track of which Message-IDs it has seen 
before in earlier deliveries and yields a true result if the message was 
seen before, e.g.:

require "vnd.dovecot.duplicate";

if duplicate {
     discard;
}

Read the specification for details ("name" argument is not yet implemented):

http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate/raw-file/4b1dbda4d3fc/doc/rfc/spec-bosch-sieve-duplicate.txt

The repository is at: http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate

This plugin is only a few hours old, experimental, and largely untested, 
so test it thoroughly before considering to use this. Read the INSTALL 
file for compile and installation instructions.

Comments are welcome.

Regards,

Stephan.