dovecot sieve duplicates detection

[André Rodier]

On 25/04/18 20:20, Stephan Bosch wrote:
> 
> 
> Op 23/04/2018 om 22:03 schreef André Rodier:
>> On 23/04/18 14:18, Stephan Bosch wrote:
>>>
>>>
>>> Op 11-4-2018 om 23:58 schreef André Rodier:
>>>> Hello,
>>>>
>>>> I have tested the sieve duplicate script with success so far, but I 
>>>> have
>>>> a question.
>>>
>>> Sieve duplicate script? You mean the Sieve duplicate extension (RFC 
>>> 7352)?
>>>
>>>> I would like to know if the "duplicate" sieve flag in Dovecot is global
>>>> to all folders, or specific to one folder only.
>>>
>>> It uses the lda-dupes file in the user's home directory. So, it is 
>>> not normally related to folders, although the identifier used for 
>>> duplicate matching could be composed of the mailbox name if you want.
>>>
>>>> For instance, if I copy an email from one folder to another, and I have
>>>> a discard action on duplicate email, is this action will be applied (in
>>>> this case, discard) or not.
>>>
>>> Are you talking about IMAPSieve now? I am not sure "duplicate" is 
>>> currently even allowed in that context.
>>>
>>>> If the duplicate is global to all folders, is there a way to restrict
>>>> the search in one folder only.
>>>
>>> You can set the :uniqueid parameter accordingly.
>>>
>>> Regards,
>>>
>>> Stephan.
>>
>> Thank you, Stephan.
>>
>> Yes, I meant the Sieve duplicate extension.
>>
>> I am using a program to import email (mbsync), which use the IMAP 
>> append function. Sometimes, the import fail and I have to restart the 
>> program. Unfortunately, the same emails are imported again.
>>
>> I found a fix by using a dovecot IMAP sieve script executed on the 
>> APPEND action 
>> (https://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve). I wrote 
>> a custom sieve script that "discard" the ones that are detected as 
>> "duplicate". It worked very well and the emails were not any more 
>> imported twice.
>>
>> However, there was a huge side effect: archiving an email with 
>> Thunderbird is not working any more, and even lost! I have been able 
>> to understand the error as this:
>>
>> 1. When archiving an email with Thunderbird, it is first copied 
>> (APPEND) into the archive folder, but the original folder is not 
>> expunged.
>> 2. The sieve script detect the email as duplicate, and discard it.
>> 3. When the original folder is expunged, the source email is lost...
>>
>> My conclusion was the duplicate detection function is global to all 
>> folders.
>>
>> If I could restrict the detection of duplicates in the current folder 
>> only, this would let me run the import program again without error.
> 
> Specify the ID used for duplicate checking explicitly using the 
> :uniqueid argument (https://tools.ietf.org/html/rfc7352#section-3.1). 
> Using the variables extenion, compose the uniqueid from the message-id 
> and the mailbox name.
> 
> Regards,
> 
> Stephan.
> 

Thank you, I will try this.

André