[Dovecot] Issues with VANISHED CHANGEDSINCE

[Michael M Slusarz]

Quoting Timo Sirainen <tss at iki.fi>:

> On 6.11.2012, at 3.49, Michael J Rubinsky wrote:
>
>>> That would require infinitely storing the modseq of when each message
>>> was expunged. Not very nice. Also the RFC talks a lot about this
>>> situation. The SELECT command has two optional parameters to optimize
>>> it.
>>
>> The RFC *does* indicate that a server implementation could,  
>> strictly speaking, be considered in compliance without remembering  
>> modsequences for all expunged messages, but it does explicitly  
>> discourage such implementations. From RFC 5162 [4.1]:
>>
>>   Strictly speaking, a server implementation that doesn't remember mod-
>>   sequences associated with expunged messages can be considered
>>   compliant with this specification.  Such implementations return all
>>   expunged messages specified in the UID set of the UID FETCH
>>   (VANISHED) command every time, without paying attention to the
>>   specified CHANGEDSINCE mod-sequence.  Such implementations are
>>   discouraged, as they can end up returning VANISHED responses that are
>>   bigger than the result of a UID SEARCH command for the same UID set.
>
> This is talking about a server that doesn't permanently remember ANY  
> modseqs for expunges. Dovecot remembers them, not not infinitely.
>
>> It also gives advice to avoid infinitely storing the modsequences  
>> such as "expiring" sequences associated with older expunged  
>> messages, but assigning a single modsequence value to all of the  
>> expired expunged messages.
>
> Dovecot behaves as the section 4.3 describes. Note especially:
>
>    Note that indefinitely storing information about expunged messages
>    can cause storage and related problems for an implementation.
> ..
>    Hence, implementations are encouraged to adopt strategies to protect
>    against such storage problems, such as limiting the size of the queue
>    used to store mod-sequences for expunged messages and "expiring"
>    older records when this limit is reached.  When the selected
>    implementation-specific queue limit is reached, the oldest record(s)
>    are deleted from the queue (note that such records are located at the
>    queue head).  For all such "expired" records, the server needs to
>    store a single mod-sequence, which is the highest mod-sequence for
>    all "expired" expunged messages.
>
> This is exactly what Dovecot does. There is a single modseq  
> associated with all the previously expunged messages. If you try to  
> request expunges for that modseq, it returns all of the expunged  
> messages, which is what you're seeing as a problem.

I see your point, but the problem is that is not intuitive when  
reading the RFC.  One part of the RFC defines the behavior of VANISHED  
(EARLIER) as only returning changes since the mod-sequence given.  And  
you are correct that another part of the RFC says that, essentially, a  
server is allowed to break this required response.

I'm thinking that this is more of an issue with the way the RFC is  
written.  I'll move this over to the imap protocol list to get further  
input.

michael