HIGHESTMODSEQ tracking

[Michael M Slusarz]

Quoting Kostya Vasilyev <kmansoft at gmail.com>:

> 2014-07-10 4:05 GMT+04:00 Michael M Slusarz <slusarz at curecanti.org>:
>
>> Quoting Kostya Vasilyev <kmansoft at gmail.com>:
>>
>>  2014-07-10 1:37 GMT+04:00 Michael M Slusarz <slusarz at curecanti.org>:
>>>
>>> Quoting Kostya Vasilyev <kmansoft at gmail.com>:
>>>
>>>>
>>>> These days, you *really* should be using QRESYNC instead though.
>>>>
>>>
>>> There are some mail servers that support CONDSTORE but not QRESYNC. The
>>> old
>>> chicken and egg IMAP problem :)
>>>
>>
>> [ snip ]
>> Both Dovecot and Cyrus support both CONDSTORE and QRESYNC, and combined
>> that is more than 50% market share, so that should be incentive enough.
>>  Gmail only supports CONDSTORE, but it's the outlier.
>
>
> Gmail still does have a few users, though. A few dozen at least, maybe more
> :)
>
> And it has a big advantage, from my point of view, over Cyrus / Dovecot --
> there is but one server version that's consistent for all accounts.
>
> Yes, they do some things wrong (like not sending message flags changes over
> IDLE connections), but I can test something in my personal account, get
> feedback from 3-5-10 users with @gmail accounts, and be reasonably
> confident that everything is fine (and that I'd know know if it's not).

This is getting a bit off-topic on this list... but Gmail does a LOT  
of things wrong.  Head over to one of the IMAP lists for further  
information.

If you are testing against Gmail as the gold standard as to how a IMAP  
server should operate, I can safely say you are Doing It Wrong.

> For the "more than 50% market share" of Dovecot / Cyrus, do you have a
> breakdown by version number? At least in terms of 1.* vs 2.0 and higher?

I do not.

>> Maybe.  You can't tell until you actually see whether the EXAMINE/SELECT
>> returns HIGHESTMODSEQ or NOMODSEQ.
>
> Are you saying that Dovecot will always (*will always*, and I mean
> *always*) return NOMODSEQ after a client "expresses interested in modseq
> values" and the server can't enable it for some reason?

Much like UIDVALIDITY should never change, NOMODSEQ will never be sent  
(practical usage) for an active CONDSTORE access.  You are asking  
about a tremendously rare occurrence.

The whole deal with "HIGHESTMODSEQ 1" is irrelevant if you enable  
CONDSTORE.  I can't tell you what a server will return if you enable  
CONDSTORE in one session, but then don't in another.  But that doesn't  
matter, since you aren't using HIGHESTMODSEQ in the latter case.  As  
long as CONDSTORE is active, HIGHESTMODSEQ will be updated, at least  
in my 6 year experience with Dovecot which involves handling  
installations with millions of users.

> Or if it was previously enabled, and then well, I don't know, "something
> happened"?
>
> By *always* I mean -- since Dovecot first started having a CONDSTORE in its
> CAPS, including version a.b.c that came with now really old Debian X, and
> version h.j.k that came with now really old RHEL Y, but which are still out
> there on actual mail servers, being used in actual mail accounts?

I have never run into an issue with HIGHESTMODSEQ for a properly  
CONDSTORE-enabled session for Dovecot ever.  I was one of the first  
people (that I am aware of) that implemented CONDSTORE/QRESYNC back in  
the early days (2009) ... and Dovecot was exclusively the server I was  
developing with at that time.

> When something goes wrong in an email app, then to the user, it's always
> the email app developer's fault. Nobody gives a damn about the subtleties
> of what RFC abc says about xyz, or if server version j.k.l from three years
> ago had a bug.

Agree, but only up to a certain point.  If something is so onerous to  
work around, then it *is* ok to say "it's the server's fault and we're  
not going to work around this."  Like everything else in life, there  
is a cost/benefit analysis that must be done to determine where that  
line needs to be drawn.

> So, before enabling certain optimizations for Dovecot, I thought I'd ask on
> a Dovecot mailing list, about actual behavior for this server feature.
>
> I assume this mailing list has people with real Dovecot experience and
> knowledge, and maybe even the developers are lurking here too.
>
> Specifically, I was hoping to hear back maybe something like this:
>
> "Dovecot version X which was packaged in Debian Z, would not update the
> modseq value after command Y".
>
> Or maybe -- which would be great:
>
> "There were no issues with modseq tracking, at all, no reported bugs, code
> not touched, since the feature was originally implemented and advertised as
> CONDSTORE in CAPS in version 1.2.*".

There are certainly bugs - I found several of them years ago when the  
code was brand new (here's a thread:  
http://markmail.org/message/fj74xta5z5uv4nix).  But nothing that was  
showstopping.  And none of those versions are being run anymore for  
all intents and purposes.

The bigger issue, specifically with QRESYNC, is not implementation  
bugs but rather some deficiencies in the original standards (e.g.  
unsolicited FETCHs without UIDs; VANISHED wasn't a required response  
so sequence number tracking was still required) that weren't addressed  
until RFC 7162.  Those are more likely to trip you up then some  
transient implementation bug.

michael