HIGHESTMODSEQ tracking

Tue Jul 15 13:00:19 UTC 2014

2014-07-15 3:47 GMT+04:00 Michael M Slusarz <slusarz at curecanti.org>:

> Quoting Kostya Vasilyev <kmansoft at gmail.com>:
>
>
>> Gmail still does have a few users, though. A few dozen at least, maybe
>> more
>> :)
>>
>> And it has a big advantage, from my point of view, over Cyrus / Dovecot --
>> there is but one server version that's consistent for all accounts.
>>
>> Yes, they do some things wrong (like not sending message flags changes
>> over
>> IDLE connections), but I can test something in my personal account, get
>> feedback from 3-5-10 users with @gmail accounts, and be reasonably
>> confident that everything is fine (and that I'd know know if it's not).
>>
>
> This is getting a bit off-topic on this list... but Gmail does a LOT of
> things wrong.  Head over to one of the IMAP lists for further information.
>

This is just one glaring example. Maybe you've ran into more than I have.

In any case, the point stands - with Gmail, it's much easier to be
confident, from actual testing, that things works a certain way.

>
> If you are testing against Gmail as the gold standard as to how a IMAP
> server should operate,

I never said or implied that. In fact, I pointed out a serious issue with
Gmail's IMAP IDLE implementation, which means the exact opposite of holding
it as a gold standard.

> I can safely say you are Doing It Wrong.

It seems you enjoy pointing out to people when they're "wrong" or
"incorrect" so much, you actually put meaning into their words that's not
there? Or it it just me?

>
>
>  For the "more than 50% market share" of Dovecot / Cyrus, do you have a
>> breakdown by version number? At least in terms of 1.* vs 2.0 and higher?
>>
>
> I do not.

And without being able to get a version number from a Dovecot session (or
so it seems to me -- nothing returned from ID...).... it looks kind of sad.

>
>
>  Maybe.  You can't tell until you actually see whether the EXAMINE/SELECT
>>> returns HIGHESTMODSEQ or NOMODSEQ.
>>>
>>
>> Are you saying that Dovecot will always (*will always*, and I mean
>> *always*) return NOMODSEQ after a client "expresses interested in modseq
>> values" and the server can't enable it for some reason?
>>
>
> Much like UIDVALIDITY should never change, NOMODSEQ will never be sent
> (practical usage) for an active CONDSTORE access.  You are asking about a
> tremendously rare occurrence.
>

In theory, yes, but I just wouldn't want to be surprised (and surprise my
users).

>
> The whole deal with "HIGHESTMODSEQ 1" is irrelevant if you enable
> CONDSTORE.  I can't tell you what a server will return if you enable
> CONDSTORE in one session, but then don't in another.  But that doesn't
> matter, since you aren't using HIGHESTMODSEQ in the latter case.  As long
> as CONDSTORE is active, HIGHESTMODSEQ will be updated, at least in my 6
> year experience with Dovecot which involves handling installations with
> millions of users.

Thank you.

This is the type of response, based on actual real-world experience, that I
was looking for.

>
>
>  Or if it was previously enabled, and then well, I don't know, "something
>> happened"?
>>
>> By *always* I mean -- since Dovecot first started having a CONDSTORE in
>> its
>> CAPS, including version a.b.c that came with now really old Debian X, and
>> version h.j.k that came with now really old RHEL Y, but which are still
>> out
>> there on actual mail servers, being used in actual mail accounts?
>>
>
> I have never run into an issue with HIGHESTMODSEQ for a properly
> CONDSTORE-enabled session for Dovecot ever.  I was one of the first people
> (that I am aware of) that implemented CONDSTORE/QRESYNC back in the early
> days (2009) ... and Dovecot was exclusively the server I was developing
> with at that time.

Great. Thank you again for a data point that comes from the real world.

>
>
>  When something goes wrong in an email app, then to the user, it's always
>> the email app developer's fault. Nobody gives a damn about the subtleties
>> of what RFC abc says about xyz, or if server version j.k.l from three
>> years
>> ago had a bug.
>>
>
> Agree, but only up to a certain point.  If something is so onerous to work
> around, then it *is* ok to say "it's the server's fault and we're not going
> to work around this."  Like everything else in life, there is a
> cost/benefit analysis that must be done to determine where that line needs
> to be drawn.

Using modseq is an optimization. An optimization that makes things not work
is not something I'd like to have.

>
>
>  So, before enabling certain optimizations for Dovecot, I thought I'd ask
>> on
>> a Dovecot mailing list, about actual behavior for this server feature.
>>
>

> [snip]
>
>

> There are certainly bugs - I found several of them years ago when the code
> was brand new (here's a thread: http://markmail.org/message/
> fj74xta5z5uv4nix).  But nothing that was showstopping.  And none of those
> versions are being run anymore for all intents and purposes.
>
>
Thanks.

It's somewhat worrying that enabling CONDSTORE just once will cause the
server to always track modseq values from that point on -- causing new code
paths to execute.

But again, thanks for your data points rooted in the real world.

-- K