Progress messages

[Timo Sirainen]

On 23 Apr 2016, at 01:26, Joel Roth <joelz at pobox.com> wrote:
> 
> Hi Dovecot maintainers,
> 
> I'm maintainer of Net::IMAP::Client, a perl IMAP client
> library.[1]
> 
> I've had two bug reports related to Dovecot's in-progress
> messages.[2,3]
> 
> While patches have been submitted to resolve both of these
> bugs, I would like to minimize the possibilty of future 
> breakage. Hence the following questions:
> 
> 1. Which parts of the RFC are relevant to the in-progress messages?
> 
> 2. Do you have a list of all of Dovecot's in-progress messages?
> 
> 3. If, in future, other in-progress messages may be added, 
> can you provide a standard syntax that I may use
> to strip them from the reply stream.

IMAP server is free to send any untagged replies to the client at any time. The IMAP clients should parse the wanted information from the received untagged replies and ignore the rest. It sounds like your IMAP library is doing exactly the opposite by treating any unexpected untagged replies as errors.

Here's an excerpt from the RFC:

   Certain server data MUST be recorded by the client when it is
   received; this is noted in the description of that data.  Such data
   conveys critical information which affects the interpretation of all
   subsequent commands and responses (e.g., updates reflecting the
   creation or destruction of messages).

   Other server data SHOULD be recorded for later reference; if the
   client does not need to record the data, or if recording the data has
   no obvious purpose (e.g., a SEARCH response when no SEARCH command is
   in progress), the data SHOULD be ignored.

> 2.  https://rt.cpan.org/Public/Bug/Display.html?id=84623

Strictly speaking a SEARCH command can return multiple untagged SEARCH replies, all of which should be merged together.. Luckily there are no servers actually doing it (that I'm aware of) and I'm not planning on changing Dovecot to do that either.

> 3.  https://rt.cpan.org/Public/Bug/Display.html?id=113489

This isn't enough. There are other untagged replies that can be sent. Dovecot can also send "* NO .." while it's waiting for locks.

I'm also wondering how your client will behave when receiving unsolicited FETCH replies caused by other concurrent clients. For example this can happen:

x uid fetch 1:* internaldate
* 1 FETCH (UID 1 INTERNALDATE "22-Apr-2016 20:41:08 +0300")
* 2 FETCH (UID 2 INTERNALDATE "22-Apr-2016 20:41:08 +0300")
* 3 FETCH (UID 3 INTERNALDATE "22-Apr-2016 20:41:08 +0300")
* 2 FETCH (FLAGS (\Seen))
* 4 EXISTS
x OK Fetch completed (0.001 + 0.000 secs).

Looks like Dovecot currently sends all these unsolicited responses after the requested replies, but that's not required by RFC. I'm not planning on changing it though, except that enabling NOTIFY extension can cause them to happen at any time.

Also, I assume your library doesn't try to use message sequence numbers for anything and in general doesn't try to keep track of the latest mailbox state? Because that would require tracking the EXPUNGE/EXISTS/FETCH replies, which could happen as a result of most of the IMAP commands (except EXPUNGE has restrictions).

Also a few links about writing IMAP clients:

http://imapwiki.org/ClientImplementation
http://dovecot.org/imap-client-coding-howto.html
http://dovecot.org/client-commandments.txt