[Dovecot] Who's wrong, atmail or dovecot?

[Bill Cole]

At 8:51 AM -0400 3/11/08, Andy Dills wrote:
>We recently switched to atmail, as well as dovecot. I noticed in atmail
>the size of the mailboxes was always reported as 0kb.
>
>So, I did some debugging, and it boiled down to the fact that the regular
>expression used by dovecot expected UID before SIZE,

I assume you meant to say 'atmail' there and not 'dovecot'

>  but Dovecot returned
>SIZE before UID. No biggy, I changed the regex, but I was curious if there
>was a standard.

Yes, IMAP4[rev1] is an explicitly defined protocol, not a mix of 
specification and traditions (for a contrast, see NNTP.) The 
versioning is a bit weird, but all 3 RFC's specifying IMAP4 and 
IMAP4rev1 published in 1994, 1996, and 2003 agree on this issue. This 
is a goofy piece of the IMAP4 spec, but it isn't really unclear and 
even someone coding to the examples in the RFC's  would not make the 
error you are seeing in atmail.

RFC3501, like RFC2060 and RFC1730 before it, does not specify any 
ordering to the parts of a FETCH response type, and the example of a 
response given for a UID FETCH command in both RFC's has the UID as 
the last field. RFC3501 also points implementors to RFC2683, in which 
section 3.4.4 explicitly warns client authors to not rely on any 
particular order or structure in a FETCH response more restrictive 
than the specification, and points out just how variable a FETCH 
response can be. Any IMAP software that depends on an IMAP server 
answering with fields in a particular order would seem to be written 
by someone who had not bothered with a serious reading of the 
specifications of the protocol or looked at the implementation 
hints/warnings pointed to by the full specification.

>Here's the imap query that is sent:
>UID FETCH 1:* (RFC822.SIZE)
>
>Here's the diff I implemented to make it work:
>http://www.xecu.net/atmail/dovecot_sizes.diff
>
>So...is this something that is standard or something atmail needs to
>handle by making their regex more dynamic?

As Timo pointed out, a regular expression is a poor choice for 
parsing an IMAP FETCH response to a UID FETCH command. I'm not sure 
that it is formally impossible for all RE flavors and certainly not 
for that specific UID FETCH command, but the style used in the code 
you have patched, with hard-coded field names, is extremely unwise. 
The FETCH response format is rigorously defined and is parseable, but 
the right approach would be to decompose the response  structurally 
into name/value pairs rather than look for specific names in a 
particular order.

-- 
Bill Cole
bill at scconsult.com