[imaptest] Explaining errors

[Timo Sirainen]

On 23 May 2017, at 12.10, aduprat <aduprat at linagora.com> wrote:
> 
> Hi,
> 
> I'm an Apache James committer and we are curious to use imaptest in order to validate our IMAP protocol implementation.
> I'm using the nightlybuild : imaptest-20170506
> 
> 
> I follow the examples given in the /T/ /est IMAP server compliancy/ examples https://imapwiki.org/ImapTest/Examples.
> 
> And I wanted to analyse the errors reported, here are some:
> Error: user1 at james.org[11]: seq too high (16 > 15, state=APPEND): * 16 FETCH (BODY[HEADER.FIELDS (In-Reply-To From) ] "From: marcelo at carpa.ciagri.usp.br")
> Error: user1 at james.org[14]: seq too high (26 > 25, state=APPEND): * 26 FETCH (MODSEQ (6068) FLAGS (\Answered \Draft \Flagged))
> Error: user1 at james.org[14]: seq too high (20 > 16, state=APPEND): * 20 EXPUNGE

These above are saying that the message sequence number handling is broken in some way. For example in the first case imaptest has calculated that there are supposed to be 15 mails in the mailbox based on the EXISTS/EXPUNGE replies, but it's returning mail with sequence 16. Similarly the other two. Depending on why exactly these happened, they could be pretty bad if the client thinks that wrong messages were being expunged. Or it might be just that your server hasn't yet notified IMAP client that there are 16 messages (with EXISTS), but internally it already sees the 16 messages so 1:* returns the FETCH result for 16th mail.

> Error: user10 at james.org[18]: Keyword used without being in FLAGS: $Label1: * 1 FETCH (MODSEQ (8) FLAGS (\Answered \Deleted \Draft $Label1 $Label3))

When SELECTing the folder, * FLAGS & * OK [PERMANENTFLAGS] replies didn't include $Label1. These should be sent again with $Label1 included, before returning FETCH FLAGS with $Label1.

> Error: Checkpoint: Total RECENT count 6 larger than current message count 3
> Warning: Disabling \Recent flag tracking

I think there's been "* 6 RECENT" untagged reply when there were only 3 messages.

> Error: user1 at james.org[5]: Owned flag changed: \Deleted: * 7 FETCH (FLAGS (\Answered \Deleted \Recent $Label2))

Flag changes done concurrently are being lost. For example mail 7 originally had flags (\Recent $Label2). Then session A did a STORE 7 +FLAGS \Answered while simultaneously session B did STORE 1 -FLAGS \Deleted. The result was that \Answered got added, but \Deleted wasn't removed. I guess your server isn't handling +FLAGS and -FLAGS atomically, but rather as a "get flags, do changes, write flags" so there's a race condition if two sessions do it at the same time. I guess not practically a big problem since IMAP clients normally wouldn't try to do that.

> Error: user1 at james.org[5]: STORE didn't return FETCH FLAGS for seq 18 (expunged=yes): 5.75 OK STORE completed.

I think this means that session A expunged mail 18. Then it still existed session B, which did STORE 18 +FLAGS (something). Your server returned OK, but no * 18 FETCH FLAGS (something). imaptest either expects the FETCH reply or that STORE returns NO if it didn't want to do anything.

> Error: user1 at james.org[1]: SEARCH result missing seq 2 (uid 0): 1.13 OK SEARCH completed.

uid 0?.. not sure, maybe related to the first problem with sequence numbers being broken.

> Error: Keyword '$Label1' dropped, but it still had 1 references

Untagged * FLAGS reply was sent in the middle of a session, but it didn't include $Label1 even though imaptest knows there are still mails that have that flag set.

> Do you have a link explaining what are those failures ?
> Is there a report by IMAP command giving us an idea about our compliancy ?

Did you run "imaptest test=tests"? That's mainly how those compliancy numbers are done.

> Do you allow us to communicate about such tests, for example:
> - in news http://james.apache.org/#posts ?
> - in medium article ?

Fine with me.

> We also would like to have James mentioned in the servers status page https://imapwiki.org/ImapTest/ServerStatus
> How may we help you for that ?

Mainly run the imaptest test=tests and it should output the results. You can change the page yourself, but it asks for a captcha. I'll mail you the answer for that privately.