[Dovecot] Assertion failure in maildir-mail.c for gzipped maildir files in 1.1.15

[David S. Madole]

I attempted an upgrade from 1.1.7 to 1.1.15 and encountered an apparent
regression in handling gzipped maildir files.

I have tried to narrow down the conditions under which it happens and
reproduce it with as simple a case as possible. The actual mail
application that exposed this problem is Squirrelmail.

Starting off with the index files removed from the "Folder" maildir
directory:

* OK Dovecot ready.
1 login x y
1 OK Logged in.
2 select Folder
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)]
Flags permitted.
* 13 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1220278111] UIDs valid
* OK [UIDNEXT 14] Predicted next UID
2 OK [READ-WRITE] Select completed.
3 UID FETCH 3 (RFC822.SIZE INTERNALDATE)
* 3 FETCH (UID 3 RFC822.SIZE 1085 INTERNALDATE "04-Jan-2004 09:23:45 -0500")
3 OK Fetch completed.
4 UID FETCH 4 (RFC822.SIZE INTERNALDATE)
Connection closed by foreign host.

The fetch of message 3, which is not gzipped succeeds, but the fetch of
message 4, which is gzipped, fails with the following assertion in the log:

Panic: imap: user=x, Status: file maildir-mail.c: line 85
(maildir_mail_stat): assertion failed: (fd != -1)

Repeating, but first fetching only INTERNALDATE from the gzipped message
succeeds, and once that value is cached in the index files, the same
query that causes the failure before will now succeed, as it doesn't
need to read the gzipped file any more to get INTERNALDATE:

* OK Dovecot ready.
1 login x y
1 OK Logged in.
2 select Folder
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)]
Flags permitted.
* 13 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1220278111] UIDs valid
* OK [UIDNEXT 14] Predicted next UID
2 OK [READ-WRITE] Select completed.
3 UID FETCH 4 (INTERNALDATE)
* 4 FETCH (UID 4 INTERNALDATE "02-Dec-2003 10:39:18 -0500")
3 OK Fetch completed.
4 UID FETCH 4 (RFC822.SIZE INTERNALDATE)
* 4 FETCH (UID 4 RFC822.SIZE 6035 INTERNALDATE "02-Dec-2003 10:39:18 -0500")
4 OK Fetch completed.
5 logout
* BYE Logging out
5 OK Logout completed.
Connection closed by foreign host.

Now that the value is cached, the same previously failing query will
continue to execute fine, as long as the indexes remain in place:

* OK Dovecot ready.
1 login x y
1 OK Logged in.
2 select Folder
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)]
Flags permitted.
* 13 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1220278111] UIDs valid
* OK [UIDNEXT 14] Predicted next UID
2 OK [READ-WRITE] Select completed.
3 UID FETCH 4 (RFC822.SIZE INTERNALDATE)
* 4 FETCH (UID 4 RFC822.SIZE 6035 INTERNALDATE "02-Dec-2003 10:39:18 -0500")
3 OK Fetch completed.
4 logout
* BYE Logging out
4 OK Logout completed.
Connection closed by foreign host.

Removing the indexes again, and this time first fetching just
RFC822.SIZE first:

* OK Dovecot ready.
1 login x y
1 OK Logged in.
2 select Folder
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)]
Flags permitted.
* 13 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1220278111] UIDs valid
* OK [UIDNEXT 14] Predicted next UID
2 OK [READ-WRITE] Select completed.
3 UID FETCH 4 (RFC822.SIZE)
* 4 FETCH (UID 4 RFC822.SIZE 6035)
3 OK Fetch completed.
4 UID FETCH 4 (RFC822.SIZE INTERNALDATE)
Connection closed by foreign host.

In this case, the assertion fails again. Fetching RFC822.SIZE alone does
not populate the indexes in a way that then allows RFC822.SIZE and
INTERNALDATE to be fetched, as fetching INTERNALDATE alone does.

It's all confusing to me, hopefully this make some sense to others. Let
me know if there is other testing I can help with.

I'm running on Slackware 12.1 and have reproduced the same issue with
both kernel 2.6.24.4 and 2.6.27.7. The filesystem is XFS.

The content of the gzipped message does not seem to matter.

David