IMAP4rev2: extended LIST drops \Sent, \Drafts, \Trash and \Junk unless the client sends the RFC 6154 return option that IMAP4rev2 does not define
SYSTEM INFORMATION
Dovecot version: 2.4.4 (8b687aa65c), Pigeonhole 2.4.4 (3beb331a) OS: Linux 6.12.76-linuxkit aarch64, Debian 13.5 (official dovecot/dovecot:2.4.4 container image, digest sha256:723e3392fe16c6fad8ddc605ea767cc01b4bad 9cd9f13eb1dbac15e79c89b2d4) Host: Docker 29.6.2 (linux/arm64) on macOS 26, Apple silicon CPU architecture: aarch64 (arm64) Filesystem: Docker named volume (overlay2) inside the Docker Desktop VM. Not NFS. Feature: experimental IMAP4rev2 (imap4rev2_enable, mail_utf8_extensions) Client: our own IMAP client library (Rust), but reproduced below with openssl s_client so that no client code is involved
The configuration is the image's own /etc/dovecot/dovecot.conf, unmodified: nothing of ours is in it. Rather than paste 200 lines of your own defaults, "doveconf -n" for it is
docker run --rm --entrypoint doveconf dovecot/dovecot:2.4.4 -n
and we will gladly send the output if you would prefer it inline. The three settings that bear on this report are imap4rev2_enable = yes, mail_utf8_extensions = yes and mailbox_list_utf8 = yes, all three of which the image sets itself.
SUMMARY
In an IMAP4rev2 session, an extended LIST (one carrying any RETURN (...) clause) omits the special-use mailbox attributes unless the client also sends RETURN (SPECIAL-USE). IMAP4rev2 folded those attributes into the base protocol and defines no SPECIAL-USE return option, so a client that implements only RFC 9051 has no defined way to ask for them, and therefore cannot obtain them from any extended LIST at all.
The practical effect for such a client is that every folder loses its role the moment the client asks for anything else -- in our case RETURN (STATUS (UNSEEN)), to fetch unread counts in the same round trip. It cannot find the \Sent folder to place a copy of a sent message in, cannot find \Trash to move a deleted message to, and renders the mailbox list without role icons or ordering.
We are not asking for a workaround: we already send RETURN (SPECIAL-USE) on both dialects, since Dovecot advertises the SPECIAL-USE capability in a rev2 session too. This report is about what a strictly RFC 9051 client is able to do.
That decision in our client is here, in case it is of interest -- the comment above it records what we found on your server:
https://github.com/allodia-eu/email-calendar-sync-engine/blob/fa85b881211329...
REPRODUCTION
No configuration is required. dovecot/dovecot:2.4.4 ships imap4rev2_enable = yes and mail_utf8_extensions = yes in its own /etc/dovecot/dovecot.conf, and its "@mailbox_defaults = english" creates Drafts/Junk/Sent/Trash with special-use attributes.
docker run -d --name dovecot-stock -p 127.0.0.1:11999:31993
-e USER_PASSWORD=stock-test-pw
dovecot/dovecot:2.4.4
{ printf 'a1 LOGIN alice@example.com stock-test-pw\r\n'; sleep 2;
printf 'a2 ENABLE IMAP4rev2\r\n'; sleep 1;
printf 'a3 LIST "" "*"\r\n'; sleep 1;
printf 'a4 LIST "" "*" RETURN (STATUS (UNSEEN))\r\n'; sleep 1;
printf 'a5 LIST "" "*" RETURN (SPECIAL-USE STATUS (UNSEEN))\r\n';
sleep 1;
printf 'a6 LOGOUT\r\n'; sleep 1; }
| openssl s_client -connect 127.0.0.1:11999 -quiet
TRANSCRIPT
Verbatim; the greeting and the CAPABILITY lines are elided.
S: * ENABLED IMAP4REV2 S: a2 OK Enabled (0.001 + 0.000 secs).
C: a3 LIST "" "*" S: * LIST (\HasNoChildren \Drafts) "/" Drafts S: * LIST (\HasNoChildren \Junk) "/" Junk S: * LIST (\HasNoChildren \Sent) "/" Sent S: * LIST (\HasNoChildren \Trash) "/" Trash S: * LIST (\HasNoChildren) "/" INBOX S: a3 OK List completed (0.001 + 0.000 secs).
--> correct: the role attributes are present
C: a4 LIST "" "*" RETURN (STATUS (UNSEEN)) S: * LIST () "/" Drafts S: * STATUS Drafts (UNSEEN 0) S: * LIST () "/" Junk S: * STATUS Junk (UNSEEN 0) S: * LIST () "/" Sent S: * STATUS Sent (UNSEEN 0) S: * LIST () "/" Trash S: * STATUS Trash (UNSEEN 0) S: * LIST () "/" INBOX S: * STATUS INBOX (UNSEEN 0) S: a4 OK List completed (0.034 + 0.000 + 0.033 secs).
--> the problem: every role attribute is gone, and IMAP4rev2
gives the client no option to ask for them back
C: a5 LIST "" "*" RETURN (SPECIAL-USE STATUS (UNSEEN)) S: * LIST (\Trash) "/" Trash S: * STATUS Trash (UNSEEN 0) S: * LIST (\Sent) "/" Sent S: * STATUS Sent (UNSEEN 0) S: * LIST (\Junk) "/" Junk S: * STATUS Junk (UNSEEN 0) S: * LIST (\Drafts) "/" Drafts S: * STATUS Drafts (UNSEEN 0) S: * LIST () "/" INBOX S: * STATUS INBOX (UNSEEN 0) S: a5 OK List completed (0.002 + 0.000 + 0.001 secs).
--> present again, but only via the RFC 6154 return option,
which RFC 9051 does not define
WHY WE BELIEVE a4 IS NOT CORRECT FOR IMAP4rev2
RFC 9051 section 7.3.1 makes the special-use attributes ordinary LIST mailbox attributes:
"In addition to the base mailbox name attributes defined above, an IMAP server MAY also include any or all of the following attributes that denote 'role' (or 'special-use') of a mailbox. These attributes are included along with base attributes defined above."
Appendix E item 2 lists "the list of new mailbox attributes from SPECIAL-USE [RFC6154]" among what IMAP4rev2 folded in.
RFC 9051 section 6.3.9.2 defines the return options for IMAP4rev2 as exactly SUBSCRIBED, CHILDREN and STATUS. There is no SPECIAL-USE return option. Nor is SPECIAL-USE among the selection options in section 6.3.9.1, which are SUBSCRIBED, REMOTE and RECURSIVEMATCH; it survives in RFC 9051 only as an unexplained
C: A002 LIST (SPECIAL-USE) "" "*"
in the NAMESPACE examples in section 6.3.10, which may itself be worth an erratum.
Section 6.3.9 says "a client MUST NOT send an option for which the server has not advertised support", so a strictly rev2 client will not invent one.
Taken together, an RFC 9051-only client can obtain the role attributes on a basic LIST but never on an extended one, which means it cannot combine roles with RETURN (STATUS ...) -- the single round trip that RFC 5819 exists to provide. Since RFC 9051 section 6.3.9 also states that return options "MUST NOT cause the server to report information about additional mailbox names", but says nothing about removing base attributes, we read a4 as a case where an attribute defined as base data is being withheld.
Note the contrast with \HasNoChildren in the same transcript: it also disappears in a4, but CHILDREN is a return option in RFC 9051, so a rev2 client can ask for it back. For special-use there is no equivalent.
WHAT WE BELIEVE CORRECT BEHAVIOUR WOULD BE
In a session where IMAP4rev2 has been ENABLEd, include the special-use attributes in extended LIST responses without requiring RETURN (SPECIAL-USE) -- that is, treat them as the base mailbox attributes that RFC 9051 section 7.3.1 defines them to be. The RFC 6154 return option would remain accepted for IMAP4rev1 sessions.
NOTE ON IMAP4rev1: WE BELIEVE THAT BEHAVIOUR IS CORRECT
The same server on an IMAP4rev1 session behaves identically, and there we read it as compliant. RFC 6154 section 2 says "The extended LIST command MAY return SPECIAL-USE attributes even if the client does not specify the return option", and MAY is not MUST. This report concerns only IMAP4rev2 sessions, where the way to ask was removed along with the need to.
CONTACT
Happy to test a patch. The reproduction is a single "docker run", and we keep both an IMAP4rev1 and an IMAP4rev2 Dovecot fixture in our CI, so a fix can be regression-tested on both dialects.
participants (2)
-
Aki Tuomi
-
dennis@dennisameling.com