[Dovecot] Problem listing SPECIAL-USE flags
Timo,
we want to implement SPECIAL-USE in Thunderbird (TB), but we've run into problems. I believe you could help change that.
Here's the problem:
By default TB only lists subscribed (LSUB) folders when it accesses an IMAP server. However when I do a LSUB on recent Dovecot 2.1 code from mercurial I don't get to see the SPECIAL-USE flags. Therefore TB fails to identify which folders it should use and map to until someone tells it to list all folders (LIST) manually.
Central part of the SPECIAL-USE idea was/is to require as little user interaction as possible to automatically setup a MUA to do the right thing.
Speculating that probably most MUAs will do a LSUB by default and never will get to see SPECIAL-USE flags I think it is easier to have the server-side promote the flags.
Do you think that would be possible?
Just to verify I configured Dovecot as you would expect for SPECIAL-USE:
I've set Dovecot 2.1 to autocreate and autosubscribe SPECIAL-USE mailboxes like this:
namespace inbox { ... mailbox AltDrafts { special_use = \Drafts auto = subscribe } ... }
This is what I get:
- OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5] Dovecot 2.1 ready. t1 login alice@example.com secret t1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS FUZZY SPECIAL-USE] Logged in t2 LSUB "" "%"
- LSUB () "." "Trash"
- LSUB () "." "AltDrafts"
- LSUB () "." "AltSent"
- LSUB () "." "Archives"
- LSUB () "." "AltTrash"
- LSUB () "." "AltJunk"
- LSUB () "." "AltArchive"
- LSUB () "." "Junk" t2 OK Lsub completed. t3 LIST "" "%"
- LIST (\HasNoChildren \Sent) "." "AltSent"
- LIST (\HasChildren) "." "Archives"
- LIST (\HasNoChildren \Archive) "." "AltArchive"
- LIST (\HasNoChildren \Junk) "." "AltJunk"
- LIST (\HasNoChildren) "." "Junk"
- LIST (\HasNoChildren \Trash) "." "AltTrash"
- LIST (\HasNoChildren \Drafts) "." "AltDrafts"
- LIST (\HasNoChildren) "." "Trash"
- LIST (\HasNoChildren) "." "INBOX" t3 OK List completed. t4 logout
- BYE Logging out t4 OK Logout completed. Connection closed by foreign host.
Thanks,
p@rick
-- state of mind ()
Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
Quoting Patrick Ben Koetter p@state-of-mind.de:
we want to implement SPECIAL-USE in Thunderbird (TB), but we've run into problems. I believe you could help change that.
Here's the problem:
By default TB only lists subscribed (LSUB) folders when it accesses an IMAP server. However when I do a LSUB on recent Dovecot 2.1 code from mercurial I don't get to see the SPECIAL-USE flags. Therefore TB fails to identify which folders it should use and map to until someone tells it to list all folders (LIST) manually.
Central part of the SPECIAL-USE idea was/is to require as little user interaction as possible to automatically setup a MUA to do the right thing.
Speculating that probably most MUAs will do a LSUB by default and never will get to see SPECIAL-USE flags I think it is easier to have the server-side promote the flags.
Except you never, never, never, never, never can rely on LSUB for
mailbox attributes. It says so right in RFC 3501 [6.3.9]:
The returned untagged LSUB response MAY contain different mailbox
flags from a LIST untagged response. If this should happen, the
flags in the untagged LIST are considered more authoritative.See also RFC 5258 [3.1], which discusses the difference between the
base RFC 3501 LSUB command and the LIST-EXTENDED SUBSCRIBED option:
This option is
intended to supplement the LSUB command. Of particular note are
the mailbox attributes as returned by this option, compared with
what is returned by LSUB. With the latter, the attributes
returned may not reflect the actual attribute status on the
mailbox name, and the \NoSelect attribute has a second special
meaning (it indicates that this mailbox is not, itself,
subscribed, but that it has descendant mailboxes that are). With
the SUBSCRIBED selection option described here, the attributes are
accurate and complete, and have no special meanings. "LSUB" and
"LIST (SUBSCRIBED)" are, thus, not the same thing, and some
servers must do significant extra work to respond to "LIST
(SUBSCRIBED)". Because of this, clients SHOULD continue to use
"LSUB" unless they specifically want the additional information
offered by "LIST (SUBSCRIBED)".Even if Timo changes things on Dovecot, you still can't rely on the
flags returned from an LSUB command because other servers are not
guaranteed to act the same way.
That's just the reality of IMAP. LSUB is only useful for grabbing the
list of subscribed mailboxes (I *really* hope TB is not using flags
from the LSUB response for any meaningful purpose - that is simply
badly broken behavior).
In conclusion, there are 2 ways of handling special-use flags:
- If the server that supports SPECIAL-USE also supports LIST-EXTENDED
(RFC 5258), you are in luck. You can then use LIST-EXTENDED which can
accurately retrieve both the list of subscribed mailboxes and
special-use flags in a single command. - If the server supports SPECIAL-USE but does NOT support
LIST-EXTENDED, you MUST make a LIST call. A LSUB call is not
sufficient. It's inefficient, but that's the only way to be 100% sure
the information in the response is accurate.
michael
On 21.12.2011, at 8.24, Michael M Slusarz wrote:
That's just the reality of IMAP. LSUB is only useful for grabbing the list of subscribed mailboxes (I *really* hope TB is not using flags from the LSUB response for any meaningful purpose - that is simply badly broken behavior).
It does:
# tb-lsub-flags: # Show \Noselect flags for LSUB replies with LAYOUT=fs (e.g. mbox). # This makes Thunderbird realize they aren't selectable and show them # greyed out, instead of only later giving "not selectable" popup error.
In conclusion, there are 2 ways of handling special-use flags:
- If the server that supports SPECIAL-USE also supports LIST-EXTENDED (RFC 5258), you are in luck. You can then use LIST-EXTENDED which can accurately retrieve both the list of subscribed mailboxes and special-use flags in a single command.
- If the server supports SPECIAL-USE but does NOT support LIST-EXTENDED, you MUST make a LIST call. A LSUB call is not sufficient. It's inefficient, but that's the only way to be 100% sure the information in the response is accurate.
Agreed. I thought about adding the flags to LSUB, but then thought that other servers probably won't do it either. Note how GMail sends them with XLIST command, but there is no XLSUB command.
- Michael M Slusarz slusarz@curecanti.org:
Quoting Patrick Ben Koetter p@state-of-mind.de:
we want to implement SPECIAL-USE in Thunderbird (TB), but we've run into problems. I believe you could help change that.
Here's the problem:
By default TB only lists subscribed (LSUB) folders when it accesses an IMAP server. However when I do a LSUB on recent Dovecot 2.1 code from mercurial I don't get to see the SPECIAL-USE flags. Therefore TB fails to identify which folders it should use and map to until someone tells it to list all folders (LIST) manually.
Central part of the SPECIAL-USE idea was/is to require as little user interaction as possible to automatically setup a MUA to do the right thing.
Speculating that probably most MUAs will do a LSUB by default and never will get to see SPECIAL-USE flags I think it is easier to have the server-side promote the flags.
Except you never, never, never, never, never can rely on LSUB for mailbox attributes. It says so right in RFC 3501 [6.3.9]:
The returned untagged LSUB response MAY contain different mailbox flags from a LIST untagged response. If this should happen, the flags in the untagged LIST are considered more authoritative.See also RFC 5258 [3.1], which discusses the difference between the base RFC 3501 LSUB command and the LIST-EXTENDED SUBSCRIBED option:
This option is intended to supplement the LSUB command. Of particular note are the mailbox attributes as returned by this option, compared with what is returned by LSUB. With the latter, the attributes returned may not reflect the actual attribute status on the mailbox name, and the \NoSelect attribute has a second special meaning (it indicates that this mailbox is not, itself, subscribed, but that it has descendant mailboxes that are). With the SUBSCRIBED selection option described here, the attributes are accurate and complete, and have no special meanings. "LSUB" and "LIST (SUBSCRIBED)" are, thus, not the same thing, and some servers must do significant extra work to respond to "LIST (SUBSCRIBED)". Because of this, clients SHOULD continue to use "LSUB" unless they specifically want the additional information offered by "LIST (SUBSCRIBED)".
Thanks for the clarification. I did some reading to get a better understanding. We'll probably stick with this and may go for LIST if the server does not announce LIST-EXTENTED:
t2 LIST "" "%" RETURN (SPECIAL-USE) p@rick
- LIST (\Sent) "." "AltSent"
- LIST () "." "Archives"
- LIST (\Archive) "." "AltArchive"
- LIST (\Junk) "." "AltJunk"
- LIST () "." "Junk"
- LIST (\Trash) "." "AltTrash"
- LIST (\Drafts) "." "AltDrafts"
- LIST () "." "Trash"
- LIST () "." "INBOX" t2 OK List completed.
-- state of mind ()
Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
On 2011-12-21 2:42 PM, Patrick Ben Koetter p@state-of-mind.de wrote:
Thanks for the clarification. I did some reading to get a better understanding. We'll probably stick with this and may go for LIST if the server does not announce LIST-EXTENTED
So... out of curoisity, does dovecot support LIST-EXTENDED?
--
Best regards,
Charles
- Charles Marcus dovecot@dovecot.org:
On 2011-12-21 2:42 PM, Patrick Ben Koetter p@state-of-mind.de wrote:
Thanks for the clarification. I did some reading to get a better understanding. We'll probably stick with this and may go for LIST if the server does not announce LIST-EXTENTED
So... out of curoisity, does dovecot support LIST-EXTENDED?
Yes, it does. See the full session below:
- OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5] Dovecot 2.1 ready. 1 login alice@example.com secret 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS FUZZY SPECIAL-USE] Logged in t2 LIST "" "%" RETURN (SPECIAL-USE)
- LIST (\Sent) "." "AltSent"
- LIST () "." "Archives"
- LIST (\Archive) "." "AltArchive"
- LIST (\Junk) "." "AltJunk"
- LIST () "." "Junk"
- LIST (\Trash) "." "AltTrash"
- LIST (\Drafts) "." "AltDrafts"
- LIST () "." "Trash"
- LIST () "." "INBOX" t2 OK List completed. t3 logout
- BYE Logging out t3 OK Logout completed.
-- state of mind ()
Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
Timo,
if I read the specs correctly http://tools.ietf.org/html/rfc6154#section-5.2 says the following command should work too:
C: t3 LIST (SPECIAL-USE) "" "*"
S: * LIST (\Sent) "/" SentMail
S: * LIST (\Marked \Drafts) "/" MyDrafts
S: * LIST (\Trash) "/" Trash
S: t3 OK doneHowever if I send that command it fails:
t2 LIST (SPECIAL-USE) "" "*" t2 BAD Error in IMAP command LIST: Unknown select options
Is that a bug or was a decision on purpose?
p@rick
-- state of mind ()
Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666
Amtsgericht München Partnerschaftsregister PR 563
On 21.12.2011, at 21.56, Patrick Ben Koetter wrote:
if I read the specs correctly http://tools.ietf.org/html/rfc6154#section-5.2 says the following command should work too:
C: t3 LIST (SPECIAL-USE) "" "*"
Yes.
However if I send that command it fails:
t2 LIST (SPECIAL-USE) "" "*" t2 BAD Error in IMAP command LIST: Unknown select options
Is that a bug or was a decision on purpose?
I forgot that existed. I'll add it.
participants (4)
-
Charles Marcus
-
Michael M Slusarz
-
Patrick Ben Koetter
-
Timo Sirainen