[Dovecot] Pine and prefix - LIST command bug?
I've been having problems with Pine 4.62 talking to Dovecot 1.0 stable. It's fine when the folder collection path is empty, but if I have "~/mail/" or "mail/" it gives access errors (I'm using the "~/mail/", "mail/" etc. hidden namespaces for backwards compatability).
It seems to me that Dovecot is not replying correctly to a "LIST" command with empty mailbox name: -
Dovecot:
xyz LIST "" ""
- LIST (\NoSelect) "/" "" xyz LIST "~/mail/" ""
- LIST (\NoSelect) "/" "" xyz LIST "mail/" ""
- LIST (\NoSelect) "/" ""
UW-IMAP;
xyz LIST "" ""
- LIST (\NoSelect) "/" "" xyz LIST "~/mail/" ""
- LIST (\NoSelect) "/" ~/ xyz LIST "mail/" ""
- LIST (\NoSelect) "/" mail/
RFC 2060 says:
An empty ("" string) mailbox name argument is a special request to return the hierarchy delimiter and the root name of the name given in the reference. The value returned as the root MAY be null if the reference is non-rooted or is null. In all cases, the hierarchy delimiter is returned. This permits a client to get the hierarchy delimiter even when no mailboxes by that name currently exist.
Mind you, it looks like at least our UW-IMAP also gets it wrong for "~/mail/". Of course, I may be misunderstanding something!
Best Wishes, Chris
--+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
On 18.3.2005, at 19:36, Chris Wakelin wrote:
An empty ("" string) mailbox name argument is a special request to return the hierarchy delimiter and the root name of the name given in the reference. The value returned as the root MAY be null if the reference is non-rooted or is null. In all cases, the hierarchy delimiter is returned. This permits a client to get the hierarchy delimiter even when no mailboxes by that name currently exist. .. Mind you, it looks like at least our UW-IMAP also gets it wrong for "~/mail/". Of course, I may be misunderstanding something!
No, after reading it through several times and testing how UW-IMAP reacts to different input, I finally understood it. The "root name" means the first part of the hierarchical mailbox name (um. is that any more understandable?). Eg. "foo/bar/baz" -> "foo/". I'll change it to return that.
On Fri, 18 Mar 2005 19:52:34 +0200 Timo Sirainen <tss@iki.fi> wrote:
On 18.3.2005, at 19:36, Chris Wakelin wrote:
An empty ("" string) mailbox name argument is a special request to return the hierarchy delimiter and the root name of the name given in the reference. The value returned as the root MAY be null if the reference is non-rooted or is null. In all cases, the hierarchy delimiter is returned. This permits a client to get the hierarchy delimiter even when no mailboxes by that name currently exist. .. Mind you, it looks like at least our UW-IMAP also gets it wrong for "~/mail/". Of course, I may be misunderstanding something!
No, after reading it through several times and testing how UW-IMAP reacts to different input, I finally understood it. The "root name" means the first part of the hierarchical mailbox name (um. is that any more understandable?). Eg. "foo/bar/baz" -> "foo/". I'll change it to return that.
Ah! That makes more sense, and in that case UW-IMAP isn't getting it wrong for "~/mail/" after all. I've tried your fix, but it doesn't seem to work. I'm still getting '* LIST (\NoSelect) "/" ""' for all the hidden namespaces. Looking at the code, my guess is that the namespace_find() call is modifying "ref" and so the strchr() never finds a match. This patch --- dovecot-1.0-stable.orig/src/imap/cmd-list.c Fri Mar 18 20:00:44 2005 +++ dovecot-1.0-stable/src/imap/cmd-list.c Mon Mar 21 11:53:26 2005 @@ -366,7 +366,8 @@ concept which probably no other client uses than Pine. Just try our best to emulate UW-IMAP behavior and hopefully we're fine. */ - ns = namespace_find(client->namespaces, &ref); + const char *ref_copy = t_strdup(ref); + ns = namespace_find(client->namespaces, &ref_copy); if (ns == NULL) { const char *empty = ""; ns = namespace_find(client->namespaces, &empty); seems to work, but I'm no expert in C (I program by imitation ;-) ), so I may have made a mistake with pointers, consts, string functions etc. Having said that, Pine is still getting confused. This time it's trying 'LIST "" ~/mail' and getting no results whereas UW-IMAP gives '* LIST (\NoSelect) "/" ~/mail'. Best Wishes, Chris --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
On 21.3.2005, at 14:09, Chris Wakelin wrote:
Having said that, Pine is still getting confused. This time it's trying 'LIST "" ~/mail' and getting no results whereas UW-IMAP gives '* LIST (\NoSelect) "/" ~/mail'.
How about if LIST "~/mail/" "" returned ~/mail/ even if the namespace is hidden? diff -u -r1.45 cmd-list.c --- cmd-list.c 18 Mar 2005 19:45:59 -0000 1.45 +++ cmd-list.c 21 Mar 2005 12:36:45 -0000 @@ -382,14 +382,13 @@ str_printfa(str, "* LIST (\\Noselect) \"%s\" ", ns->sep_str); - if (*ns->prefix != '\0' && !ns->hidden) { - /* public namespace, use it as the root name */ + if (*ns->prefix != '\0') { + /* use namespace as the root name */ imap_quote_append_string(str, ns->prefix, FALSE); } else { - /* private namespace, or empty namespace - prefix. use the mailbox name's first part - as the root. */ + /* empty namespace prefix. use the mailbox + name's first part as the root. */ const char *p = strchr(ref, ns->sep); if (p == NULL)
On Mon, 21 Mar 2005 14:38:19 +0200 Timo Sirainen <tss@iki.fi> wrote:
On 21.3.2005, at 14:09, Chris Wakelin wrote:
Having said that, Pine is still getting confused. This time it's trying 'LIST "" ~/mail' and getting no results whereas UW-IMAP gives '* LIST (\NoSelect) "/" ~/mail'.
How about if LIST "~/mail/" "" returned ~/mail/ even if the namespace is hidden?
If we're imitating UW-IMAP then LIST "~/mail/" "" should return "~/". I think you were right the first time; it's just that the "ref" variable was getting clobbered before it was partially copied.
I guess the second Pine problem is that it tries to make sure the "~/mail/" prefix exists and tries to create it if it doesn't. I'm not sure whether it has a right to expect this to work.
This opens up the whole question of how much effort should be put into making Dovecot cope with strange client behaviour. I tend to take a pragmatic view that if a client is popular, we have to try and accomodate it. I've got ~20000 users, and the idea is to replace UW-IMAP with Dovecot without them noticing!
In this respect, Outlook, Outlook Express, Thunderbird, Netscape, Mozilla, Pine, probably Apple mail, possibly kmail, Eudora etc. ought to work. The one I've been using, an old version of Simeon, doesn't work, incidentally, but I'm almost certainly the only user.
Best Wishes, Chris
--+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
On Mon, 2005-03-21 at 13:11 +0000, Chris Wakelin wrote:
On Mon, 21 Mar 2005 14:38:19 +0200 Timo Sirainen <tss@iki.fi> wrote:
On 21.3.2005, at 14:09, Chris Wakelin wrote:
Having said that, Pine is still getting confused. This time it's trying 'LIST "" ~/mail' and getting no results whereas UW-IMAP gives '* LIST (\NoSelect) "/" ~/mail'.
How about if LIST "~/mail/" "" returned ~/mail/ even if the namespace is hidden?
If we're imitating UW-IMAP then LIST "~/mail/" "" should return "~/".
Yes, but I was thinking we could do it a bit more easily by changing the behavior to be different from UW-IMAP, as long as Pine worked with it. I don't think the reply above would necessarily be against IMAP spec, because it would be correct for visible namespaces.
I guess the second Pine problem is that it tries to make sure the "~/mail/" prefix exists and tries to create it if it doesn't. I'm not sure whether it has a right to expect this to work.
It does if the first reply returned ~/. That's why I wanted it to make return ~/mail directly so I wouldn't have to create LIST matching for namespace prefixes too :)
Did you try if the patch in previous mail happened to work? If not, I guess I'll have to add that prefix-LISTing code..
Timo Sirainen wrote:
On Mon, 2005-03-21 at 13:11 +0000, Chris Wakelin wrote:
On Mon, 21 Mar 2005 14:38:19 +0200 Timo Sirainen <tss@iki.fi> wrote:
On 21.3.2005, at 14:09, Chris Wakelin wrote:
Having said that, Pine is still getting confused. This time it's trying 'LIST "" ~/mail' and getting no results whereas UW-IMAP gives '* LIST (\NoSelect) "/" ~/mail'.
How about if LIST "~/mail/" "" returned ~/mail/ even if the namespace is hidden?
If we're imitating UW-IMAP then LIST "~/mail/" "" should return "~/".
Yes, but I was thinking we could do it a bit more easily by changing the behavior to be different from UW-IMAP, as long as Pine worked with it. I don't think the reply above would necessarily be against IMAP spec, because it would be correct for visible namespaces.
I guess the second Pine problem is that it tries to make sure the "~/mail/" prefix exists and tries to create it if it doesn't. I'm not sure whether it has a right to expect this to work.
It does if the first reply returned ~/. That's why I wanted it to make return ~/mail directly so I wouldn't have to create LIST matching for namespace prefixes too :)
Did you try if the patch in previous mail happened to work? If not, I guess I'll have to add that prefix-LISTing code..
I've just tried it, and the same happens; PC-Pine (this time) is trying
LIST "" ~/mail/
and getting no matches, then when you answer yes to the question "Exit, saving changes and creating path?" it tries
CREATE ~/mail/
which presumably fails. It then disconnects with "IMAP connection broken (server response)".
It looks like Dovecot is crashing on the "CREATE" command:
dovecot: Mar 28 12:14:59 Error: IMAP(vis99003): file strfuncs.c: line 165 (p_strndup): assertion failed: (max_chars != (size_t)-1) dovecot: Mar 28 12:14:59 Error: child 28188 (imap) killed with signal 6
There's a core file for imap which gives
... (gdb) bt #0 0xff21c724 in _libc_kill () from /usr/lib/libc.so.1 #1 0xff1b5a84 in abort () from /usr/lib/libc.so.1 #2 0x5ea2c in i_internal_fatal_handler (status=490288, fmt=0xffbef238 "", args=0x5ea10) at failures.c:374
if that's any help!
Best Wishes, Chris
-- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
On Mon, 28 Mar 2005 12:20:48 +0100 Chris Wakelin <c.d.wakelin@reading.ac.uk> wrote: ...
I've just tried it, and the same happens; PC-Pine (this time) is trying
LIST "" ~/mail/
and getting no matches, then when you answer yes to the question "Exit, saving changes and creating path?" it tries
CREATE ~/mail/
which presumably fails. It then disconnects with "IMAP connection broken (server response)".
It looks like Dovecot is crashing on the "CREATE" command:
dovecot: Mar 28 12:14:59 Error: IMAP(vis99003): file strfuncs.c: line 165 (p_strndup): assertion failed: (max_chars != (size_t)-1) dovecot: Mar 28 12:14:59 Error: child 28188 (imap) killed with signal 6
I've found the problem, I think :- gdiff -ruN cmd-create.c.orig cmd-create.c --- cmd-create.c.orig Mon Jan 31 16:37:54 2005 +++ cmd-create.c Wed Mar 30 18:26:56 2005 @@ -20,6 +20,13 @@ return TRUE; len = strlen(mailbox); + + /* If supplied mailbox is namespace prefix then len=0; exit if so */ + if (len == 0) { + client_send_tagline(client, "OK if it makes you happy."); + return TRUE; + } + if (mailbox[len-1] != mail_storage_get_hierarchy_sep(storage)) directory = FALSE; else { In the "len = strlen(mailbox);", len will be 0 if the supplied mailbox - after "client_find_storage(client, &mailbox)" - was a namespace prefix. That causes "mailbox = t_strndup(mailbox, len-1)" to fail. What I don't understand is why CREATE "#maildir/" seems not to crash, (where "#maildir/" is another namespace prefix) when CREATE "mail/" does, though! It makes Pine moderately happy at last. It might still be better to respond as it expects to the LIST "" ~/mail/ command. Best Wishes, Chris --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
On Wed, 30 Mar 2005 18:40:24 +0100 (GMT Daylight Time) Chris Wakelin <c.d.wakelin@reading.ac.uk> wrote: ...
It makes Pine moderately happy at last. It might still be better to respond as it expects to the LIST "" ~/mail/ command.
Best Wishes, Chris
It seems to me that Outlook Express 6 (on XP SP2 plus, I think, all recommended patches) is behaving in a similar way to Pine, trying 'LIST "" "~/mail"' followed by 'CREATE "" "~/mail/"' I've tried to implement what may work for 'LIST "" "<namespace>"' :- --- dovecot-1.0-stable.orig/src/imap/cmd-list.c Fri Mar 18 20:00:44 2005 +++ dovecot-1.0-stable/src/imap/cmd-list.c Thu Mar 31 16:16:38 2005 @@ -107,7 +107,8 @@ name_str = t_str_new(256); while ((list = mail_storage_mailbox_list_next(ctx->list_ctx)) != NULL) { str_truncate(name_str, 0); - str_append(name_str, ctx->ns->prefix); + if (!(ctx->ns->hidden && ctx->lsub)) + str_append(name_str, ctx->ns->prefix); str_append(name_str, list->name); if (ctx->ns->sep != ctx->ns->real_sep) { @@ -366,7 +367,8 @@ concept which probably no other client uses than Pine. Just try our best to emulate UW-IMAP behavior and hopefully we're fine. */ - ns = namespace_find(client->namespaces, &ref); + const char *ref_copy = t_strdup(ref); + ns = namespace_find(client->namespaces, &ref_copy); if (ns == NULL) { const char *empty = ""; ns = namespace_find(client->namespaces, &empty); @@ -399,6 +401,28 @@ } client_send_tagline(client, "OK List completed."); } else { + + if (*ref == '\0' && !lsub) { + /* LIST "" "something" */ + const char *mask_copy = t_strdup(mask); + ns = namespace_find(client->namespaces, &mask_copy); + if (ns == NULL) { + const char *empty = ""; + ns = namespace_find(client->namespaces, &empty); + } + if (*ns->prefix != '\0' && *mask_copy == '\0') { + /* namespace, return it */ + string_t *str = t_str_new(64); + str_printfa(str, "* LIST (\\Noselect) \"%s\" ", + ns->sep_str); + imap_quote_append_string(str, ns->prefix, + FALSE); + client_send_line(client, str_c(str)); + client_send_tagline(client, "OK List completed."); + return TRUE; + } + } + ctx = p_new(client->cmd_pool, struct cmd_list_context, 1); ctx->ref = ref; ctx->mask = mask; The first two chunks are my fix for the fact that our .mailboxlist files have "~/mail/somefolder" in them (probably not generally needed?) plus your first suggested fix with my amendment for 'LIST "~/mail/" ""' not returning "~/". The last hunk attempts to return the namespace for any 'LIST "" "<namespace>"' command. It seems to work for me for both PC-Pine and OE6. I'm surprised nobody else has had this problem. Is no-one else using folder prefixes together with hidden namespaces in Dovecot-1.0? Best Wishes, Chris --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-- Christopher Wakelin, c.d.wakelin@reading.ac.uk IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
participants (2)
-
Chris Wakelin
-
Timo Sirainen