[Dovecot] Compatibility namespaces containing a leading ~ appear to be broken in 1.1.x

[David McBride]

Greetings,

I'm building a new 1.1.x-series Dovecot installation, and I have a
problem setting up compatibility namespaces intended to handle old
client-side settings.

(See also:
http://wiki.dovecot.org/Namespaces#head-0f3ab5ecab632eb517866c3508021e0605d54d4d)

I have produced a slightly cut-down example configuration which
demonstrates the problem:

# dovecot -n
# 1.1.3: /etc/dovecot/dovecot.conf
protocols: imap imaps pop3 pop3s
ssl_cert_file(default): /etc/ssl/certs/imap.pem
ssl_cert_file(imap): /etc/ssl/certs/imap.pem
ssl_cert_file(pop3): /etc/ssl/certs/pop.pem
ssl_key_file(default): /etc/ssl/private/imap.key
ssl_key_file(imap): /etc/ssl/private/imap.key
ssl_key_file(pop3): /etc/ssl/private/pop.key
disable_plaintext_auth(default): yes
disable_plaintext_auth(imap): yes
disable_plaintext_auth(pop3): no
login_dir: /var/run/dovecot/login
login_executable(default): /usr/lib/dovecot/imap-login
login_executable(imap): /usr/lib/dovecot/imap-login
login_executable(pop3): /usr/lib/dovecot/pop3-login
login_max_processes_count: 1024
max_mail_processes: 32768
verbose_proctitle: yes
first_valid_uid: 100
mail_privileged_group: mail
mail_location: mbox:~/IMAP:INBOX=~/.email:INDEX=/data/dovecot-indexes/%1u/%u
mmap_disable: yes
mail_nfs_storage: yes
mail_nfs_index: yes
mail_executable(default): /usr/lib/dovecot/imap
mail_executable(imap): /usr/lib/dovecot/imap
mail_executable(pop3): /usr/lib/dovecot/pop3
mail_plugins(default): zlib quota imap_quota
mail_plugins(imap): zlib quota imap_quota
mail_plugins(pop3):
mail_plugin_dir(default): /usr/lib/dovecot/modules/imap
mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3
imap_client_workarounds(default): delay-newmail outlook-idle
tb-extra-mailbox-sep
imap_client_workarounds(imap): delay-newmail outlook-idle
tb-extra-mailbox-sep
imap_client_workarounds(pop3):
pop3_uidl_format(default): %08Xu%08Xv
pop3_uidl_format(imap): %08Xu%08Xv
pop3_uidl_format(pop3): %08Xv%08Xu
namespace:
  type: private
  separator: /
  inbox: yes
  list: yes
  subscriptions: yes
namespace:
  type: private
  separator: /
  prefix: Mail/
  location: mbox:~/Mail:INDEX=/data/dovecot-indexes/%1u/%u
  hidden: yes
  subscriptions: yes
namespace:
  type: private
  separator: /
  prefix: ~/Mail/
  location: mbox:~/Mail:INDEX=/data/dovecot-indexes/%1u/%u
  hidden: yes
  subscriptions: yes
auth default:
  mechanisms: gssapi gss-spnego plain
  passdb:
    driver: pam
  userdb:
    driver: passwd
plugin:
  quota: fs:Disk Quota

Given this configuration, a Thunderbird client using a folder prefix of
Mail/ is successfully able to enumerate (and access) non-INBOX folders:

5 list "" "Mail/%"
* LIST (\Noselect \HasChildren) "/" "Mail/archive"^M
* LIST (\NoInferiors \UnMarked) "/" "Mail/Trash"^M
5 OK List completed.^M
6 list "" "Mail/%/%"
* LIST (\Noselect \HasChildren) "/" "Mail/archive/"^M
* LIST (\NoInferiors \UnMarked) "/" "Mail/archive/INBOX-2008-10"^M
* LIST (\NoInferiors \UnMarked) "/" "Mail/archive/INBOX-2008-09"^M
[...]
6 OK List completed.^M
7 list "" "INBOX"
* LIST (\NoInferiors \UnMarked) "/" "INBOX"^M
7 OK List completed.^M
8 list "" "Mail/archive"
* LIST (\Noselect \HasChildren) "/" "Mail/archive"^M
8 OK List completed.^M

... Whereas a client using a folder prefix of ~/Mail/ is only able to
see its INBOX:

5 list "" "~/Mail/%"
5 OK List completed.^M
6 list "" "~/Mail/%/%"
6 OK List completed.^M
7 list "" "INBOX"
* LIST (\NoInferiors \UnMarked) "/" "INBOX"^M
7 OK List completed.^M
8 create "~/Mail/Trash"
8 NO Mailbox exists.^M

Note that even though it was unable to execute a listing of the hidden
namespace, it was clearly aware of it, and able to discover that the
'Trash' mailbox that Thunderbird automatically tries to create already
existed in the underlying filesystem.

Through experimentation, it appears that Dovecot 1.1.x doesn't correctly
handle namespaces that *begin* with the character '~'; using ~ in the
middle of a prefix, e.g. "flibble~/Mail", worked as documented, leading
me to suspect a bug in the ~-expansion handling of Dovecot.

(Dovecot 1.0.x worked as expected in this configuration.)

Unfortunately, an hg bisect run on recent Dovecot-1.1 versions didn't
expose the offending commit that introduced this problem, so this bug
(assuming it is a bug) was probably inadvertently introduced during the
1.1-development series.

Can anyone suggest where in the source tree I should start looking to
correct this behaviour?

Cheers,
David
-- 
David McBride <dwm at tastycake.net>