Howdy folks,
we're using Dovecot 2.3.19 in combination with vmailmgr as the virtual user management backend (please don't judge, there are plans to move away from it for various reasons, but as of today it's there), using the wrapper from https://wiki.dovecot.org/HowTo/VMailMgr - and we're facing an interesting problem with colons in home directories. I think this problem is not specific to vmailmgr but happens in any scenario that includes ":" in directory names.
tl;dr:
The checkvpw-dovecot-wrapper from the Dovecot wiki converts ":" in directory names to "::" which works fine with general IMAP use, but at least the autoexpunge plugin and ManageSieve don't handle this escaping and thus break on those users.
When patching the ":"->"::" conversion out of the checkvpw-dovecot-wrapper, Dovecot itself bails on it, interpreting it as a sub-setting.
The latter is quite surprising because a HOME/mail_home directory is supposed to be exactly one directory (AFAIK), with no sub-settings like e.g. mail_location, so I did not expect it to be "interpreted" in any way.
Short story long:
When you set up a user with a name containing dots, vmailmgr converts them to colons, pretty much like qmail does (qmail maps the username "xy.z" to a file named ".qmail-xy:z"; vmailmgr creates the directory "./users/xy:z" for that username). The wrapper from the dovecot wiki escapes ":" into "::" before exporting the user's directory as the HOME environment variable:
/* escape possible ':' characters before finally exporting */
escape_colon(buffer, MAXLEN);
setenv("HOME", buffer, 1);The result looks like that:
[root@stardust ~]# doveadm auth login xy.z@coltest.uber.space supersecret passdb: xy.z@coltest.uber.space auth succeeded extra fields: user=xy.z@coltest.uber.space
userdb extra fields: xy.z@coltest.uber.space home=/home/coltest/./users/xy::z <---------- notice the double colon here uid=1009 gid=1010 auth_mech=PLAIN
So far, so good: Working with IMAP generally works fine, but there's a catch: Not every module seems prepared for this "::" syntax. The first notable exception is the autoexpunge plugin which simply bails on it:
Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock: file_create_locked(/home/coltest/./users/xy::z/dovecot.autoexpunge.lock) failed: safe_mkstemp(/home/coltest/./users/xy::z/dovecot.autoexpunge.lock) failed: No such file or directory
The second notable exception is ManageSieve which takes the escaped directory name "as is", leading to a situation where the Maildir is in "xy:z" while the Sieve stuff is being put in "xy::z" where it effectively is never picked up during delivery (as the stuff is in the wrong location and the .dovecot.sieve symlink is - silently - not created):
[coltest@stardust ~]$ ls -la users/* users/xy:z: total 28 drwx------. 9 coltest coltest 4096 Jun 16 11:39 . drwxr-xr-x. 4 coltest coltest 31 Jun 16 11:40 .. drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 cur -rw-------. 1 coltest coltest 376 Jun 16 11:40 dovecot.index.log -rw-------. 1 coltest coltest 2676 Jun 16 11:39 dovecot.list.index.log -rw-------. 1 coltest coltest 96 Jun 16 11:39 dovecot.mailbox.log -rw-------. 1 coltest coltest 51 Jun 16 11:39 dovecot-uidlist -rw-------. 1 coltest coltest 8 Jun 16 11:39 dovecot-uidvalidity -r--r--r--. 1 coltest coltest 0 Jun 16 11:39 dovecot-uidvalidity.62aafa70 drwx------. 5 coltest coltest 108 Jun 16 11:39 .Drafts drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 new drwx------. 5 coltest coltest 108 Jun 16 11:39 .Sent drwx------. 5 coltest coltest 108 Jun 16 11:39 .Spam -rw-------. 1 coltest coltest 28 Jun 16 11:39 subscriptions drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 tmp drwx------. 5 coltest coltest 108 Jun 16 11:39 .Trash
users/xy::z: total 0 drwx------. 3 coltest coltest 19 Jun 16 11:40 . drwxr-xr-x. 4 coltest coltest 31 Jun 16 11:40 .. drwx------. 3 coltest coltest 33 Jun 16 11:40 sieve
So my initial thought was: Why is the ":" even escaped to a "::" in the first place? It shouldn't be necessary according to this clean workaround that I found in a posting from 2009 with Timo advising for this type of handling:
Hmm. Although now that I looked at your code, you seem to be using only HOME. And using ~/ in mail_location should work around any ":" issues. So what exactly is the problem you're seeing?
https://www.dovecot.org/list/dovecot/2009-June/040590.html
(In the same thread, Timo _also_ introduced the "::" handling in Dovecot.)
This is exactly what we're doing at delivery time: Setting mail_home to the real pathname including a _single_ colon (not an escaped double colon) and then pointing mail_location to "~/":
/usr/libexec/dovecot/dovecot-lda -a '$RECIPIENT' -f '$SENDER' -o mail_home='$MAILDIR' -o mail_location='maildir:~/'
This works well.
So my assumption was: If the checkvpw-dovecot-wrapper returns the HOME directory of a virtual user; where's even the need to escape a single colon to a double colon? So I gave it a try and removed that one line of escaping and recompiled the wrapper. Result:
[root@stardust ~]# doveadm auth login xy.z@coltest.uber.space supersecret passdb: xy.z@coltest.uber.space auth succeeded extra fields: user=xy.z@coltest.uber.space
userdb extra fields: xy.z@coltest.uber.space home=/home/coltest/./users/xy:z <---------- single colon here now uid=1009 gid=1010 auth_mech=PLAIN
Using a real IMAP login, though, this lead to the "interesting" situation that the login itself worked correctly, but _then_ immediately an "Internal error occurred":
[root@stardust ~]# nc localhost 143
- OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SPECIAL-USE LITERAL+ STARTTLS AUTH=PLAIN] Dovecot ready. 0 LOGIN xy.z@coltest.uber.space supersecret 0 OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY PREVIEW STATUS=SIZE SAVEDATE SPECIAL-USE LITERAL+ NOTIFY SPECIAL-USE] Logged in
- BYE Internal error occurred. Refer to server log for more information.
This is the according log:
Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37 imap-login: Info: Login: user=xy.z@coltest.uber.space, method=PLAIN, rip=::1, lip=::1, mpid=22889, secured Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37 imap(xy.z@coltest.uber.space)<22889><utbow43hgJoAAAAAAAAAAAAAAAAAAAAB>: Error: Namespace '': Unknown setting: z Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37 imap(xy.z@coltest.uber.space)<22889><utbow43hgJoAAAAAAAAAAAAAAAAAAAAB>: Info: Disconnected: Namespace '': Unknown setting: z in=0 out=428 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0
So it looks like Dovecot tries to do some interpretation on the HOME value, splitting it at the ":" and interpret the "z" as some kind of sub-setting - which a HOME variable (in contrast to, e.g. mail_location) is not supposed to have.
Here I'm a bit stuck. My idea is to work around this by eliminating the ":" from directory names altogether by passing an explicit directory name to vadduser:
[coltest@stardust ~]$ echo supersecret | vadduser --directory=./users/ab.c ab.c vadduser: user 'ab.c' successfully added [coltest@stardust ~]$ ls -l users total 0 drwx------. 5 coltest coltest 39 Jun 16 10:26 ab.c drwx------. 5 coltest coltest 39 Jun 16 09:31 xy:z
But still, it would be nicer if Dovecot cleanly supported the ":" in directory names, as it seems to _want_ to support it (by introducing the "::" escaping) but then only does it halfway.
Could this be fixed in whatever direction, or is there any other advice how we should handle this?
All the best, Jonas