[Dovecot] mdbox: Cannot create subfolder called "dbox-Mails" (2.0beta5)
I am trying out "mdbox" under Dovecot 2.0beta5.
Looking in the "mailboxes" directory under the mdbox storage root ("~/dbox" in my case), I can see that the mail folders are mapped into filesystem directories.
But Dovecot seems to put all the message list information ("dovecot.index.cache", "dovecot.index.log") for any given mail folder into a directory called "dbox-Mails"...
But this directory name exists in the same namespace as that used for mail subfolders...
So this means we cannot create a mail subfolder whose name is "dbox-Mails"!
I concede that this eventuality may be unlikely, but when designing in a green field, surely we should avoid such obvious pitfalls?
Surely we would want to make it so that these objects exist in different namespaces, so that mail folders can be created with any name?
So perhaps the directory "dbox-Mails" should use a character in its name which could not be used in mail folder names.
Or mail folder names could have a non-identity mapping into directory names such that the directory name can never be "dbox-Mails".
Regards,
Bill
On to, 2010-05-13 at 10:14 +0100, William Blunn wrote:
So this means we cannot create a mail subfolder whose name is "dbox-Mails"!
Yes. That's why it's called dbox-Mails, it's unlikely people will want to try to create it :)
I concede that this eventuality may be unlikely, but when designing in a green field, surely we should avoid such obvious pitfalls?
Surely we would want to make it so that these objects exist in different namespaces, so that mail folders can be created with any name?
Those conflict with how people often want to manage mailboxes on filesystem..
So perhaps the directory "dbox-Mails" should use a character in its name which could not be used in mail folder names.
Only such characters are control characters. Admins wouldn't be very happy with those. Maybe some filesystems wouldn't allow them either (I know OSX has some limitations).
Or mail folder names could have a non-identity mapping into directory names such that the directory name can never be "dbox-Mails".
There could be some escaping, but just to allow dbox-Mails mailbox name that seems overly complex.
The final solution in future would be to prefer using mailbox GUIDs as the directory names. This fixes some other bugs as well related to swapping one mailbox to another while it's already open in one session. But it also pretty much requires mailbox list indexes, so it's not possible yet.
Timo Sirainen wrote:
On to, 2010-05-13 at 10:14 +0100, William Blunn wrote:
So this means we cannot create a mail subfolder whose name is "dbox-Mails"!
Yes. That's why it's called dbox-Mails, it's unlikely people will want to try to create it :)
You have a kind of "Special value" which has a different meaning to normal values.
This is sort of like a Rogue value, but in this case you cannot guarantee that it will always be distinct from normal legal values.
A "Special value" such as this should be carefully chosen so as to minimise the risk of colliding with normal values.
What about people who have e-mail relating to "postfix", "exim", and "dbox" and who want to file them in folders?
And what if those people are the kind of people who have the propensity to put the name of what a thing is within its name?
They might decide to create folders thus:
"postfix-Mails" "exim-Mails" "dbox-Mails"
Oops. We just collided with the Special value.
The folder name 'dbox-Mails' is comprised of two normal words, and combined in a way which makes a relatively meaningful phrase.
As such it isn't especially unlikely, and therefore a poor choice for a Special value.
Compare and contrast an alternative possibility "zgo0kq2njs". This is the uncommon character 'z' followed by nine random alphanumeric characters, for a total of 10 characters, and as such should have equal storage complexity to the original 10 ASCII character proposal "dbox-Mails". But it does not make any word or phrase in any language I know of.
This should make a better Special value because it should be less likely to collide with any normal value.
For sysadminning it should be no problem. The sysadmin types "z" (or "zg" or perhaps "zgo") then presses TAB and completion fills in the rest. It seems the obscurer the better because it is less likely to have partial head matches with normal directory names.
One possibility would be to make the Special value for the dbox subdirectory be a configuration option.
Bill
William Blunn wrote:
The folder name 'dbox-Mails' is comprised of two normal words, and combined in a way which makes a relatively meaningful phrase.
... especially in this context!
The things we are naming are e-mail folders, i.e. folders which contain e-mails or "Mails".
So the suffix "-Mails" is actually quite likely!
Bill
On 2.7.2010, at 23.39, William Blunn wrote:
William Blunn wrote:
The folder name 'dbox-Mails' is comprised of two normal words, and combined in a way which makes a relatively meaningful phrase.
... especially in this context!
The things we are naming are e-mail folders, i.e. folders which contain e-mails or "Mails".
So the suffix "-Mails" is actually quite likely!
I doubt it. Plus there are already lots of installations who have .dovecot.sieve files in their maildirs, which makes it impossible to create a mailbox "sieve" whose parent is "dovecot". I've tried to educate them as much as possible everywhere I know of, but it's still common.
Anyway, I think the dbox-Mails is a pretty good compromise. The name needs to be informative enough for sysadmins and unlikely enough to be used by users. It's also case-sensitive in most filesystems, so dbox-mails or Dbox-Mails would be fine too.
Anyway, the long term solution is to get rid of mailbox names in filesystem entirely and just use mailbox GUIDs and a mailbox list index. For now I think dbox-Mails is fine.
Timo Sirainen wrote:
On 2.7.2010, at 23.39, William Blunn wrote:
William Blunn wrote:
The folder name 'dbox-Mails' is comprised of two normal words, and combined in a way which makes a relatively meaningful phrase.
... especially in this context!
The things we are naming are e-mail folders, i.e. folders which contain e-mails or "Mails".
So the suffix "-Mails" is actually quite likely!
I doubt it. Plus there are already lots of installations who have .dovecot.sieve files in their maildirs, which makes it impossible to create a mailbox "sieve" whose parent is "dovecot". I've tried to educate them as much as possible everywhere I know of, but it's still common.
Anyway, I think the dbox-Mails is a pretty good compromise. The name needs to be informative enough for sysadmins and unlikely enough to be used by users. It's also case-sensitive in most filesystems, so dbox-mails or Dbox-Mails would be fine too.
Anyway, the long term solution is to get rid of mailbox names in filesystem entirely and just use mailbox GUIDs and a mailbox list index. For now I think dbox-Mails is fine.
Thank-you for taking the time to consider this issue.
OK I see what you're saying.
In thinking about how I was going to think about this, I came up with the idea below, which is worth mentioning, though as you say the likely eventuality is for the problem to go away by another means.
Taking into account the additional requirement to make things easy for the sysadmin, one idea would be to make the special value be something like "DbOx-mAiLs".
That way it should still be abundantly mnemonic for the sysadmin, but the unusual casing reduces the likelyhood of collisions to negligible.
In fact the unusual casing might even serve as as clue/reminder to the sysadmin that this is not a mail folder, but rather part of the storage format, so might be more helpful to the sysadmin than "dbox-Mails".
Bill
Bill,
-----Original Message-----
Taking into account the additional requirement to make things easy for the sysadmin, one idea would be to make the special value be something like "DbOx-mAiLs".
IMO, I'd rather have to explain to users why they can't create a particular quite unlikely folder name, than deal with having strange directories (either silly caps, or some totally random string of characters) all over the place.
Either way, it sounds like Timo's not convinced that it's worth changing.
-Brad
On 07/02/2010 07:37 PM, William Blunn wrote:
They might decide to create folders thus:
"postfix-Mails" "exim-Mails" "dbox-Mails"
Oops. We just collided with the Special value.
In this case, I'll have to agree with Steve Jobs[0] and say: "Change your folders [sic] name. Not that big of a deal."
[0] http://thenextweb.com/2009/11/19/steve-jobs-small-developer-change-apps/
Since in a mailbox you will be inevitably storing mails, the -Mail part is unnecessary anyway.
Compare and contrast an alternative possibility "zgo0kq2njs". This is the uncommon character 'z' followed by nine random alphanumeric characters, for a total of 10 characters, and as such should have equal storage complexity to the original 10 ASCII character proposal "dbox-Mails". But it does not make any word or phrase in any language I know of.
This should make a better Special value because it should be less likely to collide with any normal value.
The same argument that you use to say that dbox-Mails should be allowed can be used to justify the need for a folder called "zgo0kq2njs". One just has to be more creative. :-)
And even if it's easy to type with command-completion, I doubt anyone would have a clue to the purpose of a folder named like this, unlike a folder with a descriptive name.
On 13/05/2010 10:14, William Blunn wrote:
I am trying out "mdbox" under Dovecot 2.0beta5.
Looking in the "mailboxes" directory under the mdbox storage root ("~/dbox" in my case), I can see that the mail folders are mapped into filesystem directories.
But Dovecot seems to put all the message list information ("dovecot.index.cache", "dovecot.index.log") for any given mail folder into a directory called "dbox-Mails"...
But this directory name exists in the same namespace as that used for mail subfolders...
So this means we cannot create a mail subfolder whose name is "dbox-Mails"!
Looking at some new (1 Sep 2010) documentation from Timo at http://wiki2.dovecot.org/MailLocation, it looks like in the meantime we have grown a new configuration directive "DIRNAME" which can be used in the mailbox location specification to change the metadata directory name to something other than the default "dbox-Mails".
So if site mail admins are concerned about clashes with user mail folders called "dbox-Mails" they can choose another name for the metadata directory.
OK fine so far, but then I saw the next bit.
We can also use "DIRNAME" with Maildir (if we have also specified LAYOUT=fs) to specify that {new,cur,tmp} folders should be stored in a subdirectory named by DIRNAME. This prevents clashes between user mail folder names and the {new,cur,tmp} folders. It is then up to the site admin to choose a value for DIRNAME which they think won't clash with user mail folder names.
OK, but then it occurred to me, if we can use DIRNAME with Maildir, how about LAYOUT with dbox?
How about having the ability to specify Maildir++ folder layout under dbox? For example:
# Note: Following configuration line is hypothetical mail_location = mdbox:~/mdbox:LAYOUT=maildir++
~/mdbox/mailboxes/dbox-Mails (mail folder INBOX) ~/mdbox/mailboxes/.foo/dbox-Mails (mail folder foo) ~/mdbox/mailboxes/.foo.bar/dbox-Mails (mail folder foo/bar) ~/mdbox/mailboxes/.foo.bar.baz/dbox-Mails (mail folder foo/bar/baz)
One upshot of this would be that user folder names would be in a different namespace to the DIRNAME folder.
Also for admins who prefer fewer filesystem directory levels, this would make them happy because all the mail folder levels would be at a single directory level in the filesystem.
I suppose another upshot would be that it would look more like Maildir++, which admins may already be familiar with, and might provide a smoother/easier (mental) transition to dbox for admins.
Bill
On 04/09/2010 16:37, Timo Sirainen wrote:
On 4.9.2010, at 11.40, William Blunn wrote:
OK, but then it occurred to me, if we can use DIRNAME with Maildir, how about LAYOUT with dbox?
How about having the ability to specify Maildir++ folder layout under dbox? For example:
You can.
I had an idea you might say that :-)
Bill
On 04/09/2010 16:37, Timo Sirainen wrote:
On 4.9.2010, at 11.40, William Blunn wrote:
OK, but then it occurred to me, if we can use DIRNAME with Maildir, how about LAYOUT with dbox?
How about having the ability to specify Maildir++ folder layout under dbox? For example:
You can.
I've documented this at
http://wiki2.dovecot.org/MailLocation/dbox#Directory_layout
Bill
On 04/09/2010 16:37, Timo Sirainen wrote:
On 4.9.2010, at 11.40, William Blunn wrote:
OK, but then it occurred to me, if we can use DIRNAME with Maildir, how about LAYOUT with dbox?
How about having the ability to specify Maildir++ folder layout under dbox? For example:
You can.
I tried :LAYOUT=maildir++ to work with dbox but it didn't work for me.
I have configurations:
lda_mailbox_autocreate = yes mail_location = mdbox:~/mdbox:LAYOUT=maildir++
If I try to deliver a message to a folder 'foo' using dovecot-lda
( echo 'Subject: test' ; echo ; echo 'test' ) | /usr/lib/dovecot/dovecot-lda -m foo
I get an error in the logs:
Sep 6 14:41:08 pod dovecot: lda(bill): Error: user bill: Initialization failed: Initializing mail storage from mail_location setting failed: Mailbox list driver maildir++: maildir_name not supported by this driver Sep 6 14:41:08 pod dovecot: lda(bill): Fatal: Invalid user settings. Refer to server log for more information.
I am using 2.0.1 auto 28 downloaded from xi.rename-it.nl today.
Bill
On Mon, 2010-09-06 at 16:01 +0100, William Blunn wrote:
Sep 6 14:41:08 pod dovecot: lda(bill): Error: user bill: Initialization failed: Initializing mail storage from mail_location setting failed: Mailbox list driver maildir++: maildir_name not supported by this driver
This should help: http://hg.dovecot.org/dovecot-2.0/rev/b00d3a367d79
On Mon, 2010-09-06 at 16:35 +0100, Timo Sirainen wrote:
On Mon, 2010-09-06 at 16:01 +0100, William Blunn wrote:
Sep 6 14:41:08 pod dovecot: lda(bill): Error: user bill: Initialization failed: Initializing mail storage from mail_location setting failed: Mailbox list driver maildir++: maildir_name not supported by this driver
This should help: http://hg.dovecot.org/dovecot-2.0/rev/b00d3a367d79
Oh, also add :DIRNAME=
participants (4)
-
Brad Davidson
-
Eduardo M KALINOWSKI
-
Timo Sirainen
-
William Blunn