[Dovecot] Configuration file changes in CVS
OK, it's there now. Not much tested yet, so there's probably some bugs. I didn't yet add support for templates, but I probably will later. The new format is like in the two example config files I sent before:
http://dovecot.procontrol.fi/list/dovecot/2003-July/001832.html
dovecot-example.conf contains the simple configuration now. New code also tries to be backwards compatible with most settings.
New code will also try to check that the configuration is valid after SIGHUP before actually using it, so the server won't die anymore if you made a typo somewhere. It doesn't check all the settings though.
Namespace configuration comes probably next. Something like:
namespace { prefix = INBOX type = private separator = / location = mbox:/var/mail/%u }
namespace { prefix = type = private separator = . location = maildir:~/ }
namespace { prefix = lists/ type = public separator = / location = mbox:/var/mail/lists/archives }
Timo Sirainen wrote:
OK, it's there now. Not much tested yet, so there's probably some bugs. I didn't yet add support for templates, but I probably will later. The new format is like in the two example config files I sent before:
http://dovecot.procontrol.fi/list/dovecot/2003-July/001832.html
dovecot-example.conf contains the simple configuration now. New code also tries to be backwards compatible with most settings.
New code will also try to check that the configuration is valid after SIGHUP before actually using it, so the server won't die anymore if you made a typo somewhere. It doesn't check all the settings though.
Namespace configuration comes probably next. Something like:
namespace { prefix = INBOX type = private separator = / location = mbox:/var/mail/%u }
namespace { prefix = type = private separator = . location = maildir:~/ }
namespace { prefix = lists/ type = public separator = / location = mbox:/var/mail/lists/archives }
does this means that all namespace for all users or...?
-- Levente "Si vis pacem para bellum!"
On Thu, 2003-07-10 at 10:11, Farkas Levente wrote:
does this means that all namespace for all users or...?
Yes. I don't know if there's any real use for user-specific namespaces.
Actually I'm not really sure how things like INBOX should actually be defined. Or some other single mailbox file in the top level. INBOX shouldn't be shown as a separate namespace, but you might want it to have different settings.
Or what if you want Maildir INBOX and mbox others? Maildir would support subfolders, so you'd have to include it in the namespace reply at least if it's hierarchy separator is different than with the mboxes.
This is getting dirty :)
--On 10 July 2003 12:54 +0300 Timo Sirainen tss@iki.fi wrote:
Actually I'm not really sure how things like INBOX should actually be defined. Or some other single mailbox file in the top level. INBOX shouldn't be shown as a separate namespace, but you might want it to have different settings.
Perhaps there could be one level of 'folder' containers allowed inside 'namespace' in the config...
namespace { prefix = type = private separator = . location = maildir:~/ folder { name = INBOX location = mbox:/var/mail/%u separator = / } }
I think it may be something of a problem to expose different separators for different parts of the hierarchy... any clients which don't understand 'namespace' won't have a clue, and I suspect that some clients which do understand 'namespace' might not cope with different folder separators in different namespaces. So I think that 'separator' in namespace/folder config should refer to 'internal' namespace, and translate for the user. e.g. like <http://asg.web.cmu.edu/cyrus/download/imapd/altnamespace.html#unixhier sep>
What does INBOX.foo do? What about INBOX/foo? Maybe there should be allow_subs = (yes|no) for folders (to set whether subfolders should be created with that 'style' or whether they should be created in the parent namespace).
If namespace configuration is being looked at now, it would probably make sense to ensure that it will be flexible enough to cope with as many future requirements as possible without change. I'm thinking of post-1.0 possiblities here, i.e. ACLs...I think that it might be worth examining any possible namespace config to see whether it's capable of implementing ACLs without big changes to the design.
Besides 'system' shared folders with ACLs, Cyrus has 'user' shared folders which it places in an 'other users' namespace (normally user.*)
- this is *really* useful... Clients which know how to deal with ACLs don't require admin assistance to share a folder. e.g. login to an ACL-capable server with Mulberry (anonymous login to imap.cyrusoft.com will give some idea, though you can't change anything with that login), right-click a mail folder and choose 'Details' and you get an extra "access-control" tab with a GUI for adding users.
This is getting dirty :)
You said it (-:
On Thursday, Jul 10, 2003, at 15:06 Europe/Helsinki, Stuart Henderson wrote:
Perhaps there could be one level of 'folder' containers allowed inside 'namespace' in the config...
namespace { prefix = type = private separator = . location = maildir:~/ folder { name = INBOX location = mbox:/var/mail/%u separator = / } }
Yes, it has to be done something like this. namespace { .. } would map into one real namespace reported by NAMESPACE command.
I think it may be something of a problem to expose different separators for different parts of the hierarchy... any clients which don't understand 'namespace' won't have a clue, and I suspect that some clients which do understand 'namespace' might not cope with different folder separators in different namespaces.
Possibly. It may be required however if some namespace contains mailboxes containing the default separator, and vice versa. And separators can't be escaped.
Anyway, I think I won't allow using multiple different separators inside one namespace. That may be allowed by IMAP RFC, maybe even the NAMESPACE RFC but it's clearly against how it was thought to be used.
So I think that 'separator' in namespace/folder config should refer to 'internal' namespace, and translate for the user.
There's no point in specifying internal separator in most cases. Only possibly useful case would be if you wanted Maildir++ directories to use something else than '.', but that wouldn't be Maildir++ anymore and if you really wanted that, you could just recompile with some #define changed..
What does INBOX.foo do? What about INBOX/foo? Maybe there should be allow_subs = (yes|no) for folders (to set whether subfolders should be created with that 'style' or whether they should be created in the parent namespace).
I think it's better to always create them under the child mailbox. It's cleaner that way and less confusing. So if you had mbox INBOX while rest was Maildir, you'd have the INBOX marked \NoInferiors tag. Although I'm not sure if it should be done when there's another namespace that begins with INBOX.. Probably shouldn't, especially because not all clients understand namespaces. That makes it a bit more complicated..
If namespace configuration is being looked at now, it would probably make sense to ensure that it will be flexible enough to cope with as many future requirements as possible without change. I'm thinking of post-1.0 possiblities here, i.e. ACLs...I think that it might be worth examining any possible namespace config to see whether it's capable of implementing ACLs without big changes to the design.
Yes.. But I can't really think of what problems the above configuration would have after adding ACLs. You could just add some acl = <value> entry there.
Besides 'system' shared folders with ACLs, Cyrus has 'user' shared folders which it places in an 'other users' namespace (normally user.*) - this is *really* useful...
Yes, this is a separate namespace type as well. I just didn't mention it yet since it's a whole different problem which is most likely post-1.0 feature.
Timo Sirainen tss@iki.fi writes:
Or what if you want Maildir INBOX and mbox others? Maildir would support subfolders, so you'd have to include it in the namespace reply at least if it's hierarchy separator is different than with the mboxes.
No-one's supposed to use mbox. Optimize as much as you want, it just doesn't scale at all.
-- Matthias Andree
On Thu, 10 Jul 2003, Matthias Andree wrote:
Timo Sirainen tss@iki.fi writes:
Or what if you want Maildir INBOX and mbox others? Maildir would support subfolders, so you'd have to include it in the namespace reply at least if it's hierarchy separator is different than with the mboxes.
No-one's supposed to use mbox. Optimize as much as you want, it just doesn't scale at all.
Converting from mbox to Maildir is easy. Everyone should be encouraged to do it. [My 2c worth]
-- Charlie
On Thu, 2003-07-10 at 16:20, Matthias Andree wrote:
Or what if you want Maildir INBOX and mbox others? Maildir would support subfolders, so you'd have to include it in the namespace reply at least if it's hierarchy separator is different than with the mboxes.
No-one's supposed to use mbox. Optimize as much as you want, it just doesn't scale at all.
mbox is fine as long as you don't want to delete messages. It's perfect for read-only mailing list archives.
On Thu, Jul 10, 2003 at 05:52:08PM +0300, Timo Sirainen wrote:
On Thu, 2003-07-10 at 16:20, Matthias Andree wrote:
Or what if you want Maildir INBOX and mbox others? Maildir would support subfolders, so you'd have to include it in the namespace reply at least if it's hierarchy separator is different than with the mboxes.
No-one's supposed to use mbox. Optimize as much as you want, it just doesn't scale at all.
mbox is fine as long as you don't want to delete messages. It's perfect for read-only mailing list archives.
Indeed. mbox is fine for some people and some things, and in fact for archived mboxes Maildir can just be a big waste.
It is nice to be open to well-established storage formats. Bear in mind that there are people who refuse to use Maildir formats too.
On this whole storage/namespace/search-path thing-- wouldn't it be nice if various imap/pop server implementations could settle on some standard ways of dealing with things? Certainly not the same techniques, but the same effects and terms and storage layouts -- making it possible to switch between various implementations. There always seem to be confusions and incompatibilities regarding various implementation details:
Namespace: even how to *talk* about namespace across various implementations can be confusing, let alone the fact that namespace conventions do not necessarily translate into file naming conventions (which is proper- but I'm just saying it adds to the confusion when discussing it).
Filesystem layout: for example a big bugaboo is that once you choose a Maildir as your primary mailbox (in a Maildir++ -like world), you're locked in there. Plenty of users already have their own idea of where they'd like their folders to be, and in what format (some maildir, some mbox), and it's just not reasonable to expect them to give that up. It ought to be possible to easily mix and match collections of Maildir, Maildir++, and mbox formats throughout a users' home space (and elsewhere, if allowed). Over in binciland there is the notion of an IMAPdir which addresses some of this although I don't think it's perfect. (I believe Andreas is here too..)
Implementation of quotas, shared folders, bulletins, etc.
Anyway-- a little cross-pollenization of ideas could help here.
mm
On Thu, 10 Jul 2003, Mark E. Mallett wrote:
On Thu, Jul 10, 2003 at 05:52:08PM +0300, Timo Sirainen wrote:
mbox is fine as long as you don't want to delete messages. It's perfect for read-only mailing list archives.
Indeed. mbox is fine for some people and some things, and in fact for archived mboxes Maildir can just be a big waste.
It is nice to be open to well-established storage formats. Bear in mind that there are people who refuse to use Maildir formats too.
Sure, but that doesn't mean that such people *must* be catered for by every imap daemon. Trying to keep everybody happy pretty much guarantees that nobody will be completely happy.
Back to the topic, I'd rather there wasn't a configuration file at all. I've got very used to Dan Berstein/Bruce Guenter software where environment variables and command line arguments are the only configuration. I'm only using the "imap" program from dovecot(*), and that follows the same convention (in standard usage, its environment variables are set by the dovecot frontends).
(*)I'm using tcpserver for network connection control, modified stunnel for TLS, and imapfront for authentication/uid switching.
-- Charlie
On Thu, 10 Jul 2003 14:31:44 -0400 (EDT) Charlie Brady charlieb-dovecot@e-smith.com wrote:
Back to the topic, I'd rather there wasn't a configuration file at all. I've got very used to Dan Berstein/Bruce Guenter software where environment variables and command line arguments are the only configuration. I'm only using the "imap" program from dovecot(*), and that follows the same convention (in standard usage, its environment variables are set by the dovecot frontends).
When replacing a configuration file with command-line options and/or environment variables, you still end up with configuration files. The difference is in syntax; with environment variables and CLI options, your "configuration files" will take the form of executable scripts, most likely in either shell or Perl syntax.
It's also nastier to represent complex structures in single-line expressions, though I don't believe dovecot currently has anything so complex.
On Thu, Jul 10, 2003 at 02:37:06PM -0400, David B Harris wrote:
When replacing a configuration file with command-line options and/or environment variables, you still end up with configuration files. The difference is in syntax; with environment variables and CLI options, your "configuration files" will take the form of executable scripts, most likely in either shell or Perl syntax.
Good point. Instead of a configuration file, you have configuration logic. Although often that's more friendly to hackers :-)
mm
On Thu, Jul 10, 2003 at 02:37:06PM -0400, David B Harris wrote:
When replacing a configuration file with command-line options and/or environment variables, you still end up with configuration files. The difference is in syntax; with environment variables and CLI options, your "configuration files" will take the form of executable scripts, most likely in either shell or Perl syntax.
Here you have well understood syntax, and very well tested parsing code.
-- Charlie
On Thu, 2003-07-10 at 21:31, Charlie Brady wrote:
Back to the topic, I'd rather there wasn't a configuration file at all. I've got very used to Dan Berstein/Bruce Guenter software where environment variables and command line arguments are the only configuration. I'm only using the "imap" program from dovecot(*), and that follows the same convention (in standard usage, its environment variables are set by the dovecot frontends).
I'll continue passing the configuration in environment for imap/pop3. Something like NAMESPACE_1=a:b:c NAMESPACE_2=b:c:d or whatever.
On Thu, Jul 10, 2003 at 02:31:44PM -0400, Charlie Brady wrote:
On Thu, 10 Jul 2003, Mark E. Mallett wrote:
On Thu, Jul 10, 2003 at 05:52:08PM +0300, Timo Sirainen wrote:
mbox is fine as long as you don't want to delete messages. It's perfect for read-only mailing list archives.
Indeed. mbox is fine for some people and some things, and in fact for archived mboxes Maildir can just be a big waste.
It is nice to be open to well-established storage formats. Bear in mind that there are people who refuse to use Maildir formats too.
Sure, but that doesn't mean that such people *must* be catered for by every imap daemon. Trying to keep everybody happy pretty much guarantees that nobody will be completely happy.
I buy that too- but we're not talking about some fringe format. As a somewhat related comment, I originally gravitated towards dovecot because it supported {imap,pop}{Maildir,mbox} using one code base and one tool. While there are things to be said for using multiple tools for multiple things, it's nice to have one motif and one philosophy behind access to a set of mailboxex.
But yeah, catering to every little new format often detracts from the whole. (That's where plug-ins are useful.)
Back to the topic, I'd rather there wasn't a configuration file at all. I've got very used to Dan Berstein/Bruce Guenter software where environment variables and command line arguments are the only configuration.
I also agree here- that kind of control is nice for some things, particularly when those things are simple boolean switches (enable/disable), and especially when it makes sense to control those things at runtime on a discrete basis. However note "environment variables and command line arguments only" is not necessarily the philosophy for djb's stuff (I'm afraid I have no experience with the other). qmail, for example, has a raft of control files that, well, control its operation.
And sometimes more complex configurations do make sense, for example for where encapsulations of multiple settings are common.
-mm-
On Thu, 10 Jul 2003, Mark E. Mallett wrote:
Back to the topic, I'd rather there wasn't a configuration file at all. I've got very used to Dan Berstein/Bruce Guenter software where environment variables and command line arguments are the only configuration.
I also agree here- that kind of control is nice for some things, particularly when those things are simple boolean switches (enable/disable), and especially when it makes sense to control those things at runtime on a discrete basis.
Agreed.
However note "environment variables and command line arguments only" is not necessarily the philosophy for djb's stuff (I'm afraid I have no experience with the other).
However, avoiding parsing *is* a clear policy of his, with, I think, considerable payback in reliability and simplicity of control.
qmail, for example, has a raft of control files that, well, control its operation.
Sure, but they are all *very* simple in format.
And sometimes more complex configurations do make sense, for example for where encapsulations of multiple settings are common.
Such encapsulation can usually be done another way, for example by having
multiple directories, each holding a collection of simple config files.
ezmlm, for instance, works this way.
-- Charlie
On Thu, Jul 10, 2003 at 04:02:17PM -0400, Charlie Brady wrote:
However, avoiding parsing *is* a clear policy of his, with, I think, considerable payback in reliability and simplicity of control.
qmail, for example, has a raft of control files that, well, control its operation.
Sure, but they are all *very* simple in format.
Yes and no. Files like:
tag:value
are easy to read. Files like
field1:field2:field3:field4:field5:field6:field7:field8
or a collection ofa-gazillion easy-to-read but a-lifetime-to-wade-through files make it easy for the computer but very difficult for the human. I kinda like it when the computer makes it easy for the human. There is certainly middle ground where parsing is not error prone (realizing that all programs have to be debugged): somewhere in between LISP and a natural language system is just fine with me. I'm not going to belabor it, though.. Timo will do what he will do :-)
mm
On Thursday, Jul 10, 2003, at 18:48 Europe/Helsinki, Mark E. Mallett wrote:
On this whole storage/namespace/search-path thing-- wouldn't it be nice if various imap/pop server implementations could settle on some standard ways of dealing with things?
I wouldn't mind.
Certainly not the same techniques, but the same effects and terms and storage layouts -- making it possible to switch between various implementations.
That's what I've been trying to do, make it as easy as possible to switch to Dovecot :)
Filesystem layout: for example a big bugaboo is that once you choose a Maildir as your primary mailbox (in a Maildir++ -like world), you're locked in there. Plenty of users already have their own idea of where they'd like their folders to be, and in what format (some maildir, some mbox), and it's just not reasonable to expect them to give that up. It ought to be possible to easily mix and match collections of Maildir, Maildir++, and mbox formats throughout a users' home space (and elsewhere, if allowed). Over in binciland there is the notion of an IMAPdir which addresses some of this although I don't think it's perfect. (I believe Andreas is here too..)
I've thought about implementing IMAPdir-like directory structure before too. I guess it might as well be IMAPdir exactly. IMAPdir-like structure should give users possibility to store the mailboxes in any format anywhere they want by using symlinks. At least that sounds much better to me than having to parse configuration file in user's home directory.
Timo Sirainen tss@iki.fi writes:
On Thu, 2003-07-10 at 16:20, Matthias Andree wrote:
Or what if you want Maildir INBOX and mbox others? Maildir would support subfolders, so you'd have to include it in the namespace reply at least if it's hierarchy separator is different than with the mboxes.
No-one's supposed to use mbox. Optimize as much as you want, it just doesn't scale at all.
mbox is fine as long as you don't want to delete messages. It's perfect for read-only mailing list archives.
Unless you run incremental backups with systems such as IBM/Tivoli Distributed Storage Manager that save complete files. Switching from mbox to Maildir has a *major* effect on the nightly backup data volume. With many users or users with big mboxes, or even with read-only list archives that include "current month", it's just prohibitively expensive.
I'd let my sister use it on her computer, but not in serious production -- I'd use NNTP to offer read-only archives rather than IMAP, but that's a personal preference.
-- Matthias Andree
On Fri, 11 Jul 2003, Matthias Andree wrote:
No-one's supposed to use mbox. Optimize as much as you want, it just doesn't scale at all. mbox is fine as long as you don't want to delete messages. It's perfect for read-only mailing list archives. Unless you run incremental backups with systems such as IBM/Tivoli Distributed Storage Manager that save complete files. Switching from mbox to Maildir has a *major* effect on the nightly backup data volume. With many users or users with big mboxes, or even with read-only
Timo Sirainen tss@iki.fi writes: list archives that include "current month", it's just prohibitively expensive. I'd let my sister use it on her computer, but not in serious production -- I'd use NNTP to offer read-only archives rather than IMAP, but that's a personal preference.
I totally disagree with you here. What is lost with backups is gained in overall load on your system.
I know for a _fact_ (my own personal experience) that there are large free email providers that use Maildir for this very purpose.
Andy
-- Andreas Aardal Hanssen
Andreas Aardal Hanssen dovecot@andreas.hanssen.name writes:
I'd let my sister use it on her computer, but not in serious production -- I'd use NNTP to offer read-only archives rather than IMAP, but that's a personal preference.
I totally disagree with you here. What is lost with backups is gained in overall load on your system.
I know for a _fact_ (my own personal experience) that there are large free email providers that use Maildir for this very purpose.
Either you've lost me or we're talking past each other. I prefer Maildir, in case that isn't clear. I've heard too much whining about mbox corruption to care any longer about mbox format. It may work for static archives, but it'd best be accompanied by a useful index in that case, to avoid the recurring search "where is my <CR><LF>From<SP> line".
-- Matthias Andree
On Fri, 11 Jul 2003, Matthias Andree wrote:
Andreas Aardal Hanssen dovecot@andreas.hanssen.name writes:
I totally disagree with you here. What is lost with backups is gained in overall load on your system. I know for a _fact_ (my own personal experience) that there are large free email providers that use Maildir for this very purpose. Either you've lost me or we're talking past each other. I prefer Maildir, in case that isn't clear. I've heard too much whining about mbox corruption to care any longer about mbox format. It may work for static archives, but it'd best be accompanied by a useful index in that case, to avoid the recurring search "where is my <CR><LF>From<SP> line".
Whooh - my bad. :P
Andy
-- Andreas Aardal Hanssen
participants (10)
-
Andreas Aardal Hanssen
-
Charlie Brady
-
Charlie Brady
-
David B Harris
-
Farkas Levente
-
Mark E. Mallett
-
Matthias Andree
-
Matthias Andree
-
Stuart Henderson
-
Timo Sirainen