[Dovecot] Shared mailbox plans
** Configuration **
namespace shared { prefix = user/%u/ location = maildir:/home/%u/Maildir:INDEX=~/Maildir/shared-indexes }
So the only difference to how it's configured now is that %u is
expanded to whatever user whose mailboxes we're accessing. ACL plugin
then uses that user as the owner in the namespace's mailbox owner.
Since the default ACL is to not give any access to non-owners, the
mailboxes without dovecot-acl files will be invisible.
If ACL plugin isn't loaded, I suppose the mailboxes can be accessed
if the process has filesystem permissions to access them.
** ACL vfile speedups **
Currently it stat() the dovecot-acl file a bit too often. It doesn't
do it more than once a second, but there's really no need to do it
even that often. I guess it could be changed to be configurable with
default being something like 5 minutes.
Create a new dovecot-acl-list file which lists all the mailboxes
which have dovecot-acl file with "l" right (visible in mailbox list)
to any non-owner. It could be in "<timestamp> <mailbox name>" format.
This allows other users to quickly get a list of mailboxes that are
potentially visible to them. They still need to read the dovecot-acl
file from the mailbox before knowing for sure.
If the dovecot-acl-list file doesn't exist or if at any time any
timestamp doesn't match dovecot-acl file's current mtime, the file is
rebuilt by looking into all mailboxes' dovecot-acl files (if the
reading user has write permissions to the directory, otherwise it
just keeps it in memory). Whenever Dovecot modifies dovecot-acl file,
it also updates the dovecot-acl-list file. This means that the file
can contain stale data only if a new dovecot-acl file is created
without updating dovecot-acl-list file. Even that will be fixed the
next time the owner does a LIST (or select the mailbox, or anything
that causes its dovecot-acl file to be stated), which causes ACL
plugin to notice the desync.
There exist also global ACL files. These need to be included in the
dovecot-acl-list file as well. The only issue with them is that user
doesn't necessarily have a mailbox even though the global ACL file
exists. So these need to be marked somehow in the dovecot-acl-list
file as non-existing mailboxes. Then if user creates a mailbox with
such name, the mark is removed. Deleting the mailbox puts the mark back.
After the visible mailbox list has been figured out once, only the
dovecot-acl-list file needs to be stat()ed once in a while instead of
all the mailboxes' dovecot-acl files.
** User list **
The next problem is how to quickly get a list of users whose
mailboxes are visible to ourself so they can be included in mailbox
list. I couldn't figure out anything new and great for this, so the
options are the same as in http://dovecot.org/list/dovecot/2006-
October/017082.html
Although for dict backend I could make it a bit more secure.
Currently dict has "private" and "shared read-write" options to store
the data with. The shared users list could be done with "shared read-
only".
** Virtual domains **
ACL plugin could support virtual "group=@domain" to give access to
usernames ending with @domain. Or perhaps the group could support
wildcards? "group=*@domain"? That way it wouldn't be required to have
@ really in the username.
Anyway, I think "anyone" should then be aliased to "group=*@domain".
If there are multiple virtual domains and you let the users
manipulate the ACLs themselves, I'd think they want anyone to mean
anyone within my domain (my organization) rather than really anyone
who just happens to be using the same server.
I think it would also be annoying if anyone could really force their
own mailboxes to be visible to a lot of people. That could be used
for spamming as well..
If administrator wants to have global mailboxes that really are
visible to all domains, they could be done with public namespaces
(not shared).
** Quota **
This is kind of problematic. The behavior depends entirely on quota
backend:
Filesystem quota of course tracks the quota based on the file's
ownerMaildir++ quota counts the quota for the user whose maildir the
shared mailbox exists in. This means it also requires filesystem
level access to maildirsize file, otherwise the quota updating
happens sometimes later when the quota is recalculated.Dict quota.. I think it'll need a new path for each shared
mailbox, which is modifiable by any user. So while your own quota
would exist in private/quota/storage, the shared mailboxes' quota
would exist in shared/quota/storage. This means that the dict quota
needs to be able to know what mailboxes the current user has shared.
It also gets a bit tricky to update the quota if mailbox is unshared.
The quota could be tracked individually for each mailbox also, but
I'm not sure if that's such a great idea either.
Perhaps a new quota backend could be implemented to make all this
easier. The mail sizes could be tracked in index file, so getting the
size of a mailbox could be done simply by reading the value from the
index file's header. With mailbox list indexes (in CVS HEAD) they'd
all be listed in a single mailbox.list.index file, which would make
summing them up really fast. This would require that users don't use
private index files for shared mailboxes though. But if the users
don't have shell access the shared index files are probably a good
idea anyway (at least from performance point of view).
** Symlink shares **
Currently the only way to implement shared mailboxes among users is
for administrator to create symlinks. This will be possible in future
too with a configuration like:
namespace shared { prefix = Shared/ location = maildir:~/Maildir/shared-symlinks:INDEX=~/Maildir/ shared-indexes }
The "owner" ACL won't match anyone. So it'll work exactly as it does
now.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Timo Sirainen schrieb:
** Configuration **
namespace shared { prefix = user/%u/ location = maildir:/home/%u/Maildir:INDEX=~/Maildir/shared-indexes }
So the only difference to how it's configured now is that %u is expanded to whatever user whose mailboxes we're accessing. ACL plugin then uses that user as the owner in the namespace's mailbox owner. Since the default ACL is to not give any access to non-owners, the mailboxes without dovecot-acl files will be invisible.
If ACL plugin isn't loaded, I suppose the mailboxes can be accessed if the process has filesystem permissions to access them.
** ACL vfile speedups **
Currently it stat() the dovecot-acl file a bit too often. It doesn't do it more than once a second, but there's really no need to do it even that often. I guess it could be changed to be configurable with default being something like 5 minutes.
Create a new dovecot-acl-list file which lists all the mailboxes which have dovecot-acl file with "l" right (visible in mailbox list) to any non-owner. It could be in "<timestamp> <mailbox name>" format. This allows other users to quickly get a list of mailboxes that are potentially visible to them. They still need to read the dovecot-acl file from the mailbox before knowing for sure.
If the dovecot-acl-list file doesn't exist or if at any time any timestamp doesn't match dovecot-acl file's current mtime, the file is rebuilt by looking into all mailboxes' dovecot-acl files (if the reading user has write permissions to the directory, otherwise it just keeps it in memory). Whenever Dovecot modifies dovecot-acl file, it also updates the dovecot-acl-list file. This means that the file can contain stale data only if a new dovecot-acl file is created without updating dovecot-acl-list file. Even that will be fixed the next time the owner does a LIST (or select the mailbox, or anything that causes its dovecot-acl file to be stated), which causes ACL plugin to notice the desync.
There exist also global ACL files. These need to be included in the dovecot-acl-list file as well. The only issue with them is that user doesn't necessarily have a mailbox even though the global ACL file exists. So these need to be marked somehow in the dovecot-acl-list file as non-existing mailboxes. Then if user creates a mailbox with such name, the mark is removed. Deleting the mailbox puts the mark back.
After the visible mailbox list has been figured out once, only the dovecot-acl-list file needs to be stat()ed once in a while instead of all the mailboxes' dovecot-acl files.
** User list **
The next problem is how to quickly get a list of users whose mailboxes are visible to ourself so they can be included in mailbox list. I couldn't figure out anything new and great for this, so the options are the same as in http://dovecot.org/list/dovecot/2006-October/017082.html
Although for dict backend I could make it a bit more secure. Currently dict has "private" and "shared read-write" options to store the data with. The shared users list could be done with "shared read-only".
** Virtual domains **
ACL plugin could support virtual "group=@domain" to give access to usernames ending with @domain. Or perhaps the group could support wildcards? "group=*@domain"? That way it wouldn't be required to have @ really in the username.
Anyway, I think "anyone" should then be aliased to "group=*@domain". If there are multiple virtual domains and you let the users manipulate the ACLs themselves, I'd think they want anyone to mean anyone within my domain (my organization) rather than really anyone who just happens to be using the same server.
I think it would also be annoying if anyone could really force their own mailboxes to be visible to a lot of people. That could be used for spamming as well..
If administrator wants to have global mailboxes that really are visible to all domains, they could be done with public namespaces (not shared).
** Quota **
This is kind of problematic. The behavior depends entirely on quota backend:
Filesystem quota of course tracks the quota based on the file's owner
Maildir++ quota counts the quota for the user whose maildir the shared mailbox exists in. This means it also requires filesystem level access to maildirsize file, otherwise the quota updating happens sometimes later when the quota is recalculated.
Dict quota.. I think it'll need a new path for each shared mailbox, which is modifiable by any user. So while your own quota would exist in private/quota/storage, the shared mailboxes' quota would exist in shared/quota/storage. This means that the dict quota needs to be able to know what mailboxes the current user has shared. It also gets a bit tricky to update the quota if mailbox is unshared. The quota could be tracked individually for each mailbox also, but I'm not sure if that's such a great idea either.
Perhaps a new quota backend could be implemented to make all this easier. The mail sizes could be tracked in index file, so getting the size of a mailbox could be done simply by reading the value from the index file's header. With mailbox list indexes (in CVS HEAD) they'd all be listed in a single mailbox.list.index file, which would make summing them up really fast. This would require that users don't use private index files for shared mailboxes though. But if the users don't have shell access the shared index files are probably a good idea anyway (at least from performance point of view).
** Symlink shares **
Currently the only way to implement shared mailboxes among users is for administrator to create symlinks. This will be possible in future too with a configuration like:
namespace shared { prefix = Shared/ location = maildir:~/Maildir/shared-symlinks:INDEX=~/Maildir/shared-indexes }
The "owner" ACL won't match anyone. So it'll work exactly as it does now. Hi Timo, for acl public folders with virtual domains, wouldnt it be a good idea to have them in sql as backend?
Mit freundlichen Gruessen Best Regards
Robert Schetterer
https://www.schetterer.org Munich/Bavaria/Germany -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org
iD8DBQFGF4D7fGH2AvR16oERAi57AJ9iaNF3vEWr2wLXURuslUPteTZZPACffqE/ dw3GBitZB5FNXEkhIUSTAjI= =QDH3 -----END PGP SIGNATURE-----
Timo Sirainen schrieb:
On 7.4.2007, at 14.31, Robert Schetterer wrote:
for acl public folders with virtual domains, wouldnt it be a good idea to have them in sql as backend?
Why? Hi Timo, as imap clients that are able to edit imap acls are rare ( thunderbird, Outlook cant do it yet i think ) i think it would be helpfull to have acls in sql for writing php clients i.e squirrelmail plugins etc, or do i fail here?
--
Mit freundlichen Gruessen Best Regards
Robert Schetterer
https://www.schetterer.org Munich/Bavaria/Germany
On 8.4.2007, at 1.04, Robert Schetterer wrote:
Timo Sirainen schrieb:
On 7.4.2007, at 14.31, Robert Schetterer wrote:
for acl public folders with virtual domains, wouldnt it be a good
idea to have them in sql as backend? Why? Hi Timo, as imap clients that are able to edit imap acls are rare ( thunderbird, Outlook cant do it yet i think ) i think it would be helpfull to have acls in sql for writing php clients i.e squirrelmail plugins etc, or do i fail here?
Keeping ACLs in SQL is different from keeping the whole mailbox data
in SQL. Currently the ACL plugin supports only vfile backend, but it
wouldn't be too difficult to add support for SQL backend.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 8.4.2007, at 1.04, Robert Schetterer wrote:
Timo Sirainen schrieb:
On 7.4.2007, at 14.31, Robert Schetterer wrote:
for acl public folders with virtual domains, wouldnt it be a good idea to have them in sql as backend? Why? Hi Timo, as imap clients that are able to edit imap acls are rare ( thunderbird, Outlook cant do it yet i think ) i think it would be helpfull to have acls in sql for writing php clients i.e squirrelmail plugins etc, or do i fail here?
Keeping ACLs in SQL is different from keeping the whole mailbox data in SQL. Currently the ACL plugin supports only vfile backend, but it wouldn't be too difficult to add support for SQL backend. Hi Timo, nice to hear, perhaps it can be done in the future but for now going to stable version 1 seems to me
Timo Sirainen schrieb: the more urgent job. thx so much for coding dovecot Happy Easter!
Mit freundlichen Gruessen Best Regards
Robert Schetterer
https://www.schetterer.org Munich/Bavaria/Germany -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org
iD8DBQFGGK1zfGH2AvR16oERAiTTAJ0b0UYLH7/O3waCIIXwIwZIG1ySYACffNQ3 LQvaM0mDLS/07FWxnYTDNXs= =VPGk -----END PGP SIGNATURE-----
Timo Sirainen writes:
Keeping ACLs in SQL is different from keeping the whole mailbox data
in SQL. Currently the ACL plugin supports only vfile backend, but it
wouldn't be too difficult to add support for SQL backend.
Having ACLs in SQL allows companies to have a centralized place for all ACLs.
For instance our mail tables are generated from other tables.
Users update the "primary" tables in a management screen. A program takes data from these primary tables and populates the appropriate tables. All backend mailstore machines connect to the database to get their information.
If ACLs were on files one would have to have a specialized program to grab data from a centralized database to create the local files. For one machine that is likely ok but for many machines it starts to get complex (ie you have to sync all the programs accross all the servers to be up to date).
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, 11 Apr 2007, Francisco Reyes wrote:
Having ACLs in SQL allows companies to have a centralized place for all ACLs.
Well, this statement is true for any backend, incl. LDAP.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBRiiRrC9SORjhbDpvAQJYvggA2PG6AlJgFMwUoKme6EqLxoKzeXPhaFyb cKnfF7aBKYENubsSqkSIdJDcxLz+ySyqQIBqlzVF2LFY0Tjf4C0tbzyzqTQAEbc2 6QKhxBy4qoOM6b+rIWA63LOjh/CUYVr9oDcC6kTcA0cvahuBZj/4GV+BEJ5FVzZl 2jnCjwvAfUcxWPddF0VbflfcM9a3SGdA2v0kJLgYEivVNnuGM9zbO7wv4NF0NVdm YMKbQPSQKXNnluniu673SkVRwL8XEcn+risPHAwrrX5XrdCfgz9XzGIgQRh75+Hp R3UMqFulxlvPlAKo3XNABATVL/7sfE6oBL6pE13NimMDZHfimvZdkQ== =Khj+ -----END PGP SIGNATURE-----
Steffen Kaiser writes:
Having ACLs in SQL allows companies to have a centralized place for all ACLs.
Well, this statement is true for any backend, incl. LDAP.
Correct. But many companies do not have a LDAP infrastructure. It is much easier to setup a Mysql DB and put a table for ACL than it is to setup an LDAP infrastructure.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Sun, 22 Apr 2007 17:49:52 -0400 lists@stringsutils.com wrote:
Steffen Kaiser writes:
Having ACLs in SQL allows companies to have a centralized place for all ACLs.
Well, this statement is true for any backend, incl. LDAP.
Correct. But many companies do not have a LDAP infrastructure. It is much easier to setup a Mysql DB and put a table for ACL than it is to setup an LDAP infrastructure.
Doesn't that largely depend on your skills with LDAP or your skills with MySQL? You don't really need any more "infrastructure" with LDAP than you do with MySQL. As always, use the tool you know and that works best for you. But I digress. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFGK9p9R/DW9IB/aBQRAs1hAJ9Ld+0qa+f4DDqODSEb8D0eLKV0yQCeK7l3 RTBv6ywkCOBvN7zu3RXoUS4= =3ciZ -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Sun, 22 Apr 2007, Francisco Reyes wrote:
Well, this statement is true for any backend, incl. LDAP.
Correct. But many companies do not have a LDAP infrastructure. It is much easier to setup a Mysql DB and put a table for ACL than it is to setup an LDAP infrastructure.
This is a religious war :-) Let's add: mySQL vs. Postgres. (E.g. we focus on Postgres).
Well, just kidding.
My statement is simply my point of view, that the API should be independed of the backend.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iQEVAwUBRiyA4i9SORjhbDpvAQLO+wgAyfqWAHzwLKn4tiFNXCH8VMaJyd45tyhA 0FsdnkbwLCuo90MCJtUmLCAAomh2A55qL0oTRbOAzDeqQfokbt6bKYUMDyqZKZAY K5kAqivRZAHGxqpySmw43phrCcY1lgnbhkfWG+y50P6GGRfBNWxslz/kWnTS95Zq ZkS3Rrx38x9vHeDA63osFkiXb1zoD/EtT03Z6/KSJKYMz8yF9lrQIJoLiZPGab0d 9KAV6TteO0e7H/g1sENolQGyHcRYROOVG4kXfMCfH960qyZgmIiQnCxuYOvsbq4R k9b7O/5CPz5jXu7JwDuTZE3bpNI370U0tquVpNzenjmaGH8r0o0dTw== =yQkg -----END PGP SIGNATURE-----
Steffen Kaiser writes:
My statement is simply my point of view, that the API should be independed of the backend.
LDAP, Postgresql, Mysql.. I think any of them is fine. I think anything BUT the initial setup that Timo mentioned. Files in the mailstore I think is the least scalable option.
Timo Sirainen wrote:
If ACL plugin isn't loaded, I suppose the mailboxes can be accessed if the process has filesystem permissions to access them.
Kind of, but it doesn't really work in real life. I tried setting it up initially around... say 1.0rc23 and it runs into permissions problems writing the various dovecot.index files because they're 0600 mode to that specific imap logged in user.
The config (commented out right now) looks like: (*)
# Spam Training namespace public { separator = . prefix = SpamTraining. location = maildir:/var/spool/imap_shared/SpamTraining }
What I was finding with testers is that each person's login process was rewriting permissions on the subscriptions file and the index files didn't work out for the same reason; 1 person would drop an email into a subfolder ("MissedSpam" e.g.), Dovecot would write indexes 0600 to that named user then everyone else would start getting errors.
Unfortunately I didn't have the time to do massive debugging right then so just had to comment it out and save it for another day. :-/
-te
(*) right now it works with gnarly ugly symlinks in a user's ~/Maildir under Courier-IMAP, and I have two shellscripts that nightly read the contents of two subfolders - MissedSpam and FalsePositives - and passes those to SpamAssassin with 'sa-learn --spam' e.g. then deletes each mail file.
-- Troy Engel | Systems Engineer Fluid Inc. | http://www.fluid.com
On 7.4.2007, at 19.02, Troy Engel wrote:
What I was finding with testers is that each person's login process
was rewriting permissions on the subscriptions file and the index
files didn't work out for the same reason; 1 person would drop an
email into a subfolder ("MissedSpam" e.g.), Dovecot would write
indexes 0600 to that named user then everyone else would start
getting errors.
You'll need to create dovecot-shared file as http://wiki.dovecot.org/
SharedMailboxes describes. Although it doesn't affect the
subscriptions file. I'll have to figure out what to do with
subscriptions in general. I think all the subscriptions for all the
namespaces should exist in a single file.
One more thing:
I think dovecot-shared file is pointless. It can be implemented by taking the rw-permissions from the Maildir directory (or mbox file).
Only problem that I see is if Maildir unintentionally has too wide permissions, this change would make the created mails have too wide permissions as well. Hmm. Perhaps with Maildir the permissions could be taken from Maildir/new directory instead which has a bit better chance of being correct.
I think I'll also add a check to compare Maildir, new, cur and tmp directories' permissions and log a warning if they're not the same.
Timo Sirainen wrote:
I think I'll also add a check to compare Maildir, new, cur and tmp directories' permissions and log a warning if they're not the same.
Just as a sanity check, mine differ (for whatever reason - I probably used 'maildirmake' from Courier) like so:
$ ls -gGld SpamTraining/ drwxrwxr-x 7 117 Feb 2 10:21 SpamTraining/
$ ls -gGld SpamTraining/[new,cur,tmp]* drwxrwx--- 2 6 Sep 9 2004 SpamTraining/cur drwxrwx--- 2 6 Sep 9 2004 SpamTraining/new drwxrwx--- 2 6 Sep 9 2004 SpamTraining/tmp
$ ls -gGld SpamTraining/.MissedSpam/ drwxrwxr-t 5 36 Apr 7 09:17 SpamTraining/.MissedSpam/
$ ls -gGld SpamTraining/.MissedSpam/[new,cur,tmp]* drwxrwxrwt 2 6 Feb 3 02:06 SpamTraining/.MissedSpam/cur drwxrwxrwt 2 6 Feb 2 10:26 SpamTraining/.MissedSpam/new drwxrwxrwt 2 6 Feb 2 10:28 SpamTraining/.MissedSpam/tmp
So whether or not *my* permissions are correct (they actually look pretty screwed up), implementing that check would start blowing out warnings in the logfile; yet these folders work perfectly fine I suppose.
FYI only. :) -te
-- Troy Engel | Systems Engineer Fluid, Inc | http://www.fluid.com
participants (6)
-
Francisco Reyes
-
Joakim Ryden
-
Robert Schetterer
-
Steffen Kaiser
-
Timo Sirainen
-
Troy Engel