[Dovecot] Shared folders plans for the future
So while driving for several hours in my car recently and having nothing else to do, I started thinking how they could be done.
I think the "public folders" configuration will stay as it is now. Does it really even need anything else? The problematic case is how the users can share their mailboxes to other users. There are two problems related to it:
- How to get (quickly) a list of another user's mailboxes that I have
access to?
- How to get quickly a list of all users who have mailboxes that I have
access to?
The 1) problem is easier to solve. Create a directory listing all the mailboxes that are shared to at least someone (have ACLs set). It could be configured like:
namespace shared { # IMAP namespace prefix, %u expands to shared user prefix = user/%u/ # Directory containing symlinks to user's all shared mailboxes share_dir = ~/Maildir/shared # Where to store indexes for other users' accessed mailboxes index_dir = ~/Maildir/others-indexes }
The ~/Maildir/shared directory could then contain symlinks to all the shared mailboxes. The directory list could probably be in Maildir++ layout even though the symlinks would point to mboxes. This requires adding code that can handle mixed maildir/mbox directories, but it'll come anyway..
The 2) problem is more difficult. There are 3 choices:
a) Use a shared database file. lib-dict could be used for this, so the database could be in SQL or in a berkeley db file. The downside to this is that the file needs to be readable and writable to everyone, so if users have shell access the users could break the file.
They dictionary would contain data in format "<destination_user>/<sharing_user>" = 1 and "<destination_group>/<sharing_user>" = 1, ie. the value would be ignored and only the existence of the keys would be important. Then iterating through "<my_username>/*" and "<all_my_groups>/*" would list the users who probably have mailboxes where I have access to.
b) Use a directories with sticky bit set. Well, basically the same idea as in a), except filesystem would be used. Actually perhaps this could be made another lib-dict backend..
c) Brute force: Just assume everyone potentially have shared mailboxes. Get the list of users by readdir()ing through /home (configurable).
I guess each of these could be made an option.. So something like:
namespace shared { .. # a) shared_dict = db:/var/lib/dovecot/shared-mailboxes.db # b) shared_dict = dir:/var/lib/dovecot/shared-mailboxes # c) shared_dict = bruteforce:/home }
I'm currently also implementing a "mailbox list index", which contains a list of all the user's mailboxes and some useful metadata for them, such as number of messages and so on. So that STATUS command wouldn't need to open the mailbox index files if they haven't changed.
The mailbox list index updating is normally done when directory's mtime has changed. For list of shared users that would also work in 2b) and 2c) cases, but for 2a) I'd either have to always sync everything or alternatively add some sequence/<dest_user/group>=# value which is grown for each update, and the syncing is done only when the sequence is updated.
Hi Timo,
I'm reading/thinking about the direction you're headed, but had a question...
Here you say:
The ~/Maildir/shared directory could then contain symlinks to all the shared mailboxes. The directory list could probably be in Maildir++ layout even though the symlinks would point to mboxes.
Why would the symlinks point to mboxes? Is it required for shared mailboxes to be in mbox format? If so, I'm confused - didn't you say that the support for Public Shared folders in maildir format should work?
Or would this only apply to each Users Shared folders? If so, why? (please don't misunderstand, I'm not contradicting you, but I'm curious)...
Lastly, are you contemplating adding support for Client-side control of the ACLs for the users Shared folders? I'm not sure about anyone else, but I don't give my users shell access, so they wouldn't be able set these up themselves...
Thanks for sharing your thoughts out loud...
On Tue, 2006-10-24 at 14:57 -0400, Charles Marcus wrote:
The ~/Maildir/shared directory could then contain symlinks to all the shared mailboxes. The directory list could probably be in Maildir++ layout even though the symlinks would point to mboxes.
Why would the symlinks point to mboxes? Is it required for shared mailboxes to be in mbox format? If so, I'm confused - didn't you say that the support for Public Shared folders in maildir format should work?
There's a bug: s/even though/even if/ :)
So I mean simply that no matter if you actually use mboxes or maildirs, the shared directory would contain symlinks like:
my_stuff -> ../.my_stuff my_stuff.subfolder -> ../.my_stuff.subfolder
And with mboxes (assuming mail in ~/mail/ and shared in ~/.shared):
my_stuff -> ../mail/my_stuff my_dir.subfolder -> ../mail/my_dir/subfolder
Lastly, are you contemplating adding support for Client-side control of the ACLs for the users Shared folders? I'm not sure about anyone else, but I don't give my users shell access, so they wouldn't be able set these up themselves...
Yea. This would just need updating the ACL plugin to support writing the dovecot-acls files. And then creating a new imap-acl plugin.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Tue, 24 Oct 2006, Timo Sirainen wrote:
I think the "public folders" configuration will stay as it is now. Does it really even need anything else? The problematic case is how the users can share their mailboxes to other users. There are two problems related to it:
- How to get (quickly) a list of another user's mailboxes that I have
access to?
You need this information for "LIST"? Just asking, it would help for large folder structures only, right? Otherwise, it adds yet another redundant stuff, one needs to maintain and probably fix, e.g. run a nightly job to verify the symlinks.
- How to get quickly a list of all users who have mailboxes that I have
access to?
Why do you need this information? Wouldn't it better to pass this information araund via, say, EMail? Then an user may SELECT a specific mailbox directly or can LIST an specific user.
===
What worries me more is the information in the following two pages: http://wiki.dovecot.org/ACL http://wiki.dovecot.org/SharedFolders
For virtual users (with just one account for all users) there is no problem, but for real users ACLs superceed filesystem permissions. Dovecot would need to maintain the "dovecot-shared" files as well as mangle the permissions correctly.
For global shared mailboxes one need to create a group giving all users get read / write permission, possibly narrowed down by the ACLs. But users with shell access can bypass the ACLs.
When an user shares a mailbox to other users, either all these users must belong to one group or Dovecot need to create a group dynamically for them.
Bye,
Steffen Kaiser
On 25.10.2006, at 16.31, Steffen Kaiser wrote:
I think the "public folders" configuration will stay as it is now.
Does it really even need anything else? The problematic case is how the
users can share their mailboxes to other users. There are two problems
related to it:
- How to get (quickly) a list of another user's mailboxes that I
have access to?
You need this information for "LIST"? Just asking, it would help
for large folder structures only, right? Otherwise, it adds yet
another redundant stuff, one needs to maintain and probably fix,
e.g. run a nightly job to verify the symlinks.
Well, I don't think the symlinks would normally break by themselves?
Unless you for some reason manually go and delete them, the only
problem is if a shared maildir is renamed outside Dovecot.
Mailbox listing is anyway done a lot by the clients, so this
operation needs to be pretty fast. Doing a stat(box/dovecot-acls) for
each mailbox could get slow if there are a lot of mailboxes.
- How to get quickly a list of all users who have mailboxes that
I have access to?
Why do you need this information? Wouldn't it better to pass this information araund via, say, EMail? Then an user may SELECT a specific mailbox directly or can LIST an
specific user.
And how do you select a specific mailbox or list a specific user
directly with any commonly used IMAP client? You don't, so you'll
have to show the list of users who have shared mailboxes to you.
What worries me more is the information in the following two pages: http://wiki.dovecot.org/ACL http://wiki.dovecot.org/SharedFolders
For virtual users (with just one account for all users) there is no
problem, but for real users ACLs superceed filesystem permissions. Dovecot would need to maintain the "dovecot-shared" files as well
as mangle the permissions correctly.
Yea, this should be implemented some day as some filesystem-backend.
When an user shares a mailbox to other users, either all these
users must belong to one group or Dovecot need to create a group
dynamically for them.
Well, I suppose dynamic groups could work, but then you'd have to
reserve one GID for each different ACL. Probably too much trouble to
implement..
How many people anyway even need to support users who have shell
access and can share mailboxes to each others? I don't think all that
many.
I think there are just two practical ways to implement this:
- Let sysadmin define all the groups and people who are in them.
Allow the filesystem ACL backend to manipulate the file mode and
group directly.
- Use one or more groups for shared mailboxes which gives a group of
people access to the mailbox, but the vfile ACL backend is still
doing the exact permission checks. Like there could be just one
"shared-mails" group which is set for all mailboxes that are shared,
but each of then then could contain dovecot-acls file which describes
who it's shared to.
Timo Sirainen wrote:
On 25.10.2006, at 16.31, Steffen Kaiser wrote:
I think the "public folders" configuration will stay as it is now. Does it really even need anything else? The problematic case is how the users can share their mailboxes to other users. There are two problems related to it:
- How to get (quickly) a list of another user's mailboxes that I have
access to?
You need this information for "LIST"? Just asking, it would help for large folder structures only, right? Otherwise, it adds yet another redundant stuff, one needs to maintain and probably fix, e.g. run a nightly job to verify the symlinks.
Well, I don't think the symlinks would normally break by themselves? Unless you for some reason manually go and delete them, the only problem is if a shared maildir is renamed outside Dovecot.
Mailbox listing is anyway done a lot by the clients, so this operation needs to be pretty fast. Doing a stat(box/dovecot-acls) for each mailbox could get slow if there are a lot of mailboxes.
- How to get quickly a list of all users who have mailboxes that I
have access to?
Why do you need this information? Wouldn't it better to pass this information araund via, say, EMail? Then an user may SELECT a specific mailbox directly or can LIST an specific user.
And how do you select a specific mailbox or list a specific user directly with any commonly used IMAP client? You don't, so you'll have to show the list of users who have shared mailboxes to you.
What worries me more is the information in the following two pages: http://wiki.dovecot.org/ACL http://wiki.dovecot.org/SharedFolders
For virtual users (with just one account for all users) there is no problem, but for real users ACLs superceed filesystem permissions. Dovecot would need to maintain the "dovecot-shared" files as well as mangle the permissions correctly.
Yea, this should be implemented some day as some filesystem-backend.
ACLs in the filesystem is support by most platforms and filesystems now (atleast those that I use), does dovecot use this in any way, or rather would a future ACL backend do this?
When an user shares a mailbox to other users, either all these users must belong to one group or Dovecot need to create a group dynamically for them.
Well, I suppose dynamic groups could work, but then you'd have to reserve one GID for each different ACL. Probably too much trouble to implement..
How many people anyway even need to support users who have shell access and can share mailboxes to each others? I don't think all that many.
I do :) I don't have a really urgent need for shared folders between users right now, and symlinks + correct ACL seems to work fine for my use (for now), but it would be a really nice feature.
I think there are just two practical ways to implement this:
- Let sysadmin define all the groups and people who are in them.
Allow the filesystem ACL backend to manipulate the file mode and group directly.
- Use one or more groups for shared mailboxes which gives a group of
people access to the mailbox, but the vfile ACL backend is still doing the exact permission checks. Like there could be just one "shared-mails" group which is set for all mailboxes that are shared, but each of then then could contain dovecot-acls file which describes who it's shared to.
Or support actual filesystem ACLs?
What would be nice, (for me :P), is to be able to have groups of users being able to see each others shared folders, say, I have a group "project1" and a group "project2", users in project1 would be able to see all the shared folders of the other users in that group, but not for anyone in the other group "project2" ..
So if I understand it correctly, if we say that only dovecot is making/changing/deleting shared folders, it would be possible to have a back end that stores what folders are shared and such for listing.. and for people to implement "special features" of what shared folders are listed (like my wanted feature above) as a plugin?
Come to think of it, when describing what folders is shared you also say who has access right? (okay, I should read the specs instead of guessing) So if a user gets access to a folder, that folder should be visible to that user? hm, makes my idea a bit redundant, but still :)
Greetings, Asbjorn Sannes
On 10/25/2006 Timo Sirainen (tss@iki.fi) wrote:
- Use one or more groups for shared mailboxes which gives a group of
people access to the mailbox, but the vfile ACL backend is still doing the exact permission checks. Like there could be just one "shared-mails" group which is set for all mailboxes that are shared, but each of then then could contain dovecot-acls file which describes who it's shared to.
If it matters, this one makes the most sense to me...
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, 25 Oct 2006, Timo Sirainen wrote:
Well, I don't think the symlinks would normally break by themselves? Unless
Not by themselves, but by a crashed server, disconnected NFS, ... .
Mailbox listing is anyway done a lot by the clients, so this operation needs to be pretty fast. Doing a stat(box/dovecot-acls) for each mailbox could get slow if there are a lot of mailboxes.
Hmm, so it rather is some sort of cache, like the index. Maybe some "re-create each day" or re-create when "shared" folder is missing, like it is done for the indexes? BTW: Will the folder go into the Control directory as well, for people having filesystem quota?
Just thinking loud: For a global shared mailbox this won't help, because all mailboxes are shared and the server does not know to which particular mailbox the particular user has which access to. I mean 1) reads: "1) How to get (quickly) a list of another user's mailboxes that I have access to?" You still need to open each -acl file and inspect it to gain the information for one particular user. To cache the resulting ACL scan for each user could allocate lots of space. Maybe you can combine 1) and 2)?
And how do you select a specific mailbox or list a specific user directly with any commonly used IMAP client? You don't, so you'll have to show the list of users who have shared mailboxes to you.
OK, I'm to much used to pine. I didn't thought about browsing to a specific mailbox.
Well, I suppose dynamic groups could work, but then you'd have to reserve one GID for each different ACL. Probably too much trouble to implement..
Yep, especially has there are a limited number of groups.
How many people anyway even need to support users who have shell access and can share mailboxes to each others? I don't think all that many.
Agreed.
- Let sysadmin define all the groups and people who are in them. Allow the
filesystem ACL backend to manipulate the file mode and group directly.
Then the user will also need to be able to manipulate the group membership of a mailbox. This would be doable for global shared ones only, I guess.
- Use one or more groups for shared mailboxes which gives a group of people
access to the mailbox, but the vfile ACL backend is still doing the exact permission checks. Like there could be just one "shared-mails" group which is set for all mailboxes that are shared, but each of then then could contain dovecot-acls file which describes who it's shared to.
This sounds best.
Bye,
Steffen Kaiser
Timo Sirainen wrote:
I think the "public folders" configuration will stay as it is now. Does it really even need anything else?
One thing I notice is that if I read a message in my inbox and then move it to a public folder, it is marked as unread in the public folder. Ideally, it would be marked as read (for me, unread for others).
I'm still on 1.0beta6 so forgive me if this has been changed in the RCs.
Mark
Mark Nienberg wrote:
Timo Sirainen wrote:
I think the "public folders" configuration will stay as it is now. Does it really even need anything else?
One thing I notice is that if I read a message in my inbox and then move it to a public folder, it is marked as unread in the public folder.
Ideally, it would be marked as read (for me, unread for others).I'm still on 1.0beta6 so forgive me if this has been changed in the RCs.
Oh, and another thing. Users cannot delete public folders from Thunderbird if they have Thunderbird configured to move deleted items to a Trash folder. In order to delete a public folder, the user has to temporarily configure Thunderbird to delete items immediately.
Mark
Mark Nienberg wrote:
Mark Nienberg wrote:
Timo Sirainen wrote:
I think the "public folders" configuration will stay as it is now. Does it really even need anything else?
One thing I notice is that if I read a message in my inbox and then move it to a public folder, it is marked as unread in the public folder. Ideally, it would be marked as read (for me, unread for others).
I'm still on 1.0beta6 so forgive me if this has been changed in the RCs.
Oh, and another thing. Users cannot delete public folders from Thunderbird if they have Thunderbird configured to move deleted items to a Trash folder. In order to delete a public folder, the user has to temporarily configure Thunderbird to delete items immediately.
But normal users shouldn't normally be able to delete public folders, should they? Unless you mean folders of *theirs* that they have shared.
Public Folders should be managed only by designated admins, no?
Charles Marcus wrote:
Mark Nienberg wrote:
Mark Nienberg wrote:
Timo Sirainen wrote:
I think the "public folders" configuration will stay as it is now. Does it really even need anything else?
One thing I notice is that if I read a message in my inbox and then move it to a public folder, it is marked as unread in the public folder. Ideally, it would be marked as read (for me, unread for others).
I'm still on 1.0beta6 so forgive me if this has been changed in the RCs.
Oh, and another thing. Users cannot delete public folders from Thunderbird if they have Thunderbird configured to move deleted items to a Trash folder. In order to delete a public folder, the user has to temporarily configure Thunderbird to delete items immediately.
But normal users shouldn't normally be able to delete public folders, should they? Unless you mean folders of *theirs* that they have shared.
Public Folders should be managed only by designated admins, no?
In our environment "public" refers to our in-house staff. They create a "public" folder for each project they work on so all team members can see all the mail related to the project. It works best as a collaborative effort. I don't really want them to have to come to me to request creation or deletion of a folder.
The most common reason to delete a folder is that someone tries to make one with a period in its name, and you know what happens then.
Mark
But normal users shouldn't normally be able to delete public folders, should they? Unless you mean folders of *theirs* that they have shared.
Public Folders should be managed only by designated admins, no?
In our environment "public" refers to our in-house staff. They create a "public" folder for each project they work on so all team members can see all the mail related to the project. It works best as a collaborative effort. I don't really want them to have to come to me to request creation or deletion of a folder.
The most common reason to delete a folder is that someone tries to make one with a period in its name, and you know what happens then.
Right - so it just boils down to how you define 'Public' - and you confirmed that these were not actually 'public' folders as *I* would define them, but folders that individuals had shared.
I would describe Public Folders as folders that were created and adminstered by the sys admin(s), for use by people in the company. Certainly, allow them deletion in down-level folders if desired, but they shouldn't be able to delete the parent shares.
So we are on the same page most likely... :)
On Wed, 2006-10-25 at 09:42 -0700, Mark Nienberg wrote:
Timo Sirainen wrote:
I think the "public folders" configuration will stay as it is now. Does it really even need anything else?
One thing I notice is that if I read a message in my inbox and then move it to a public folder, it is marked as unread in the public folder. Ideally, it would be marked as read (for me, unread for others).
I'm still on 1.0beta6 so forgive me if this has been changed in the RCs.
Mark
I'm using rc10 and this is still the case. I agree, it should be marked as read.
Gavin
On Tue, 2006-10-24 at 20:24 +0300, Timo Sirainen wrote:
I think the "public folders" configuration will stay as it is now. Does it really even need anything else?
Hi Timo,
As previously discussed on the list I'd love to see the ACL plug-in work with public folders and virtual users. Our mail server is 100% virtual users so we currently have free for all public folders, it's an accident waiting to happen...
All we require is to be able to set permissions on the server, doesn't have to be user configurable.
Gavin
On 27.10.2006, at 5.26, Fintec wrote:
On Tue, 2006-10-24 at 20:24 +0300, Timo Sirainen wrote:
I think the "public folders" configuration will stay as it is now. Does it really even need anything else?
Hi Timo,
As previously discussed on the list I'd love to see the ACL plug-in
work with public folders and virtual users. Our mail server is 100% virtual users so we currently have free for all public folders, it's an
accident waiting to happen...All we require is to be able to set permissions on the server, doesn't have to be user configurable.
I think that should be possible? Create dovecot-acls file for each
public folder with the people who have access to it. Or did you want
groups or something?
On Mon, 2006-10-30 at 10:24 +0200, Timo Sirainen wrote:
On 27.10.2006, at 5.26, Fintec wrote:
On Tue, 2006-10-24 at 20:24 +0300, Timo Sirainen wrote:
I think the "public folders" configuration will stay as it is now. Does it really even need anything else?
Hi Timo,
As previously discussed on the list I'd love to see the ACL plug-in
work with public folders and virtual users. Our mail server is 100% virtual users so we currently have free for all public folders, it's an
accident waiting to happen...All we require is to be able to set permissions on the server, doesn't have to be user configurable.
I think that should be possible? Create dovecot-acls file for each
public folder with the people who have access to it. Or did you want
groups or something?
I've tried this again with rc10 but the same problems exist.
Specifically our problems are:
- dovecot-acl file within public namespace directory isn't found
- global ACLs (vfile) partially work with virtual users but when used: a) non-ACL restricted public namespace directories stop working b) permitted users are unable to view or create sub-folders
See: http://dovecot.org/pipermail/dovecot/2006-August/015058.html and rest of thread for more details.
Gavin
As previously discussed on the list I'd love to see the ACL plug-in work with public folders and virtual users. Our mail server is 100% virtual users so we currently have free for all public folders, it's an accident waiting to happen...
All we require is to be able to set permissions on the server, doesn't have to be user configurable.
I think that should be possible? Create dovecot-acls file for each public folder with the people who have access to it. Or did you want groups or something?
Support for Groups would be very important... but for us, for now, I can live with not having support... as long as it is coming...
participants (6)
-
Asbjørn Sannes
-
Charles Marcus
-
Fintec
-
Mark Nienberg
-
Steffen Kaiser
-
Timo Sirainen