[Dovecot] shared mailboxes and indexes
Hello,
We are deploying shared mailboxes in our mail system. We are running 2.1.9 and mail backend is maildir.
As described at http://wiki.dovecot.org/SharedMailboxes/Shared when shared namespace is configured as
namespace shared { separator = / prefix = shared/%%u/ location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u subscriptions = no list = children }
each user accessing this folder has its own index, stored in ~/Maildir/shared/%%u/<shared mailbox>, hasn't it?
Our mail is store in NFS disks so we are very concerned about indexes optimizations (we had performance problems until we got all of our indexes and nfs tunned). So, is there any way so those indexes could be shared for all users (and they always would be updated).
Regarding this... if we'd use dbox instead of maildir, indexes are a really important part of the mailbox and they can't be re-constructed when they are outdated. So, how do shared mailboxes work with dbox backend? Do I have to configure indexes in any particular way?
-- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337
On 16.11.2012, at 12.11, Angel L. Mateo wrote:
We are deploying shared mailboxes in our mail system. We are running 2.1.9 and mail backend is maildir.
As described at http://wiki.dovecot.org/SharedMailboxes/Shared when shared namespace is configured as
namespace shared { separator = / prefix = shared/%%u/ location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u subscriptions = no list = children }
each user accessing this folder has its own index, stored in ~/Maildir/shared/%%u/<shared mailbox>, hasn't it?
Right.
Our mail is store in NFS disks so we are very concerned about indexes optimizations (we had performance problems until we got all of our indexes and nfs tunned). So, is there any way so those indexes could be shared for all users (and they always would be updated).
If you don't need per-user flags you can just remove the per-user INDEX. If you want per-user flags and want to share indexes, you need the new INDEXPVT option that exists in v2.2 or as a patch to v2.1: http://dovecot.org/patches/2.1/private-index.diff
Regarding this... if we'd use dbox instead of maildir, indexes are a really important part of the mailbox and they can't be re-constructed when they are outdated. So, how do shared mailboxes work with dbox backend? Do I have to configure indexes in any particular way?
INDEXPVT is a requirement with dbox if you want per-user flags.
BTW. Do you have multiple Dovecot backend servers? Director works only when you're not using shared mailboxes..
Il 23/11/2012 08:07, Timo Sirainen ha scritto:
BTW. Do you have multiple Dovecot backend servers? Director works only when you're not using shared mailboxes..
I'm not happy to hear that, so if today we are running one dovecot server with shared mailbox enabled and tomorrow we will switch to a more complex installation with Director we will be unable to still provide shared mailbox to our customers?
-- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it
On 23.11.2012, at 13.27, Alessio Cecchi wrote:
Il 23/11/2012 08:07, Timo Sirainen ha scritto:
BTW. Do you have multiple Dovecot backend servers? Director works only when you're not using shared mailboxes..
I'm not happy to hear that, so if today we are running one dovecot server with shared mailbox enabled and tomorrow we will switch to a more complex installation with Director we will be unable to still provide shared mailbox to our customers?
You can't reliably do it if the mailboxes are accessed directly via NFS. The current idea to solve this is to use imapc backend with master users, so the actual mailbox access for each user is always done by only one server. I think someone already managed to configure such a setup.
Timo Sirainen <tss@iki.fi> wrote:
On 23.11.2012, at 13.27, Alessio Cecchi wrote:
Il 23/11/2012 08:07, Timo Sirainen ha scritto:
BTW. Do you have multiple Dovecot backend servers? Director works only when you're not using shared mailboxes..
I'm not happy to hear that, so if today we are running one dovecot server with shared mailbox enabled and tomorrow we will switch to a more complex installation with Director we will be unable to still provide shared mailbox to our customers?
You can't reliably do it if the mailboxes are accessed directly via NFS. The current idea to solve this is to use imapc backend with master users, so the actual mailbox access for each user is always done by only one server. I think someone already managed to configure such a setup.
This was me.
It works (with one minor quirk, more on this later) in my current test setup like so:
a) 1 to X user-servers with the users mailboxes on them b) 1 shared-server with the shared mailboxes on them
On the user-servers I have namespace like this:
namespace { list = yes location = imapc:~/imapc-shared prefix = IMAPShared/ separator = / subscriptions = no type = public }
and a userdb like this:
userdb { args = /etc/dovecot/dovecot-ldap.conf.ext default_fields = uid=virtmail gid=virtmail home=/srv/mail/%2MLn/%Ln/ imapc_password=%w quota_rule2=Trash:ignore driver = ldap }
Note the "imapc_password=%w" in the default_fields. This is needed to pass the users login-password to the shared-server.
The imapc proxy code is configured like this:
imapc_host = m-st-sh-01.example.com imapc_master_user = %u imapc_user = shared
This uses the original user as the master-user and a fixed user with the original users password to login into the shared-server.
Note: You CANNOT have ACLs activated on the users-servers, because this will interfere with the permissions of ht IMAPShared namespace, rendering the mailboxes located in there unavailable for your users.
The shared-storage then uses a static passdb for the user "shared":
passdb { args = user=shared password=complicatedpasswordhere driver = static }
and a static userdb for the virtual-user:
userdb { args = uid=virtmail gid=virtmail home=/srv/mail/%Ln driver = static }
and additional a passdb for the "master" users:
passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap master = yes pass = yes }
which are pulled from LDAP, just like on the user-servers.
Now the mentioned quirk: Because all connections on the shared-server are made to the same user "shared" and are coming from very few IPs (the 1 to X user-servers), you need to set a very high mail_max_userip_connections value.
I set mine to 1000 just to be sure.
---> But: I have NOT configured login_trusted_networks, so this may be my error in that case.
And, a second quirk: You can only offer admin-provided shared folders, your users CANNOT share folders themselves, because there is quite some manual work involved in creating and setting them up on the central shared-server.
For example: To provide a Mailbox with the Name "Test.3" in IMAPShared I need the following directory structure on m-st-sh-01.example.com:
/srv/mail/shared/mdbox/mailboxes/Test.3/
In .../Test.3/ there is the dbox-Mails folder with the indexes and other meta information. The admin needs to create the dovecot-acl file to configure the access permissions:
root@m-st-sh-01:/srv/mail/shared/mdbox/mailboxes/Test.3/dbox-Mails# cat dovecot-acl user=gbgr14 lrwstipek user=bbgr99 lrwstipek
The users are then able to see those folders as IMAPShared/Test.3/, subscriptions are managed inside the users own subscription file on their respective "home" server, while flages and tags are shared.
Grüße, Sven.
-- Sigmentation fault. Core dumped.
On 23.11.2012, at 17.53, Sven Hartge wrote:
BTW. Do you have multiple Dovecot backend servers? Director works only when you're not using shared mailboxes..
You can't reliably do it if the mailboxes are accessed directly via NFS. The current idea to solve this is to use imapc backend with master users, so the actual mailbox access for each user is always done by only one server. I think someone already managed to configure such a setup.
This was me.
It works (with one minor quirk, more on this later) in my current test setup like so:
a) 1 to X user-servers with the users mailboxes on them b) 1 shared-server with the shared mailboxes on them
For implementing shared mailboxes between all user servers, I think what would need to be developed is:
imapc_host = m-st-sh-01.example.com imapc_master_user = %u imapc_user = shared
Somehow being able to set "imapc_user = %%u" where %%u expands to the shared namespace's username. Or maybe setting the imapc_user automatically to that when accessing it via type=shared namespace.
Note: You CANNOT have ACLs activated on the users-servers, because this will interfere with the permissions of ht IMAPShared namespace, rendering the mailboxes located in there unavailable for your users.
And some way to disable ACLs for shared namespaces that use imapc. Not sure what would be a nice way of doing this.
The attached patch contains these two changes. The first one I could commit immediately. The second one probably would need to be configurable somehow (maybe a generic disable_acls=yes setting for namespace?)
Now the mentioned quirk: Because all connections on the shared-server are made to the same user "shared" and are coming from very few IPs (the 1 to X user-servers), you need to set a very high mail_max_userip_connections value.
I set mine to 1000 just to be sure.
---> But: I have NOT configured login_trusted_networks, so this may be my error in that case.
I don't think that setting helps.
On 27.11.2012, at 3.00, Timo Sirainen wrote:
Now the mentioned quirk: Because all connections on the shared-server are made to the same user "shared" and are coming from very few IPs (the 1 to X user-servers), you need to set a very high mail_max_userip_connections value.
I set mine to 1000 just to be sure.
---> But: I have NOT configured login_trusted_networks, so this may be my error in that case.
I don't think that setting helps.
But something like this should help:
remote 10.0.0.0/8 { mail_max_userip_connections = 0 }
Timo Sirainen <tss@iki.fi> wrote:
On 23.11.2012, at 17.53, Sven Hartge wrote:
BTW. Do you have multiple Dovecot backend servers? Director works only when you're not using shared mailboxes..
You can't reliably do it if the mailboxes are accessed directly via NFS. The current idea to solve this is to use imapc backend with master users, so the actual mailbox access for each user is always done by only one server. I think someone already managed to configure such a setup.
This was me.
It works (with one minor quirk, more on this later) in my current test setup like so:
a) 1 to X user-servers with the users mailboxes on them b) 1 shared-server with the shared mailboxes on them
For implementing shared mailboxes between all user servers, I think what would need to be developed is:
imapc_host = m-st-sh-01.example.com imapc_master_user = %u imapc_user = shared
Somehow being able to set "imapc_user = %%u" where %%u expands to the shared namespace's username. Or maybe setting the imapc_user automatically to that when accessing it via type=shared namespace.
Wouldn't you still need the target users host because this will be dynamic depending on the target user?
Grüße, Sven.
-- Sigmentation fault. Core dumped.
On 27.11.2012, at 3.24, Sven Hartge wrote:
For implementing shared mailboxes between all user servers, I think what would need to be developed is:
imapc_host = m-st-sh-01.example.com imapc_master_user = %u imapc_user = shared
Somehow being able to set "imapc_user = %%u" where %%u expands to the shared namespace's username. Or maybe setting the imapc_user automatically to that when accessing it via type=shared namespace.
Wouldn't you still need the target users host because this will be dynamic depending on the target user?
imapc_host = director
Also the database of which users have mailboxes shared to others would need to be something that all the servers can access. Either via NFS or with SQL backend.
Timo Sirainen <tss@iki.fi> wrote:
On 27.11.2012, at 3.24, Sven Hartge wrote:
For implementing shared mailboxes between all user servers, I think what would need to be developed is:
imapc_host = m-st-sh-01.example.com imapc_master_user = %u imapc_user = shared
Somehow being able to set "imapc_user = %%u" where %%u expands to the shared namespace's username. Or maybe setting the imapc_user automatically to that when accessing it via type=shared namespace.
Wouldn't you still need the target users host because this will be dynamic depending on the target user?
imapc_host = director
Is this "director" intended to be a 'magic' string or the hostname of the director?
Grüße, Sven.
-- Sigmentation fault. Core dumped.
El 23/11/12 08:07, Timo Sirainen escribió:
On 16.11.2012, at 12.11, Angel L. Mateo wrote:
We are deploying shared mailboxes in our mail system. We are running 2.1.9 and mail backend is maildir.
As described at http://wiki.dovecot.org/SharedMailboxes/Shared when shared namespace is configured as
namespace shared { separator = / prefix = shared/%%u/ location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u subscriptions = no list = children }
each user accessing this folder has its own index, stored in ~/Maildir/shared/%%u/<shared mailbox>, hasn't it?
Right.
Our mail is store in NFS disks so we are very concerned about indexes optimizations (we had performance problems until we got all of our indexes and nfs tunned). So, is there any way so those indexes could be shared for all users (and they always would be updated).
If you don't need per-user flags you can just remove the per-user INDEX. If you want per-user flags and want to share indexes, you need the new INDEXPVT option that exists in v2.2 or as a patch to v2.1: http://dovecot.org/patches/2.1/private-index.diff
Regarding this... if we'd use dbox instead of maildir, indexes are a really important part of the mailbox and they can't be re-constructed when they are outdated. So, how do shared mailboxes work with dbox backend? Do I have to configure indexes in any particular way?
INDEXPVT is a requirement with dbox if you want per-user flags.
BTW. Do you have multiple Dovecot backend servers? Director works only when you're not using shared mailboxes..
Oh... sad to read this. I have multiple backend server behind a director one.
-- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337
participants (4)
-
Alessio Cecchi
-
Angel L. Mateo
-
Sven Hartge
-
Timo Sirainen