Hi,
My end goal is to set up shared mailboxes on a cluster as per:
https://wiki.dovecot.org/SharedMailboxes/ClusterSetup
I was having very little luck with it, so I had been trying to break down into pieces and get individual components working. So I have things setup on a single server, with a working dovecot instance. I have no director or any thing else running yet, and I am just trying to get the imapc portion of things working. Eventually I came across a post from Sven Hartge:
https://www.dovecot.org/list/dovecot/2012-November/087046.html
This is from several years ago, but I set up a separate dovecot instance on port 9993 specifically to run the shared folders as he describes, and copied his configuration suggestions pretty much verbatim. The only signficant difference is that I needed to not configure the imapc_user as he had done, and instead as the wiki suggests remove that config option entirely. With this, the imapc connection is being made from the primary instance on port 993 to the shared instance on port 9993.
My problem is that it doesn't appear that the imapc_master_user is being fully acknowledged. The imapc connection must be made with the original user logging in as the master user to the shared user's account, else the ACLs can't do what they are supposed to do. It does appear the original user is being used to authenticate, maybe, at the very least it is being offered in the imapc connection, but not as a master user.
To illustrate, I have two snippets of logs below.
The first is from the shared folders instance on port 9993 after logging into the primary server on port 993, which triggers the imapc connection. This shows the user bob.test, who is the imapc_master_user as defined by %u, being acknowledged and then having the username changed to that of my shared user, in this case authapps. Further down, we can see that the auth_user is bob.test, but there is no master user, and the acl username at the bottom of the snipped remains that of my shared user.
Below that, I have the same snippet of log, as obtained by connecting with openssl s_client to the shared instance on port 9993, with the string:
a login authapps*bob.test XXXXXXX
In order to get this working, I need to add to the shared instance's config file a static userdb like so:
userdb { driver = static }
this snippet shows the masteruser working as expected, and the acl username being that of the master user. Unfortunately, it has no apparent effect on the imapc login.
So it seems master user logins work, just not through imapc.
I note at the top of the first snippet, before the change in username, it appears the imapc connection is being directed to the static passdb, which I think is triggering the username change, where as the second snippet, the connection is starting with the ldap passdb. but no amount of playing "configuration file options" chairs has led me to a working solution.
I am hoping someone can provide some insight as to what I am missing here.
doveconf -n for both instances are pasted below the log snippets. If you are still reading, you have my gratitude for your attention this far...
2017-09-27 13:20:13.725943500 Sep 27 13:20:13 auth: Debug: static(bob.test,192.168.120.70,<y0vvhDFaYqDAqHhG>): lookup 2017-09-27 13:20:13.725960500 Sep 27 13:20:13 auth: Debug: static(bob.test,192.168.120.70,<y0vvhDFaYqDAqHhG>): username changed bob.test -> authapps 2017-09-27 13:20:13.726011500 Sep 27 13:20:13 auth: Debug: client passdb out: OK 1 user=authapps original_user=bob.test 2017-09-27 13:20:13.728299500 Sep 27 13:20:13 auth: Debug: master in: REQUEST 2523398145 13259 1 ba830ae1c199401e5dbee28e31025ac1 session_pid=13335 request_auth_token 2017-09-27 13:20:13.728344500 Sep 27 13:20:13 auth: Debug: master userdb out: USER 2523398145 authapps uid=509 gid=509 home=/CTFN/SharedMailboxes/CTFN/ auth_token=5e3d02714e82441cbbb3de8ff80f35bc3dc4b291 auth_user=bob.test 2017-09-27 13:20:13.728459500 Sep 27 13:20:13 imap-login: Info: Login: user=<authapps>, method=PLAIN, rip=192.168.120.70, lip=192.168.120.70, mpid=13335, TLS, session=<y0vvhDFaYqDAqHhG> 2017-09-27 13:20:13.729092500 Sep 27 13:20:13 imap(authapps): Debug: Loading modules from directory: /usr/local/lib/dovecot 2017-09-27 13:20:13.729781500 Sep 27 13:20:13 imap(authapps): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so 2017-09-27 13:20:13.729841500 Sep 27 13:20:13 imap(authapps): Debug: Module loaded: /usr/local/lib/dovecot/lib02_imap_acl_plugin.so 2017-09-27 13:20:13.730026500 Sep 27 13:20:13 imap(authapps): Debug: Effective uid=509, gid=509, home=/CTFN/SharedMailboxes/CTFN/ 2017-09-27 13:20:13.732014500 Sep 27 13:20:13 imap(authapps): Debug: Namespace CTFN: type=public, prefix=CTFN/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/CTFN/SharedMailboxes/ 2017-09-27 13:20:13.732050500 Sep 27 13:20:13 imap(authapps): Debug: maildir++: root=/CTFN/SharedMailboxes, index=, indexpvt=, control=, inbox=/CTFN/SharedMailboxes, alt= 2017-09-27 13:20:13.732053500 Sep 27 13:20:13 imap(authapps): Debug: acl: initializing backend with data: vfile 2017-09-27 13:20:13.732055500 Sep 27 13:20:13 imap(authapps): Debug: acl: acl username = authapps
2017-09-27 18:33:37.008636500 Sep 27 18:33:37 auth: Debug: ldap(bob.test,192.168.120.50,master,<y56y5TVaAYHAqHgy>): Master user lookup for login: authapps 2017-09-27 18:33:37.021400500 Sep 27 18:33:37 auth: Info: ldap(bob.test,192.168.120.50,master,<y56y5TVaAYHAqHgy>): Master user logging in as authapps 2017-09-27 18:33:37.021425500 Sep 27 18:33:37 auth: Debug: client passdb out: OK 1 user=authapps original_user=bob.test auth_user=bob.test 2017-09-27 18:33:37.023658500 Sep 27 18:33:37 auth: Debug: master in: REQUEST 2776498177 9162 1 2d247ac65c17769575edb6c9db86fdee session_pid=27552 request_auth_token 2017-09-27 18:33:37.023703500 Sep 27 18:33:37 auth: Debug: ldap(authapps,192.168.120.50,<y56y5TVaAYHAqHgy>): user search: base=CN=Users,dn=ctfn,dn=ca scope=onelevel filter=(&(sAMAccountName=authapps)) fields=homeDirectory,uidNumber,gidNumber 2017-09-27 18:33:37.026953500 Sep 27 18:33:37 auth: Error: ldap(authapps,192.168.120.50,<y56y5TVaAYHAqHgy>): ldap_search(base=CN=Users,dn=ctfn,dn=ca filter=(&(sAMAccountName=authapps))) failed: Operations error 2017-09-27 18:33:37.026984500 Sep 27 18:33:37 auth: Debug: master userdb out: USER 2776498177 authapps uid=509 gid=509 home=/CTFN/SharedMailboxes/CTFN/ master_user=bob.test auth_token=12961a562e9ef0698b363714d758d95cea9ff3f2 auth_user=bob.test 2017-09-27 18:33:37.027163500 Sep 27 18:33:37 imap-login: Info: Login: user=<authapps>, method=PLAIN, rip=192.168.120.50, lip=192.168.120.70, mpid=27552, TLS, session=<y56y5TVaAYHAqHgy> 2017-09-27 18:33:37.027826500 Sep 27 18:33:37 imap(authapps): Debug: Loading modules from directory: /usr/local/lib/dovecot 2017-09-27 18:33:37.028479500 Sep 27 18:33:37 imap(authapps): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so 2017-09-27 18:33:37.028561500 Sep 27 18:33:37 imap(authapps): Debug: Module loaded: /usr/local/lib/dovecot/lib02_imap_acl_plugin.so 2017-09-27 18:33:37.028607500 Sep 27 18:33:37 imap(authapps): Debug: Added userdb setting: plugin/master_user=bob.test 2017-09-27 18:33:37.028685500 Sep 27 18:33:37 imap(authapps): Debug: Effective uid=509, gid=509, home=/CTFN/SharedMailboxes/CTFN/ 2017-09-27 18:33:37.030993500 Sep 27 18:33:37 imap(authapps): Debug: Namespace CTFN: type=public, prefix=CTFN/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/CTFN/SharedMailboxes/CTFN 2017-09-27 18:33:37.031019500 Sep 27 18:33:37 imap(authapps): Debug: maildir++: root=/CTFN/SharedMailboxes/CTFN, index=, indexpvt=, control=, inbox=/CTFN/SharedMailboxes/CTFN, alt= 2017-09-27 18:33:37.031040500 Sep 27 18:33:37 imap(authapps): Debug: acl: initializing backend with data: vfile 2017-09-27 18:33:37.031047500 Sep 27 18:33:37 imap(authapps): Debug: acl: acl username = bob.test
PRIMARY SERVER ON PORT 993: doveconf -i dovecotserver -n # 2.2.31 (65cde28): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.19 (e5c7051) # OS: Linux 4.9.0-3-amd64 x86_64 Debian 9.1 auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_verbose_passwords = plain hostname = imap.ctfn.ca imapc_host = masterchieflian.ctfn.ca imapc_master_user = %u imapc_password = # hidden, use -P to show it imapc_port = 9993 imapc_ssl = imaps instance_name = dovecotserver lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = 192.168.120.70 log_path = /dev/stderr login_greeting = CTFN IMAP server mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = stats quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vacation-seconds mmap_disable = yes namespace CTFN { list = yes location = imapc:/CTFN/mail/CTFN:INDEXPVT=/CTFN/mail/%n/Maildir/CTFN prefix = CTFN/ separator = / subscriptions = no type = public } namespace inbox { inbox = yes location = mailbox Archives { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox LearnAsSpam { auto = subscribe } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap master = yes } plugin { acl_shared_dict = fs:posix:prefix=/CTFN/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size vsize from subject flags sieve = ~/.dovecot.sieve sieve_default = /usr/local/lib/dovecot/default.sieve sieve_dir = ~/sieve sieve_extensions = +vacation +vacation-seconds sieve_max_actions = 92 sieve_max_redirects = 24 sieve_max_script_size = 10M sieve_quota_max_scripts = 20 sieve_quota_max_storage = 200000 } protocols = imap sieve service auth { client_limit = 1024 unix_listener auth-userdb { group = vmail mode = 0660 user = dovecot } } service doveadm { inet_listener { address = 192.168.120.70 port = 9092 } } service imap-login { inet_listener imap { address = 127.0.0.1 port = 143 ssl = no } inet_listener imaps { address = 192.168.120.70 port = 993 ssl = yes } process_min_avail = 5 service_count = 0 } service managesieve-login { inet_listener sieve-local { address = 127.0.0.1 port = 4190 } inet_listener sieve { address = 192.168.120.70 port = 4190 } process_min_avail = 5 service_count = 1 vsz_limit = 64 M } ssl = required ssl_ca = /var/CA/ctfn.ca/RapidSSLWildcard/2017/geotrust.intermediate.pem ssl_cert =
SHARED SERVER ON PORT 9993 doveconf -i dovshare -n # 2.2.31 (65cde28): /usr/local/etc/dovecot/dovshare.conf # Pigeonhole version 0.4.19 (e5c7051) # OS: Linux 4.9.0-3-amd64 x86_64 Debian 9.1 auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_verbose_passwords = plain base_dir = /usr/local/var/run/dovshare/ hostname = imapshare.ctfn.ca instance_name = dovshare listen = 192.168.120.70 log_path = /dev/stderr login_greeting = CTFN IMAP Shared Mailboxes mail_debug = yes mail_fsync = always mail_gid = vmail mail_location = maildir:~ mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = stats quota acl mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mmap_disable = yes namespace CTFN { inbox = yes location = maildir:/CTFN/SharedMailboxes/CTFN prefix = CTFN/ separator = / type = public } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap master = yes } passdb { args = user=authapps password=XXXXXXXXXX driver = static } plugin { acl = vfile acl_shared_dict = fs:posix:prefix=/CTFN/Config/dovecot/dovecot.acls/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size vsize from subject flags } protocols = imap service auth { client_limit = 1024 unix_listener auth-userdb { group = vmail mode = 0660 user = dovecot } } service imap-login { inet_listener imaps { address = 192.168.120.70 port = 9993 ssl = yes } process_min_avail = 5 service_count = 0 } ssl = required ssl_ca =
-- Bob Miller Cell: 867-334-7117 Office: 867-633-3760 www.computerisms.ca
On 28 Sep 2017, at 20.50, Computerisms Corporation bob@computerisms.ca wrote:
Hi,
My end goal is to set up shared mailboxes on a cluster as per:
https://wiki.dovecot.org/SharedMailboxes/ClusterSetup
I was having very little luck with it, so I had been trying to break down into pieces and get individual components working. So I have things setup on a single server, with a working dovecot instance. I have no director or any thing else running yet, and I am just trying to get the imapc portion of things working. Eventually I came across a post from Sven Hartge:
https://www.dovecot.org/list/dovecot/2012-November/087046.html
This is from several years ago, but I set up a separate dovecot instance on port 9993 specifically to run the shared folders as he describes, and copied his configuration suggestions pretty much verbatim. The only signficant difference is that I needed to not configure the imapc_user as he had done, and instead as the wiki suggests remove that config option entirely. With this, the imapc connection is being made from the primary instance on port 993 to the shared instance on port 9993.
yes you do need to define imapc_user if you want to switch user and master user around for imapc. like:
imapc_user = authapps imapc_master_user = %u
without imapc_user dovecot would login to the secondary server like A bob.test*bob.test password
SHARED SERVER ON PORT 9993
passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap master = yes } passdb { args = user=authapps password=XXXXXXXXXX driver = static }
This is probably correct except that now user authapps can also login directly with password XXXXXXXXXX without master user login. Which is probably OK as long as the password is kept secret
Sami
Hi Sami,
Thank you for your reply.
yes you do need to define imapc_user if you want to switch user and master user around for imapc. like:
imapc_user = authapps imapc_master_user = %u
without imapc_user dovecot would login to the secondary server like A bob.test*bob.test password
When I add the 'imapc_user = authapps' option to the primary instance running on port 993, it passes the imapc_password as the password for the master user. here is what the logs on the shared instance show when I log into the primary instance:
2017-09-28 12:47:35.361064500 Sep 28 12:47:35 auth: Debug:
ldap(bob.test,192.168.120.70,master,
The given password in the logs is the password supplied as imapc_password.
if I change the imapc_password to be that of the bob.test user, the imapc login does succeed with bob.test as the master user.
I followed this before, and it seemed the solution was to set
imapc_password = %w
However, dovecot will not start with this configuration.
In Sven's email, he places the imapc_password = %w as a default_field in the userdb on the primary instance. If I remove the default field, I get the invalid credentials reported as above. However, I still think this is correct way to pass %w, because if I remove 'imapc_user = authapps' from the global config *and* 'imapc_password = %w' from the default_fields in the userdb, the logs on the shared instance show that the user password is not being passed in the imapc login:
2017-09-28 12:57:10.409884500 Sep 28 12:57:10 auth: Debug: static(bob.test,192.168.120.70,<rvFSUEVaxLfAqHhG>): lookup 2017-09-28 12:57:10.409903500 Sep 28 12:57:10 auth: Debug: static(bob.test,192.168.120.70,<rvFSUEVaxLfAqHhG>): username changed bob.test -> authapps 2017-09-28 12:57:10.409905500 Sep 28 12:57:10 auth: Info: static(authapps,192.168.120.70,<rvFSUEVaxLfAqHhG>): No password returned (and no nopassword) 2017-09-28 12:57:12.412437500 Sep 28 12:57:12 auth: Debug: client passdb out: FAIL 11 user=authapps original_user=bob.test
so maybe I am not passing the %w in the correct spot?
truly appreciate you taking a look at this, thank you.
passdb { args = user=authapps password=XXXXXXXXXX driver = static }
This is probably correct except that now user authapps can also login directly with password XXXXXXXXXX without master user login. Which is probably OK as long as the password is kept secret
Once I get the imapc master user sorted out, I will play with the config to see what I can trim, but as of now, dropping just the password, or the entire args line, also causes the imapc login to fail...
truly appreciate you taking a look at this, thank you.
Sami
On 28 Sep 2017, at 23.08, Computerisms Corporation bob@computerisms.ca wrote:
Hi Sami,
Thank you for your reply.
yes you do need to define imapc_user if you want to switch user and master user around for imapc. like: imapc_user = authapps imapc_master_user = %u without imapc_user dovecot would login to the secondary server like A bob.test*bob.test password
When I add the 'imapc_user = authapps' option to the primary instance running on port 993, it passes the imapc_password as the password for the master user. here is what the logs on the shared instance show when I log into the primary instance:
2017-09-28 12:47:35.361064500 Sep 28 12:47:35 auth: Debug: ldap(bob.test,192.168.120.70,master,
): Master user lookup for login: authapps 2017-09-28 12:47:35.364892500 Sep 28 12:47:35 auth: Info: ldap(bob.test,192.168.120.70,master, ): invalid credentials (given password: XXXXXXXXX) 2017-09-28 12:47:37.367173500 Sep 28 12:47:37 auth: Debug: client passdb out: FAIL 2 user=bob.test The given password in the logs is the password supplied as imapc_password.
if I change the imapc_password to be that of the bob.test user, the imapc login does succeed with bob.test as the master user.
I followed this before, and it seemed the solution was to set
imapc_password = %w
However, dovecot will not start with this configuration.
In Sven's email, he places the imapc_password = %w as a default_field in the userdb on the primary instance. If I remove the default field, I get the invalid credentials reported as above. However, I still think this is correct way to pass %w, because if I remove 'imapc_user = authapps' from the global config *and* 'imapc_password = %w' from the default_fields in the userdb, the logs on the shared instance show that the user password is not being passed in the imapc login:
2017-09-28 12:57:10.409884500 Sep 28 12:57:10 auth: Debug: static(bob.test,192.168.120.70,<rvFSUEVaxLfAqHhG>): lookup 2017-09-28 12:57:10.409903500 Sep 28 12:57:10 auth: Debug: static(bob.test,192.168.120.70,<rvFSUEVaxLfAqHhG>): username changed bob.test -> authapps 2017-09-28 12:57:10.409905500 Sep 28 12:57:10 auth: Info: static(authapps,192.168.120.70,<rvFSUEVaxLfAqHhG>): No password returned (and no nopassword) 2017-09-28 12:57:12.412437500 Sep 28 12:57:12 auth: Debug: client passdb out: FAIL 11 user=authapps original_user=bob.test
so maybe I am not passing the %w in the correct spot?
truly appreciate you taking a look at this, thank you.
Now this starts to be a bit complex. Not sure if you can get this working by returning imapc_password = %w from the first passdb. Atleast if it works it will only work with PLAIN auth scheme. What I would do here is to just trust that the user is already authenticated with the first ldap passdb in the primary server and then switch the imapc connection to both master user and master password.
So just put imapc_password=masterpassword in dovecot.conf of the primary server and on secondary server modify ldap config not to fetch the user password but always return password=masterpassword.
and maybe protect the authentication scheme with allow_nets=127.0.0.1 from external abuse
Sami
Hi Sami
I followed this before, and it seemed the solution was to set
imapc_password = %w
However, dovecot will not start with this configuration.
In Sven's email, he places the imapc_password = %w as a default_field in the userdb on the primary instance. If I remove the default field, I get the invalid credentials reported as above. However, I still think this is correct way to pass %w, because if I remove 'imapc_user = authapps' from the global config *and* 'imapc_password = %w' from the default_fields in the userdb, the logs on the shared instance show that the user password is not being passed in the imapc login:
2017-09-28 12:57:10.409884500 Sep 28 12:57:10 auth: Debug: static(bob.test,192.168.120.70,<rvFSUEVaxLfAqHhG>): lookup 2017-09-28 12:57:10.409903500 Sep 28 12:57:10 auth: Debug: static(bob.test,192.168.120.70,<rvFSUEVaxLfAqHhG>): username changed bob.test -> authapps 2017-09-28 12:57:10.409905500 Sep 28 12:57:10 auth: Info: static(authapps,192.168.120.70,<rvFSUEVaxLfAqHhG>): No password returned (and no nopassword) 2017-09-28 12:57:12.412437500 Sep 28 12:57:12 auth: Debug: client passdb out: FAIL 11 user=authapps original_user=bob.test
Now this starts to be a bit complex. Not sure if you can get this working by returning imapc_password = %w from the first passdb. Atleast if it works it will only work with PLAIN auth scheme. What I would do here is to just trust that the user is already authenticated with the first ldap passdb in the primary server and then switch the imapc connection to both master user and master password.
So just put imapc_password=masterpassword in dovecot.conf of the primary server and on secondary server modify ldap config not to fetch the user password but always return password=masterpassword.
It took me a while to wrap my head around what you are saying here, but once I got it I find it a simple, elegant and absolutely brilliant solution.
In thinking about it, it occurred to me I could maybe do the same without even doing an ldap look up, so I changed the userdb/passdb stanzas on the shared instance like so:
userdb { args = uid=vmail gid=vmail home=/CTFN/SharedMailboxes/CTFN/ driver = static } passdb { args = user=%u password=XXXXXXX driver = static master = yes } passdb { driver = static }
Where the password is the value set for imapc_password on the primary instance.
The logs now show on the shared server that the user is authapps, and it has the correct effective uid/gid/home values, and the master user and the acl username is that of the user logging into the primary instance.
So looks like this will work!
Thank you, Sami, a bottle of Canada's finest is at your disposal...
and maybe protect the authentication scheme with allow_nets=127.0.0.1 from external abuse
Sami
participants (2)
-
Computerisms Corporation
-
Sami Ketola