Hi,
Are configurations (with separate formats per namespace) - such as ...
namespace { type = shared list = children inbox = no separator = / subscriptions = no prefix = shared1/%%n/ location = maildir:/var/mail1/%%n/ }
namespace { type = shared list = children inbox = no separator = / subscriptions = no prefix = shared2/%%n/ location = mdbox:/var/mail2/%%n/ }
With separate userdbs serving part of the users with maildirs in /var/mail1/ and the other part with mdboxes in /var/mail2
... valid in dovecot ?
To be more precise:
- Is current version dovecot expected to work with configuration as above ? E.g. if some user's mail location returned from userdb doesn't match location/format from one of the above namespaces - would it be ignored for it ? A very old version of dovecot I could check quickly (2.1.7) was segfaulting (imap processes) all the time with this kind of config. While I'll be upgrading it and the whole system to modern versions, I'm wondering if this kind of thing is formally allowed at all.
An interesting variation of the above setup I tested - with the second namespace "un-variabled" and pointing to single user (with matching passwd-file returning that user) managed to work somehow - but the user itself was still created on the fly for the 1st namespace - having just a directory with empty dovecot-acl-list file. This essentially seemed to have worked like a typical public profile (shared acl db didn't seem to be used either)
- For shared namespace with variables, would overriding shared namespace location in userdb query work ? For example if we had single namespace such as the first one (maildir) above with explicit share1 name, and then in userdb for some users:
userdb_namespace/share1/location=mdbox:/var/mail2/some_user_name
Would that override a shared namespace pattern on per user basis correctly ?
Is there perhaps a way to constraint which userdbs are considered per which shared namespace ?
Not strictly related to the above, but in LDAP part of documentation - namely http://wiki2.dovecot.org/AuthDatabase/LDAP/Userdb - why
user_attrs =
=home=%{ldap:homeDirectory},
=uid=%{ldap:uidNumber},
=gid=%{ldap:gidNumber}
instead of
user_attrs =
homeDirectory=home,
uidNumber=uid,
gidNumber=gid
Do both are equivalent with differet syntax (opening the way for templates for %{} based syntax), or are there some subtle differences between those ? (aside later example with 'attr=' instead of just '=').
Would this kind of syntax also be correct for pass_attrs, such as:
pass_attrs =
=password=%{ldap:uid},
=userdb_home=%{ldap:homeDirectory}
Yeah, that's valid configuration. As long as they have unique prefix.
Aki
On January 2, 2017 at 5:58 PM Michal Soltys soltys@ziu.info wrote:
Hi,
Are configurations (with separate formats per namespace) - such as ...
namespace { type = shared list = children inbox = no separator = / subscriptions = no prefix = shared1/%%n/ location = maildir:/var/mail1/%%n/ }
namespace { type = shared list = children inbox = no separator = / subscriptions = no prefix = shared2/%%n/ location = mdbox:/var/mail2/%%n/ }
With separate userdbs serving part of the users with maildirs in /var/mail1/ and the other part with mdboxes in /var/mail2
... valid in dovecot ?
To be more precise:
- Is current version dovecot expected to work with configuration as above ? E.g. if some user's mail location returned from userdb doesn't match location/format from one of the above namespaces - would it be ignored for it ? A very old version of dovecot I could check quickly (2.1.7) was segfaulting (imap processes) all the time with this kind of config. While I'll be upgrading it and the whole system to modern versions, I'm wondering if this kind of thing is formally allowed at all.
An interesting variation of the above setup I tested - with the second namespace "un-variabled" and pointing to single user (with matching passwd-file returning that user) managed to work somehow - but the user itself was still created on the fly for the 1st namespace - having just a directory with empty dovecot-acl-list file. This essentially seemed to have worked like a typical public profile (shared acl db didn't seem to be used either)
- For shared namespace with variables, would overriding shared namespace location in userdb query work ? For example if we had single namespace such as the first one (maildir) above with explicit share1 name, and then in userdb for some users:
userdb_namespace/share1/location=mdbox:/var/mail2/some_user_name
Would that override a shared namespace pattern on per user basis correctly ?
Is there perhaps a way to constraint which userdbs are considered per which shared namespace ?
Not strictly related to the above, but in LDAP part of documentation - namely http://wiki2.dovecot.org/AuthDatabase/LDAP/Userdb - why
user_attrs =
=home=%{ldap:homeDirectory},
=uid=%{ldap:uidNumber},
=gid=%{ldap:gidNumber}instead of
user_attrs =
homeDirectory=home,
uidNumber=uid,
gidNumber=gidDo both are equivalent with differet syntax (opening the way for templates for %{} based syntax), or are there some subtle differences between those ? (aside later example with 'attr=' instead of just '=').
Would this kind of syntax also be correct for pass_attrs, such as:
pass_attrs =
=password=%{ldap:uid},
=userdb_home=%{ldap:homeDirectory}
On January 2, 2017 at 5:58 PM Michal Soltys soltys@ziu.info wrote:
Hi,
Are configurations (with separate formats per namespace) - such as ...
namespace { type = shared list = children inbox = no separator = / subscriptions = no prefix = shared1/%%n/ location = maildir:/var/mail1/%%n/ }
namespace { type = shared list = children inbox = no separator = / subscriptions = no prefix = shared2/%%n/ location = mdbox:/var/mail2/%%n/ }
With separate userdbs serving part of the users with maildirs in /var/mail1/ and the other part with mdboxes in /var/mail2
... valid in dovecot ?
To be more precise:
- Is current version dovecot expected to work with configuration as above ? E.g. if some user's mail location returned from userdb doesn't match location/format from one of the above namespaces - would it be ignored for it ? A very old version of dovecot I could check quickly (2.1.7) was segfaulting (imap processes) all the time with this kind of config. While I'll be upgrading it and the whole system to modern versions, I'm wondering if this kind of thing is formally allowed at all.
An interesting variation of the above setup I tested - with the second namespace "un-variabled" and pointing to single user (with matching passwd-file returning that user) managed to work somehow - but the user itself was still created on the fly for the 1st namespace - having just a directory with empty dovecot-acl-list file. This essentially seemed to have worked like a typical public profile (shared acl db didn't seem to be used either)
On 2017-01-02 19:21, Aki Tuomi wrote:
Yeah, that's valid configuration. As long as they have unique prefix.
Aki
Well, I retested it under 2.2.27 - and the behaviour is essentially the same (segfaults).
Below is the simplified configuration under which it can be observed with 2 passwd-files (each with 1 user, passwords removed to save space)
passwd-file local-mdbox: nmm:{SHA256}<cut>:::nmm:/var/mail2/nmm::userdb_mail=mdbox:/var/mail2/nmm userdb_home=/var/mail2/nmm
passwd-file local-maildir: msl:{SHA256}<cut>:::msl:/var/mail/msl::userdb_mail=maildir:/var/mail/msl userdb_home=/var/mail/msl
Both of the accounts have some mails/subfolders, nmm is sharing some of its contents to msl.
doveconf -n (note thare are some leftovers from old configuration - particularly weird last/first uids and mail_uid using dovecot user - but those are not relevant to the issue):
# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # OS: Linux 4.8.13-1-ARCH x86_64 ext4 auth_debug = yes auth_mechanisms = plain login disable_plaintext_auth = no first_valid_gid = 8 first_valid_uid = 105 last_valid_gid = 8 last_valid_uid = 105 listen = * log_path = /var/log/dovecot.log mail_access_groups = mail mail_debug = yes mail_gid = mail mail_location = maildir:/var/mail/%n mail_plugins = acl mail_uid = dovecot namespace { inbox = yes location = prefix = separator = / type = private } namespace share1 { inbox = no list = children location = maildir:%%h prefix = shared1/%%n/ separator = / subscriptions = no type = shared } namespace share2 { inbox = no list = children location = mdbox:%%h prefix = shared2/%%n/ separator = / subscriptions = no type = shared } passdb { args = username_format=%n /etc/dovecot/local-maildir default_fields = userdb_uid=dovecot userdb_gid=mail driver = passwd-file } passdb { args = username_format=%n /etc/dovecot/local-mdbox default_fields = userdb_uid=dovecot userdb_gid=mail driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/var/mail/shared-database/shared-mailboxes.db } protocols = imap service auth { unix_listener auth-userdb { group = mail mode = 0660 user = dovecot } user = dovecot } service imap-login { inet_listener imap { port = 0 } user = dovecot } service imap { executable = /usr/lib/dovecot/imap } userdb { driver = prefetch } userdb { args = username_format=%n /etc/dovecot/local-maildir default_fields = uid=dovecot gid=mail driver = passwd-file } userdb { args = username_format=%n /etc/dovecot/local-mdbox default_fields = uid=dovecot gid=mail driver = passwd-file } protocol imap { mail_max_userip_connections = 100 mail_plugins = acl imap_acl }
With the configuration and 2 passwd-files as above, all imap processes (when logged as user msl) constantly crash with segfaults. Replacing %%h by template such as /var/mail/%%n (as in my initial report) behaves the same way.
Now - IF share1 namespace is commented out or removed - everything works fine (and msl sees content shared by nmm under shared2/nmm/ ). Similarly - if only one shared namespace uses variables and the other points directly to some user - no crashes then.
Any ideas ?
I can get systraces/cores (though the latter without debug symbols - but I can recompile if need be).
I think I've found the reason behind those crashes as in the configuration posted in the earlier mail.
First the full backtrace:
Message: Process 13965 (imap) of user 105 dumped core.
Stack trace of thread 13965:
#0 0x00007fbdaa15929a __strcmp_sse2_unaligned (libc.so.6)
#1 0x00007fbdaa820c50 mail_storage_match_class (libdovecot-storage.so.0)
#2 0x00007fbdaa820cbf mail_storage_find (libdovecot-storage.so.0)
#3 0x00007fbdaa82103d mail_storage_create_full (libdovecot-storage.so.0)
#4 0x00007fbdaa8212c2 mail_storage_create (libdovecot-storage.so.0)
#5 0x00007fbdaa816cdc mail_namespaces_init_add (libdovecot-storage.so.0)
#6 0x00007fbdaa817694 mail_namespaces_init (libdovecot-storage.so.0)
#7 0x00007fbdaa82a4cd mail_storage_service_init_post (libdovecot-storage.so.0)
#8 0x00007fbdaa82c266 mail_storage_service_next_real (libdovecot-storage.so.0)
#9 0x00007fbdaa82c321 mail_storage_service_next (libdovecot-storage.so.0)
#10 0x00007fbdaa82c49a mail_storage_service_lookup_next (libdovecot-storage.so.0)
#11 0x00000000004314f0 client_create_from_input (imap)
#12 0x0000000000431968 login_client_connected (imap)
#13 0x00007fbdaa49b1c1 master_login_auth_finish (libdovecot.so.0)
#14 0x00007fbdaa49baca master_login_auth_callback (libdovecot.so.0)
#15 0x00007fbdaa49cae9 master_login_auth_input_user (libdovecot.so.0)
#16 0x00007fbdaa49cfb1 master_login_auth_input (libdovecot.so.0)
#17 0x00007fbdaa54a545 io_loop_call_io (libdovecot.so.0)
#18 0x00007fbdaa54ce68 io_loop_handler_run_internal (libdovecot.so.0)
#19 0x00007fbdaa54a726 io_loop_handler_run (libdovecot.so.0)
#20 0x00007fbdaa54a649 io_loop_run (libdovecot.so.0)
#21 0x00007fbdaa49ee3b master_service_run (libdovecot.so.0)
#22 0x0000000000431efb main (imap)
#23 0x00007fbdaa0ea291 __libc_start_main (libc.so.6)
#24 0x000000000040c6da _start (imap)
GNU gdb (GDB) 7.12
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib/dovecot/imap...done.
[New LWP 13965]
Core was generated by `dovecot/imap'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fbdaa15929a in __strcmp_sse2_unaligned () from /usr/lib/libc.so.6
(gdb) up
#1 0x00007fbdaa820c50 in mail_storage_match_class (storage=0xdc5800, storage_class=0x7fbdaab715c0
So the first argument passed to strcmp() was NULL. The offending part of code is:
if ((storage->class_flags & MAIL_STORAGE_CLASS_FLAG_UNIQUE_ROOT) != 0 && strcmp(storage->unique_root_dir, (set->root_dir != NULL ? set->root_dir : "")) != 0) return FALSE;
The 2nd argument is sanitized explicitly, but the first is not - and apparently it can be NULL as well. Adding same check to the 1st argument stopped segfaults and both shared namespaces seemed to be working correctly - so user 'msl' could see/subscribe shared folders from user 'nnm' and vice versa (both mailboxes being of different format). While this was enough here, something else might be needed to make it fully correct (the original strcmp() invocation would suggest that the 1st argument should never be NULL).
Configurations with multiple shared namespaces can trigger a bug where the first argument of strcmp() invocation is NULL. This patch adds an explicit check, analogously to how the second argument is sanitized. --- src/lib-storage/mail-storage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib-storage/mail-storage.c b/src/lib-storage/mail-storage.c index 1d9b1bf..3d9f5dc 100644 --- a/src/lib-storage/mail-storage.c +++ b/src/lib-storage/mail-storage.c @@ -282,7 +282,7 @@ mail_storage_match_class(struct mail_storage *storage, return FALSE; if ((storage->class_flags & MAIL_STORAGE_CLASS_FLAG_UNIQUE_ROOT) != 0 && - strcmp(storage->unique_root_dir, + strcmp((storage->unique_root_dir != NULL ? storage->unique_root_dir : ""), (set->root_dir != NULL ? set->root_dir : "")) != 0) return FALSE; -- 2.1.3
On 10 Jan 2017, at 21.58, Michal Soltys soltys@ziu.info wrote:
Configurations with multiple shared namespaces can trigger a bug where the first argument of strcmp() invocation is NULL.
This patch adds an explicit check, analogously to how the second argument is sanitized.
I think it shouldn't be NULL though.. I'd rather add some asserts and figure out why it is. I guess the attached patch assert-crashes? What's the backtrace there?
On 01/10/2017 09:31 PM, Timo Sirainen wrote:
On 10 Jan 2017, at 21.58, Michal Soltys soltys@ziu.info wrote:
Configurations with multiple shared namespaces can trigger a bug where the first argument of strcmp() invocation is NULL.
This patch adds an explicit check, analogously to how the second argument is sanitized.
I think it shouldn't be NULL though.. I'd rather add some asserts and figure out why it is. I guess the attached patch assert-crashes? What's the backtrace there?
Yea, assert triggers instantly once I try to read any folder. bt full below
#2 0x00007f1b92c53727 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:201
backtrace = 0x971fb0 "/usr/lib/dovecot/libdovecot.so.0(+0xc36d8) [0x7f1b92c536d8] -> /usr/lib/dovecot/libdovecot.so.0(+0xc4c06) [0x7f1b92c54c06] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f1b92c53a5b] -> /usr/lib/d"...
#3 0x00007f1b92c54c06 in i_internal_fatal_handler (ctx=0x7ffdee3f6fe0, format=0x7f1b93043e68 "file %s: line %d (%s): assertion failed: (%s)", args=0x7ffdee3f7000) at failures.c:670
status = 0
#4 0x00007f1b92c53a5b in i_panic (format=0x7f1b93043e68 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275
ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0}
args =
participants (3)
-
Aki Tuomi
-
Michal Soltys
-
Timo Sirainen