Hi,
Are configurations (with separate formats per namespace) - such as ...
namespace { type = shared list = children inbox = no separator = / subscriptions = no prefix = shared1/%%n/ location = maildir:/var/mail1/%%n/ }
namespace { type = shared list = children inbox = no separator = / subscriptions = no prefix = shared2/%%n/ location = mdbox:/var/mail2/%%n/ }
With separate userdbs serving part of the users with maildirs in /var/mail1/ and the other part with mdboxes in /var/mail2
... valid in dovecot ?
To be more precise:
- Is current version dovecot expected to work with configuration as above ? E.g. if some user's mail location returned from userdb doesn't match location/format from one of the above namespaces - would it be ignored for it ? A very old version of dovecot I could check quickly (2.1.7) was segfaulting (imap processes) all the time with this kind of config. While I'll be upgrading it and the whole system to modern versions, I'm wondering if this kind of thing is formally allowed at all.
An interesting variation of the above setup I tested - with the second namespace "un-variabled" and pointing to single user (with matching passwd-file returning that user) managed to work somehow - but the user itself was still created on the fly for the 1st namespace - having just a directory with empty dovecot-acl-list file. This essentially seemed to have worked like a typical public profile (shared acl db didn't seem to be used either)
- For shared namespace with variables, would overriding shared namespace location in userdb query work ? For example if we had single namespace such as the first one (maildir) above with explicit share1 name, and then in userdb for some users:
userdb_namespace/share1/location=mdbox:/var/mail2/some_user_name
Would that override a shared namespace pattern on per user basis correctly ?
Is there perhaps a way to constraint which userdbs are considered per which shared namespace ?
Not strictly related to the above, but in LDAP part of documentation - namely http://wiki2.dovecot.org/AuthDatabase/LDAP/Userdb - why
user_attrs =
=home=%{ldap:homeDirectory},
=uid=%{ldap:uidNumber},
=gid=%{ldap:gidNumber}
instead of
user_attrs =
homeDirectory=home,
uidNumber=uid,
gidNumber=gid
Do both are equivalent with differet syntax (opening the way for templates for %{} based syntax), or are there some subtle differences between those ? (aside later example with 'attr=' instead of just '=').
Would this kind of syntax also be correct for pass_attrs, such as:
pass_attrs =
=password=%{ldap:uid},
=userdb_home=%{ldap:homeDirectory}
On January 2, 2017 at 5:58 PM Michal Soltys <soltys@ziu.info> wrote:
Well, I retested it under 2.2.27 - and the behaviour is essentially the same (segfaults).
Below is the simplified configuration under which it can be observed with 2 passwd-files (each with 1 user, passwords removed to save space)
passwd-file local-mdbox: nmm:{SHA256}<cut>:::nmm:/var/mail2/nmm::userdb_mail=mdbox:/var/mail2/nmm userdb_home=/var/mail2/nmm
passwd-file local-maildir: msl:{SHA256}<cut>:::msl:/var/mail/msl::userdb_mail=maildir:/var/mail/msl userdb_home=/var/mail/msl
Both of the accounts have some mails/subfolders, nmm is sharing some of its contents to msl.
doveconf -n (note thare are some leftovers from old configuration - particularly weird last/first uids and mail_uid using dovecot user - but those are not relevant to the issue):
# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # OS: Linux 4.8.13-1-ARCH x86_64 ext4 auth_debug = yes auth_mechanisms = plain login disable_plaintext_auth = no first_valid_gid = 8 first_valid_uid = 105 last_valid_gid = 8 last_valid_uid = 105 listen = * log_path = /var/log/dovecot.log mail_access_groups = mail mail_debug = yes mail_gid = mail mail_location = maildir:/var/mail/%n mail_plugins = acl mail_uid = dovecot namespace { inbox = yes location = prefix = separator = / type = private } namespace share1 { inbox = no list = children location = maildir:%%h prefix = shared1/%%n/ separator = / subscriptions = no type = shared } namespace share2 { inbox = no list = children location = mdbox:%%h prefix = shared2/%%n/ separator = / subscriptions = no type = shared } passdb { args = username_format=%n /etc/dovecot/local-maildir default_fields = userdb_uid=dovecot userdb_gid=mail driver = passwd-file } passdb { args = username_format=%n /etc/dovecot/local-mdbox default_fields = userdb_uid=dovecot userdb_gid=mail driver = passwd-file } plugin { acl = vfile acl_shared_dict = file:/var/mail/shared-database/shared-mailboxes.db } protocols = imap service auth { unix_listener auth-userdb { group = mail mode = 0660 user = dovecot } user = dovecot } service imap-login { inet_listener imap { port = 0 } user = dovecot } service imap { executable = /usr/lib/dovecot/imap } userdb { driver = prefetch } userdb { args = username_format=%n /etc/dovecot/local-maildir default_fields = uid=dovecot gid=mail driver = passwd-file } userdb { args = username_format=%n /etc/dovecot/local-mdbox default_fields = uid=dovecot gid=mail driver = passwd-file } protocol imap { mail_max_userip_connections = 100 mail_plugins = acl imap_acl }
With the configuration and 2 passwd-files as above, all imap processes (when logged as user msl) constantly crash with segfaults. Replacing %%h by template such as /var/mail/%%n (as in my initial report) behaves the same way.
Now - IF share1 namespace is commented out or removed - everything works fine (and msl sees content shared by nmm under shared2/nmm/ ). Similarly - if only one shared namespace uses variables and the other points directly to some user - no crashes then.
Any ideas ?
I can get systraces/cores (though the latter without debug symbols - but I can recompile if need be).
I think I've found the reason behind those crashes as in the configuration posted in the earlier mail.
First the full backtrace:
Message: Process 13965 (imap) of user 105 dumped core.
Stack trace of thread 13965:
#0 0x00007fbdaa15929a __strcmp_sse2_unaligned (libc.so.6)
#1 0x00007fbdaa820c50 mail_storage_match_class (libdovecot-storage.so.0)
#2 0x00007fbdaa820cbf mail_storage_find (libdovecot-storage.so.0)
#3 0x00007fbdaa82103d mail_storage_create_full (libdovecot-storage.so.0)
#4 0x00007fbdaa8212c2 mail_storage_create (libdovecot-storage.so.0)
#5 0x00007fbdaa816cdc mail_namespaces_init_add (libdovecot-storage.so.0)
#6 0x00007fbdaa817694 mail_namespaces_init (libdovecot-storage.so.0)
#7 0x00007fbdaa82a4cd mail_storage_service_init_post (libdovecot-storage.so.0)
#8 0x00007fbdaa82c266 mail_storage_service_next_real (libdovecot-storage.so.0)
#9 0x00007fbdaa82c321 mail_storage_service_next (libdovecot-storage.so.0)
#10 0x00007fbdaa82c49a mail_storage_service_lookup_next (libdovecot-storage.so.0)
#11 0x00000000004314f0 client_create_from_input (imap)
#12 0x0000000000431968 login_client_connected (imap)
#13 0x00007fbdaa49b1c1 master_login_auth_finish (libdovecot.so.0)
#14 0x00007fbdaa49baca master_login_auth_callback (libdovecot.so.0)
#15 0x00007fbdaa49cae9 master_login_auth_input_user (libdovecot.so.0)
#16 0x00007fbdaa49cfb1 master_login_auth_input (libdovecot.so.0)
#17 0x00007fbdaa54a545 io_loop_call_io (libdovecot.so.0)
#18 0x00007fbdaa54ce68 io_loop_handler_run_internal (libdovecot.so.0)
#19 0x00007fbdaa54a726 io_loop_handler_run (libdovecot.so.0)
#20 0x00007fbdaa54a649 io_loop_run (libdovecot.so.0)
#21 0x00007fbdaa49ee3b master_service_run (libdovecot.so.0)
#22 0x0000000000431efb main (imap)
#23 0x00007fbdaa0ea291 __libc_start_main (libc.so.6)
#24 0x000000000040c6da _start (imap)GNU gdb (GDB) 7.12 Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/lib/dovecot/imap...done. [New LWP 13965] Core was generated by `dovecot/imap'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007fbdaa15929a in __strcmp_sse2_unaligned () from /usr/lib/libc.so.6 (gdb) up #1 0x00007fbdaa820c50 in mail_storage_match_class (storage=0xdc5800, storage_class=0x7fbdaab715c0 <shared_storage>, set=0x7fff217d6030) at mail-storage.c:285 285 strcmp(storage->unique_root_dir, (gdb) print storage->unique_root_dir $1 = 0x0 (gdb)
So the first argument passed to strcmp() was NULL. The offending part of code is:
if ((storage->class_flags & MAIL_STORAGE_CLASS_FLAG_UNIQUE_ROOT) != 0 && strcmp(storage->unique_root_dir, (set->root_dir != NULL ? set->root_dir : "")) != 0) return FALSE;
The 2nd argument is sanitized explicitly, but the first is not - and apparently it can be NULL as well. Adding same check to the 1st argument stopped segfaults and both shared namespaces seemed to be working correctly - so user 'msl' could see/subscribe shared folders from user 'nnm' and vice versa (both mailboxes being of different format). While this was enough here, something else might be needed to make it fully correct (the original strcmp() invocation would suggest that the 1st argument should never be NULL).
Configurations with multiple shared namespaces can trigger a bug where the first argument of strcmp() invocation is NULL. This patch adds an explicit check, analogously to how the second argument is sanitized. --- src/lib-storage/mail-storage.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib-storage/mail-storage.c b/src/lib-storage/mail-storage.c index 1d9b1bf..3d9f5dc 100644 --- a/src/lib-storage/mail-storage.c +++ b/src/lib-storage/mail-storage.c @@ -282,7 +282,7 @@ mail_storage_match_class(struct mail_storage *storage, return FALSE; if ((storage->class_flags & MAIL_STORAGE_CLASS_FLAG_UNIQUE_ROOT) != 0 && - strcmp(storage->unique_root_dir, + strcmp((storage->unique_root_dir != NULL ? storage->unique_root_dir : ""), (set->root_dir != NULL ? set->root_dir : "")) != 0) return FALSE; -- 2.1.3
On 10 Jan 2017, at 21.58, Michal Soltys <soltys@ziu.info> wrote:
I think it shouldn't be NULL though.. I'd rather add some asserts and figure out why it is. I guess the attached patch assert-crashes? What's the backtrace there?
On 01/10/2017 09:31 PM, Timo Sirainen wrote:
Yea, assert triggers instantly once I try to read any folder. bt full below
#2 0x00007f1b92c53727 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:201 backtrace = 0x971fb0 "/usr/lib/dovecot/libdovecot.so.0(+0xc36d8) [0x7f1b92c536d8] -> /usr/lib/dovecot/libdovecot.so.0(+0xc4c06) [0x7f1b92c54c06] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f1b92c53a5b] -> /usr/lib/d"... #3 0x00007f1b92c54c06 in i_internal_fatal_handler (ctx=0x7ffdee3f6fe0, format=0x7f1b93043e68 "file %s: line %d (%s): assertion failed: (%s)", args=0x7ffdee3f7000) at failures.c:670 status = 0 #4 0x00007f1b92c53a5b in i_panic (format=0x7f1b93043e68 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:275 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0} args = <error reading variable args (Attempt to dereference a generic pointer.)> #5 0x00007f1b92f4921e in mail_storage_create_full (ns=0x9927e0, driver=0x7f1b93042516 "shared", data=0x98f438 "mdbox:%h", flags=(unknown: 0), storage_r=0x7ffdee3f71d0, error_r=0x7ffdee3f7230) at mail-storage.c:407 storage_class = 0x7f1b932995c0 <shared_storage> storage = 0x995800 list = 0x994ff0 list_set = {layout = 0x7f1b9304841d "shared", root_dir = 0x98ebc8 "/var/run/dovecot", index_dir = 0x0, index_pvt_dir = 0x0, control_dir = 0x0, alt_dir = 0x0, inbox_path = 0x0, subscription_fname = 0x0, maildir_name = 0x7f1b93044073 "", mailbox_dir_name = 0x7f1b93044073 "", escape_char = 0 '\000', broken_char = 0 '\000', utf8 = false, alt_dir_nocheck = false, index_control_use_maildir_name = false} list_flags = (unknown: 0) p = 0x0 __FUNCTION__ = "mail_storage_create_full" #6 0x00007f1b92f4931d in mail_storage_create (ns=0x9927e0, driver=0x7f1b93042516 "shared", flags=(unknown: 0), error_r=0x7ffdee3f7230) at mail-storage.c:420 storage = 0x9921e0 #7 0x00007f1b92f3ecdc in mail_namespaces_init_add (user=0x98e0b0, ns_set=0x98ed70, unexpanded_ns_set=0x98e5e8, ns_p=0x992080, error_r=0x7ffdee3f7378) at mail-namespace.c:195 mail_set = 0x98e9d8 ns = 0x9927e0 driver = 0x7f1b93042516 "shared" error = 0x0 ret = 0 #8 0x00007f1b92f3f694 in mail_namespaces_init (user=0x98e0b0, error_r=0x7ffdee3f7378) at mail-namespace.c:414 mail_set = 0x98e9d8 ns_set = 0x98ecc0 unexpanded_ns_set = 0x98e538 namespaces = 0x992080 ns_p = 0x992080 i = 1 count = 3 count2 = 3 __FUNCTION__ = "mail_namespaces_init" #9 0x00007f1b92f52528 in mail_storage_service_init_post (ctx=0x97b7d0, user=0x980040, priv=0x7ffdee3f7380, mail_user_r=0x7ffdee3f7498, error_r=0x7ffdee3f7378) at mail-storage-service.c:728 mail_set = 0x98e9d8 home = 0x980be9 "/var/mail1/msl" mail_user = 0x98e0b0 #10 0x00007f1b92f542c1 in mail_storage_service_next_real (ctx=0x97b7d0, user=0x980040, mail_user_r=0x7ffdee3f7498) at mail-storage-service.c:1426 priv = {uid = 105, gid = 8, uid_source = 0x7f1b930454cc "userdb lookup", gid_source = 0x7f1b930454cc "userdb lookup", home = 0x980be9 "/var/mail1/msl", chroot = 0x971838 ""} error = 0x0 len = 0 disallow_root = true temp_priv_drop = false use_chroot = true #11 0x00007f1b92f5437c in mail_storage_service_next (ctx=0x97b7d0, user=0x980040, mail_user_r=0x7ffdee3f7498) at mail-storage-service.c:1444 old_log_prefix = 0x97fe50 "imap(msl): " ret = 0 #12 0x00007f1b92f544f5 in mail_storage_service_lookup_next (ctx=0x97b7d0, input=0x7ffdee3f7520, user_r=0x7ffdee3f7490, mail_user_r=0x7ffdee3f7498, error_r=0x7ffdee3f7518) at mail-storage-service.c:1477 user = 0x980040 ret = 1 #13 0x00000000004314f0 in client_create_from_input (input=0x7ffdee3f7520, fd_in=7, fd_out=7, client_r=0x7ffdee3f7510, error_r=0x7ffdee3f7518) at main.c:228 user = 0x7ffdee3f74d0 mail_user = 0x7ffdee3f7510 ns = 0x7f1b92c9dfb3 client = 0x979370 imap_set = 0xc00000000 lda_set = 0x971100 errstr = 0x7f1b92efeac0 <static_system_pool> "\200\352\357\222\033\177" mail_error = 32539 #14 0x0000000000431968 in login_client_connected (login_client=0x97da20, username=0x971043 "msl", extra_fields=0x9710d0) at main.c:316 input = {module = 0x43db49 "imap", service = 0x43db49 "imap", username = 0x971043 "msl", session_id = 0x97daa0 "PARRLs5FeMjAqAD+", session_id_prefix = 0x0, session_create_time = 0, local_ip = {family = 2, u = {ip6 = {__in6_u = { __u6_addr8 = "\300\250\000\374", '\000' <repeats 11 times>, __u6_addr16 = {43200, 64512, 0, 0, 0, 0, 0, 0}, __u6_addr32 = { 4227901632, 0, 0, 0}}}, ip4 = {s_addr = 4227901632}}}, remote_ip = {family = 2, u = {ip6 = {__in6_u = { __u6_addr8 = "\300\250\000\376", '\000' <repeats 11 times>, __u6_addr16 = {43200, 65024, 0, 0, 0, 0, 0, 0}, __u6_addr32 = { 4261456064, 0, 0, 0}}}, ip4 = {s_addr = 4261456064}}}, local_port = 0, remote_port = 0, userdb_fields = 0x9710d0, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0, debug = 0} client = 0x3000000018 flags = (MAIL_AUTH_REQUEST_FLAG_TLS_COMPRESSION | unknown: 32538) error = 0x7ffdee3f75f0 "0̗" __FUNCTION__ = "login_client_connected" #15 0x00007f1b92bc31c1 in master_login_auth_finish (client=0x97da20, auth_args=0x9710c8) at master-login.c:210 login = 0x97cd30 service = 0x9795e0 close_sockets = true __FUNCTION__ = "master_login_auth_finish" #16 0x00007f1b92bc3aca in master_login_auth_callback (auth_args=0x9710c8, errormsg=0x0, context=0x97da20) at master-login.c:379 client = 0x97da20 conn = 0x97d820 reply = {tag = 1, status = MASTER_AUTH_STATUS_OK, mail_pid = 20189} #17 0x00007f1b92bc4ae9 in master_login_auth_input_user (auth=0x97cdb0, args=0x97de5c "4291297281\tmsl\tuid=105\tgid=8\tmail=maildir:/var/mail1/msl\thome=/var/mail1/msl\tauth_token=18dd1092f041e803835776fae22759a100511eb8") at master-login-auth.c:244 request = 0x97cc30 list = 0x9710c0 id = 4291297281 #18 0x00007f1b92bc4fb1 in master_login_auth_input (auth=0x97cdb0) at master-login-auth.c:364 line = 0x97de57 "USER\t4291297281\tmsl\tuid=105\tgid=8\tmail=maildir:/var/mail1/msl\thome=/var/mail1/msl\tauth_token=18dd1092f041e803835776fae22759a100511eb8" ret = false #19 0x00007f1b92c72545 in io_loop_call_io (io=0x97ccb0) at ioloop.c:599 ioloop = 0x979740 t_id = 2 __FUNCTION__ = "io_loop_call_io" #20 0x00007f1b92c74e68 in io_loop_handler_run_internal (ioloop=0x979740) at ioloop-epoll.c:222 ctx = 0x97b260 events = 0x97c0d0 event = 0x97c0d0 list = 0x97cd10 io = 0x97ccb0 tv = {tv_sec = 154, tv_usec = 999457} events_count = 5 msecs = 155000 ret = 1 i = 0 j = 0 call = true __FUNCTION__ = "io_loop_handler_run_internal" #21 0x00007f1b92c72726 in io_loop_handler_run (ioloop=0x979740) at ioloop.c:648 No locals. #22 0x00007f1b92c72649 in io_loop_run (ioloop=0x979740) at ioloop.c:623 __FUNCTION__ = "io_loop_run" #23 0x00007f1b92bc6e3b in master_service_run (service=0x9795e0, callback=0x431b68 <client_connected>) at master-service.c:641 No locals. #24 0x0000000000431efb in main (argc=1, argv=0x979390) at main.c:460 set_roots = {0x43ca60 <imap_setting_parser_info>, 0x648340 <lda_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0x971048 "id=105", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x431883 <login_client_connected>, failure_callback = 0x431ad3 <login_client_failed>, request_auth_token = 1} service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = (MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT | MAIL_STORAGE_SERVICE_FLAG_AUTOEXPUNGE) username = 0x0 auth_socket_path = 0x43dc63 "auth-master" c = -1
participants (3)
-
Aki Tuomi
-
Michal Soltys
-
Timo Sirainen