On 2022-07-13 23:37, Aki Tuomi wrote:

...

On 14/07/2022 04:56 EEST Brendan Braybrook brendan@tucows.com wrote:

hi all,

i'm having a strange issue. i have a few dovecot instances running:

1- debian 10 with debian's dovecot-core 1:2.3.4.1-5+deb10u5 2- debian 10 with debian's dovecot-core 1:2.3.4.1-5+deb10u6 3- debian 11 with dovecot's dovecot-core 2:2.3.19.1-2+debian11

for all these, i've just been testing with a static userdb (ignore if the login user was different; they all had the same userdb data.. various different passdb configs were tried (none do any userdb prefetching):

userdb { driver = static args = mail=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ home=/mail/mailstore03/492/177/b@b.com/ namespace=inbox,share namespace/inbox/separator=/ namespace/share/separator=/ namespace/share/prefix=boop/ }

instance #2 and #3 don't seem to pass all the namespace directives i'm passing through to the imap process.

(i originally found it happening with a lua userdb script then noticed it with a dict userdb, but the simpler static userdb still shows it happening so i don't think it's related to the different userdb mechanisms themselves)

debug looks like:

auth: Debug: master userdb out: USER 3466461185 b@b.com mail=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ home=/mail/mailstore03/492/177/b@b.com/ namespace=inbox,share namespace/inbox/separator=/ namespace/share/separator=/ namespace/share/prefix=boop/ auth_mech=PLAIN auth_token=bcdb366d56f4fceee83775a1b28ca89ccdf881fd script-login(b@b.com)<56895><>: Debug: Added userdb setting: mail=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ script-login(b@b.com)<56895><>: Debug: Added userdb setting: namespace=inbox,share script-login(b@b.com)<56895><>: Debug: Added userdb setting: namespace/inbox/separator=/ script-login(b@b.com)<56895><>: Debug: Added userdb setting: namespace/share/prefix=boop/ script-login(b@b.com)<56895><>: Debug: Added userdb setting: namespace/share/separator=/ imap(b@b.com)<56894><9mFyornjaMcKyAUI>: Debug: Added userdb setting: mail=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ imap(b@b.com)<56894><9mFyornjaMcKyAUI>: Debug: Added userdb setting: namespace=inbox,share imap(b@b.com)<56894><9mFyornjaMcKyAUI>: Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ imap(b@b.com)<56894><9mFyornjaMcKyAUI>: Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes location=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/

the script-login task notes that these userdb settings are being added: namespace=inbox,share namespace/inbox/separator=/ namespace/share/prefix=boop/ namespace/share/separator=/

but the imap task only notes one: namespace=inbox,share

both of these two instances end up getting an error because the imap task didn't pick up the prefix: Error: namespace configuration error: Duplicate namespace prefix: ""

instance #1 behaves differently. it's script-login task debug is the same, but the imap debug notes: imap(bigip-imap@nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added userdb setting: mail=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ imap(bigip-imap@nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added userdb setting: namespace=inbox,share imap(bigip-imap@nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added userdb setting: namespace/inbox/separator=/ imap(bigip-imap@nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: tc_empty_notify_plugin init imap(bigip-imap@nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ imap(bigip-imap@nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes location=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/

on this one, the imap task notes that it added ALL the namespace directives, and consequently doesn't have any namespace prefix collisions.

the difference between 1:2.3.4.1-5+deb10u5 and 1:2.3.4.1-5+deb10u6 doesn't seem that great, and the bug tracker for it doesn't indicate anything around userdb would be patched (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970386)

but the problem still seems to exist in 2.3.19 - can anyone else see the same thing happening with the current builds?

thanks! -brendan

Hi!

Can you also provide doveconf -n? You are missing location for namespace/share the least, so it ends up in same location...

ah, yes. i stripped down the bits i was sending in the userdb result to make the case simpler. if i include namespace/share/location, the same problem seems to occur, as the imap process doesn't include that either.

Other than that, this config seems to work for me at least, with my simple config.

here's the config. i'll try it again with fewer plugins and a more minimal config as well.

update: it actually seems like some issue with the config - i tried the userdb with the stock config files and it seemed to work fine. i'll report back when i figure out what is going on.

# 2.3.19.1 (9b53102964): /var/run/dovecot/dovecot.conf.real # Pigeonhole version 0.5.19 (4eae2f79) # OS: Linux 5.10.0-16-amd64 x86_64 Debian 11.4 # Hostname: imap-p2-01 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes debug_log_path = /var/log/dovecot-debug.log first_valid_uid = 8 imap_id_retain = yes info_log_path = syslog lock_method = dotlock log_timestamp = login_trusted_networks = 10.200.0.0/16 10.5.0.0/16 10.0.0.0/8 mail_debug = yes mail_fsync = always mail_gid = mail mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = zlib quota tc_mail_log notify tc_proc tc_max_msgs tc_empty_notify lazy_expunge mail_temp_dir = /var/run/dovecot_tmp mail_uid = mail maildir_very_dirty_syncs = yes managesieve_sieve_capability = fileinto reject mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox "Sent Items" { auto = subscribe special_use = \Sent } mailbox Spam { auto = no } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = file=/home/brendan/dovecot_auth.lua blocking=yes driver = lua } plugin { acl = vfile:/he/dovecot/conf/lazyexpunge.acl acl_shared_dict = file:/mail/users/cluster/dovecot/shared-mailboxes.db antispam_allow_append_to_spam = yes antispam_backend = pipe antispam_pipe_program = /he/dovecot/utils/he_spamtrain.pl antispam_pipe_program_args = --user=%u antispam_pipe_program_notspam_arg = --falsepositive antispam_pipe_program_spam_arg = --missed antispam_pipe_tmpdir = /var/run/dovecot_as_tmp antispam_signature_missing = move antispam_spam = antispam_trash_pattern_ignorecase = trash;OSRSSoftDelete lazy_expunge = OSRSSoftDelete lazy_expunge_only_last_instance = yes mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid flags hetag memcached_servers = mcache01-stub sieve_extensions = fileinto reject zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 service anvil { unix_listener anvil-auth-penalty { mode = 00 } } service auth { client_limit = 3500 } service imap-login { inet_listener imap { address = 0 port = 143 } inet_listener imaps { address = 0 port = 0 } process_limit = 182 process_min_avail = 91 service_count = 0 } service imap-postlogin { executable = script-login -d /he/dovecot/utils/post_login.sh } service imap { executable = imap imap-postlogin process_limit = 7855 vsz_limit = 0 } service managesieve-login { inet_listener sieve { address = 0 port = 4190 } } service managesieve-postlogin { executable = script-login -d /he/dovecot/utils/managesieve_postlogin.pl } service managesieve { executable = managesieve managesieve-postlogin } service pop3-login { inet_listener pop3 { address = 0 port = 110 } inet_listener pop3s { address = 0 port = 0 } process_limit = 182 process_min_avail = 91 service_count = 0 } service pop3-postlogin { executable = script-login -d /he/dovecot/utils/post_login.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 1278 vsz_limit = 512 M } service stats { client_limit = 12000 } userdb { args = mail=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ home=/mail/mailstore03/492/177/b@b.com/ namespace=inbox,share namespace/inbox/separator=/ namespace/share/separator=/ namespace/share/prefix=boop/ driver = static } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 30 mail_plugins = zlib quota tc_mail_log notify tc_proc tc_max_msgs tc_empty_notify lazy_expunge imap_quota acl imap_acl } protocol pop3 { mail_max_userip_connections = 30 pop3_reuse_xuidl = yes pop3_uidl_format = %08Xv%08Xu }

the same config, but with a more stripped down userdb response shows the problem in action too:

auth: Debug: master userdb out: USER 1535246337 b@x.com mail=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ home=/mail/mailstore03/492/177/b@b.com/ namespace=inbox namespace/inbox/separator=/ auth_mech=PLAIN auth_token=08c7b084f58d76b028a96eed55f16c57ce3c7ff8 script-login(b@x.com)<59977><>: Debug: Added userdb setting: mail=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ script-login(b@x.com)<59977><>: Debug: Added userdb setting: namespace=inbox script-login(b@x.com)<59977><>: Debug: Added userdb setting: namespace/inbox/separator=/ imap(b@x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: Added userdb setting: mail=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/ imap(b@x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: Added userdb setting: namespace=inbox imap(b@x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: tc_empty_notify_plugin init imap(b@x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/mail/mailstore03/492/177/b@b.com/:INDEX=/mail/index03/492/177/b@b.com/

i don't get an error here since i'm only passing the one namespace, but the same thing occurs: the script-login event logs both namespace settings, but the imap event only logs the "namespace=inbox" one - the "namespace/inbox/separator=/" one never gets applied and the inbox namespace still has the default separator

the same thing occurs if i try and send prefix/hidden/list/subscriptions

• the imap process just ignores them