namespace directives from userdb not being applied
Brendan Braybrook
brendan at tucows.com
Thu Jul 14 17:22:25 UTC 2022
On 2022-07-13 23:37, Aki Tuomi wrote:
>
>> On 14/07/2022 04:56 EEST Brendan Braybrook <brendan at tucows.com> wrote:
>>
>>
>> hi all,
>>
>> i'm having a strange issue. i have a few dovecot instances running:
>>
>> 1- debian 10 with debian's dovecot-core 1:2.3.4.1-5+deb10u5
>> 2- debian 10 with debian's dovecot-core 1:2.3.4.1-5+deb10u6
>> 3- debian 11 with dovecot's dovecot-core 2:2.3.19.1-2+debian11
>>
>> for all these, i've just been testing with a static userdb (ignore if
>> the login user was different; they all had the same userdb data..
>> various different passdb configs were tried (none do any userdb
>> prefetching):
>>
>> userdb {
>> driver = static
>> args =
>> mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
>> home=/mail/mailstore03/492/177/b at b.com/ namespace=inbox,share
>> namespace/inbox/separator=/ namespace/share/separator=/
>> namespace/share/prefix=boop/
>> }
>>
>> instance #2 and #3 don't seem to pass all the namespace directives i'm
>> passing through to the imap process.
>>
>> (i originally found it happening with a lua userdb script then noticed
>> it with a dict userdb, but the simpler static userdb still shows it
>> happening so i don't think it's related to the different userdb
>> mechanisms themselves)
>>
>> debug looks like:
>>
>> auth: Debug: master userdb out: USER 3466461185 b at b.com
>> mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
>> home=/mail/mailstore03/492/177/b at b.com/ namespace=inbox,share
>> namespace/inbox/separator=/ namespace/share/separator=/
>> namespace/share/prefix=boop/ auth_mech=PLAIN
>> auth_token=bcdb366d56f4fceee83775a1b28ca89ccdf881fd
>> script-login(b at b.com)<56895><>: Debug: Added userdb setting:
>> mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
>> script-login(b at b.com)<56895><>: Debug: Added userdb setting:
>> namespace=inbox,share
>> script-login(b at b.com)<56895><>: Debug: Added userdb setting:
>> namespace/inbox/separator=/
>> script-login(b at b.com)<56895><>: Debug: Added userdb setting:
>> namespace/share/prefix=boop/
>> script-login(b at b.com)<56895><>: Debug: Added userdb setting:
>> namespace/share/separator=/
>> imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Added userdb setting:
>> mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
>> imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Added userdb setting:
>> namespace=inbox,share
>> imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Namespace inbox:
>> type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes,
>> subscriptions=yes
>> location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
>> imap(b at b.com)<56894><9mFyornjaMcKyAUI>: Debug: Namespace : type=private,
>> prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes
>> location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
>>
>> the script-login task notes that these userdb settings are being added:
>> namespace=inbox,share
>> namespace/inbox/separator=/
>> namespace/share/prefix=boop/
>> namespace/share/separator=/
>>
>> but the imap task only notes one:
>> namespace=inbox,share
>>
>> both of these two instances end up getting an error because the imap
>> task didn't pick up the prefix:
>> Error: namespace configuration error: Duplicate namespace prefix: ""
>>
>> instance #1 behaves differently. it's script-login task debug is the
>> same, but the imap debug notes:
>> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added
>> userdb setting:
>> mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
>> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added
>> userdb setting: namespace=inbox,share
>> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug: Added
>> userdb setting: namespace/inbox/separator=/
>> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug:
>> tc_empty_notify_plugin init
>> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug:
>> Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no,
>> list=yes, subscriptions=yes
>> location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
>> imap(bigip-imap at nagios.tucows.com)<2146><s3ctO7njd6cKBS39>: Debug:
>> Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes,
>> subscriptions=yes
>> location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
>>
>> on this one, the imap task notes that it added ALL the namespace
>> directives, and consequently doesn't have any namespace prefix collisions.
>>
>> the difference between 1:2.3.4.1-5+deb10u5 and 1:2.3.4.1-5+deb10u6
>> doesn't seem that great, and the bug tracker for it doesn't indicate
>> anything around userdb would be patched
>> (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970386)
>>
>> but the problem still seems to exist in 2.3.19 - can anyone else see the
>> same thing happening with the current builds?
>>
>> thanks!
>> -brendan
>
> Hi!
>
> Can you also provide doveconf -n? You are missing location for namespace/share the least, so it ends up in same location...
ah, yes. i stripped down the bits i was sending in the userdb result to
make the case simpler. if i include namespace/share/location, the same
problem seems to occur, as the imap process doesn't include that either.
> Other than that, this config seems to work for me at least, with my simple config.
here's the config. i'll try it again with fewer plugins and a more
minimal config as well.
update: it actually seems like some issue with the config - i tried the
userdb with the stock config files and it seemed to work fine. i'll
report back when i figure out what is going on.
# 2.3.19.1 (9b53102964): /var/run/dovecot/dovecot.conf.real
# Pigeonhole version 0.5.19 (4eae2f79)
# OS: Linux 5.10.0-16-amd64 x86_64 Debian 11.4
# Hostname: imap-p2-01
auth_debug = yes
auth_debug_passwords = yes
auth_verbose = yes
debug_log_path = /var/log/dovecot-debug.log
first_valid_uid = 8
imap_id_retain = yes
info_log_path = syslog
lock_method = dotlock
log_timestamp =
login_trusted_networks = 10.200.0.0/16 10.5.0.0/16 10.0.0.0/8
mail_debug = yes
mail_fsync = always
mail_gid = mail
mail_nfs_index = yes
mail_nfs_storage = yes
mail_plugins = zlib quota tc_mail_log notify tc_proc tc_max_msgs
tc_empty_notify lazy_expunge
mail_temp_dir = /var/run/dovecot_tmp
mail_uid = mail
maildir_very_dirty_syncs = yes
managesieve_sieve_capability = fileinto reject
mmap_disable = yes
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox "Sent Items" {
auto = subscribe
special_use = \Sent
}
mailbox Spam {
auto = no
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
args = file=/home/brendan/dovecot_auth.lua blocking=yes
driver = lua
}
plugin {
acl = vfile:/he/dovecot/conf/lazyexpunge.acl
acl_shared_dict = file:/mail/users/cluster/dovecot/shared-mailboxes.db
antispam_allow_append_to_spam = yes
antispam_backend = pipe
antispam_pipe_program = /he/dovecot/utils/he_spamtrain.pl
antispam_pipe_program_args = --user=%u
antispam_pipe_program_notspam_arg = --falsepositive
antispam_pipe_program_spam_arg = --missed
antispam_pipe_tmpdir = /var/run/dovecot_as_tmp
antispam_signature_missing = move
antispam_spam =
antispam_trash_pattern_ignorecase = trash;OSRSSoftDelete
lazy_expunge = OSRSSoftDelete
lazy_expunge_only_last_instance = yes
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename flag_change append
mail_log_fields = uid box msgid flags hetag
memcached_servers = mcache01-stub
sieve_extensions = fileinto reject
zlib_save = gz
zlib_save_level = 6
}
protocols = imap pop3
service anvil {
unix_listener anvil-auth-penalty {
mode = 00
}
}
service auth {
client_limit = 3500
}
service imap-login {
inet_listener imap {
address = 0
port = 143
}
inet_listener imaps {
address = 0
port = 0
}
process_limit = 182
process_min_avail = 91
service_count = 0
}
service imap-postlogin {
executable = script-login -d /he/dovecot/utils/post_login.sh
}
service imap {
executable = imap imap-postlogin
process_limit = 7855
vsz_limit = 0
}
service managesieve-login {
inet_listener sieve {
address = 0
port = 4190
}
}
service managesieve-postlogin {
executable = script-login -d /he/dovecot/utils/managesieve_postlogin.pl
}
service managesieve {
executable = managesieve managesieve-postlogin
}
service pop3-login {
inet_listener pop3 {
address = 0
port = 110
}
inet_listener pop3s {
address = 0
port = 0
}
process_limit = 182
process_min_avail = 91
service_count = 0
}
service pop3-postlogin {
executable = script-login -d /he/dovecot/utils/post_login.sh
}
service pop3 {
executable = pop3 pop3-postlogin
process_limit = 1278
vsz_limit = 512 M
}
service stats {
client_limit = 12000
}
userdb {
args =
mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
home=/mail/mailstore03/492/177/b at b.com/ namespace=inbox,share
namespace/inbox/separator=/ namespace/share/separator=/
namespace/share/prefix=boop/
driver = static
}
verbose_proctitle = yes
protocol imap {
mail_max_userip_connections = 30
mail_plugins = zlib quota tc_mail_log notify tc_proc tc_max_msgs
tc_empty_notify lazy_expunge imap_quota acl imap_acl
}
protocol pop3 {
mail_max_userip_connections = 30
pop3_reuse_xuidl = yes
pop3_uidl_format = %08Xv%08Xu
}
the same config, but with a more stripped down userdb response shows the
problem in action too:
auth: Debug: master userdb out: USER 1535246337 b at x.com
mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
home=/mail/mailstore03/492/177/b at b.com/ namespace=inbox
namespace/inbox/separator=/ auth_mech=PLAIN
auth_token=08c7b084f58d76b028a96eed55f16c57ce3c7ff8
script-login(b at x.com)<59977><>: Debug: Added userdb setting:
mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
script-login(b at x.com)<59977><>: Debug: Added userdb setting: namespace=inbox
script-login(b at x.com)<59977><>: Debug: Added userdb setting:
namespace/inbox/separator=/
imap(b at x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: Added userdb setting:
mail=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
imap(b at x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: Added userdb setting:
namespace=inbox
imap(b at x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: tc_empty_notify_plugin init
imap(b at x.com)<59976><Kc80nMbjXoQKyAUI>: Debug: Namespace inbox:
type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes,
subscriptions=yes
location=maildir:/mail/mailstore03/492/177/b at b.com/:INDEX=/mail/index03/492/177/b at b.com/
i don't get an error here since i'm only passing the one namespace, but
the same thing occurs: the script-login event logs both namespace
settings, but the imap event only logs the "namespace=inbox" one - the
"namespace/inbox/separator=/" one never gets applied and the inbox
namespace still has the default separator
the same thing occurs if i try and send prefix/hidden/list/subscriptions
- the imap process just ignores them
More information about the dovecot
mailing list