Lots of problems with Dovecot 2.4
Hello all,
I am trying to get Dovecot 2.4 running on Alpine Edge x86-64. But a myriad of problems have occurred that make it impossible to do so.
First of all was the recent off-by-one bug that was recently fixed, which caused an unavoidable segfault on start under musl libc, which Alpine uses: https://gitlab.alpinelinux.org/alpine/aports/-/issues/17050
Then, I saw that the shadow driver was removed, which necessitated the installation of PAM and my learning about it... I haven't needed it before, so I'd never used it. I guess I understand why the decision was made to use PAM, though I still don't like it.
After resolving those issue, I've run into new ones:
The passwd userdb does not seem to work correctly. It is able to get the first user and its information correctly, but any attempts to look up any user after that will seemingly always use the information from the first matched user. Setting max auths to 1 so that the auth process is discarded afterwards does not seem to help. For example, if the first user to log in to Dovecot is bob, and then user joe logs in, the information returned from the passwd userdb will all be that of user bob, not joe. This continues for all subsequent userdb lookups until Dovecot is restarted, and a new userdb lookup is persisted. Changing to passwd-file instead works and does not experience this problem.
Why is the only available username field for the passwd driver %{passwd:system_groups_user}? I understand what the purpose of the field is based on the docs, but shouldn't more than this be exposed? The fields available through passwd are also not documented, and I had to read the source to find them out. This should either be documented or a more standard username field be additionally exposed as with other drivers. Can this field even be relied on to use as the username, if the driver otherwise worked?
It seems the Exim auth driver is no longer able to successfully authenticate with Dovecot 2.4 against Exim 4.98.2. It works fine in the last version of 2.3.21.1. Has the authentication framework changed along with the other modifications that 2.4 underwent? Has any effort been made with the Exim team to resolve any incompatibilities? Can I somehow diagnose this issue if it's otherwise supposed to work?
The ldap driver can no longer look up missing fields from LDAP users. Some (but not all) of my users have the rfc2307bis schema posixAccount supplemental objectClass, and as such contain the posixHome, uidNumber, and gidNumber attributes—but even if I specify a default value, if one of the attributes is not queryable for the returned user, the lookup errors out and the login fails. This is a regression/undocumented change from 2.3.x, where instead of erroring out, the default_fields value would be used. Based on my interpretation of the design goals for 2.4, I would imagine the default filter would ignore the missing attribute error and set the default value like default_fields used to, no? Or am I missing something?
passdb_ldap_filter is always required for the ldap passdb driver, even if passdb_ldap_bind is enabled. Not only does this go against the docs, but the value is unused and it doesn't matter what it's set to. This is a regression from 2.3.x.
Overall I am very unimpressed by 2.4. It clearly needed a lot more time in the oven and much more thorough testing.
Relevant minimal config from my working 2.3.21.1 install:
auth_username_format = %Ln auth_mechanisms = plain login passdb { driver = shadow } userdb { driver = passwd override_fields = gid=mail home=/var/mail/%Ln } passdb { driver = ldap args = /etc/dovecot/ldap-passdb.conf.ext #uris = ldapi://%2frun%2fopenldap%2fslapd.sock #base = ou=accounts,dc=ldap #auth_bind = yes #auth_bind_userdn = uid=%Ln,ou=accounts,dc=ldap } userdb { driver = ldap default_fields = uid=mail gid=mail home=/var/mail/%Ln args = /etc/dovecot/ldap-userdb.conf.ext #uris = ldapi://%2frun%2fopenldap%2fslapd.sock #dn = cn=dovecot,dc=ldap #dnpass = "some pw" #base = ou=accounts,dc=ldap #iterate_filter = (objectClass=inetOrgPerson) #iterate_attrs = uid=username #user_filter = (&(objectClass=inetOrgPerson)(uid=%Ln)) #user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid } mail_location = maildir:/var/mail/%Ln mail_plugins = $mail_plugins virtual namespace inbox { inbox = yes mailbox INBOX { auto = subscribe } } namespace virtual { type = private separator = . prefix = "Search Folders." location = virtual:/etc/dovecot/virtual:INDEX=~/dovecot-virtual.cache:CONTROL=~/dovecot-virtual.cache:VOLATILEDIR=~/dovecot-virtual.cache
virtual/All/dovecot-virtual:
#!INBOX #* #-Junk #-Junk* #-Spam #-Spam* #-Trash #-Trash* #-Deleted #-Deleted* #-Deleted Items #-Deleted Items*
all
hidden = yes list = children subscriptions = no mailbox All { auto = subscribe special_use = \All comment = All messages, excluding Junk and Trash } } namespace virtual-user { type = private separator = . prefix = "User Search Folders." location = virtual:~/.virtual:INDEX=~/dovecot-virtual-user.cache:CONTROL=~/dovecot-virtual-user.cache:VOLATILEDIR=~/dovecot-virtual-user.cache:LAYOUT=maildir++ hidden = yes list = children subscriptions = no } service auth { unix_listener auth-userdb { # restrict userdb to dovecot user/group only instead of world read/write mode = 0660 user = $default_internal_user group = $default_internal_group } unix_listener auth-client { # give Exim access to native authentication mode = 0660 group = exim } } service auth-worker {
no need to run as root
user = $default_internal_user
allow pam passdb to read /etc/shadow
group = shadow } !include_try /usr/share/dovecot/protocols.d/*.conf protocol imap { mail_max_userip_connections = 25 } protocol pop3 { mail_max_userip_connections = 25 }
Converted config for 2.4:
dovecot_config_version = 2.4.1 dovecot_storage_version = 2.4.1 mail_home = /var/mail/%{user | username | lower } mail_driver = maildir mail_path = $SET:mail_home mail_plugins = virtual auth_username_format = %{user | username | lower } auth_mechanisms = plain login passdb pam { failure_show_msg = yes max_requests = 1 } userdb passwd-file { passwd_file_path = /etc/passwd fields { gid = mail } }
BUG: doesn't work right
#userdb passwd {
fields {
gid = mail
}
#} passdb ldap { ldap_uris = ldapi://%2frun%2fopenldap%2fslapd.sock ldap_connection_group = login ldap_base = ou=accounts,dc=ldap ldap_bind = yes ldap_bind_userdn = uid=%{user | username | lower },ou=accounts,dc=ldap
XXX: thought this wasnt needed with bind?
ldap_filter = z } userdb ldap { ldap_uris = ldapi://%2frun%2fopenldap%2fslapd.sock ldap_auth_dn = cn=dovecot,dc=ldap ldap_auth_dn_password = "some pw" ldap_base = ou=accounts,dc=ldap ldap_filter = (&(objectClass=inetOrgPerson)(uid=%{user | username | lower })) fields { username = %{ldap:uid} # BUG: causes errors if rfc2307bis posixAccount objectClass is not set home = %{ldap:homeDirectory | default($SET:mail_home)} uid = %{ldap:uidNumber | default(mail)} gid = %{ldap:gidNumber | default(mail)} } ldap_iterate_filter = (objectClass=inetOrgPerson) iterate_fields { username = %{ldap:uid} } } namespace inbox { inbox = yes mailbox INBOX { auto = subscribe } } namespace virtual { type = private separator = . prefix = "Search Folders." mail_driver = virtual mail_path = /etc/dovecot/virtual
virtual/All/dovecot-virtual:
#!INBOX #* #-Junk #-Junk* #-Spam #-Spam* #-Trash #-Trash* #-Deleted #-Deleted* #-Deleted Items #-Deleted Items*
all
mail_index_path = ~/dovecot-virtual.cache mail_control_path = ~/dovecot-virtual.cache mail_volatile_path = ~/dovecot-virtual.cache hidden = yes list = children subscriptions = no mailbox All { auto = subscribe special_use = \All comment = All messages, excluding Junk and Trash } } namespace virtual-user { type = private separator = . prefix = "User Search Folders." mail_driver = virtual mail_path = ~/dovecot-virtual-user mail_index_path = ~/dovecot-virtual-user.cache mail_control_path = ~/dovecot-virtual-user.cache mail_volatile_path = ~/dovecot-virtual-user.cache mailbox_list_layout = Maildir++ hidden = yes list = children subscriptions = no } service auth { unix_listener auth-userdb { # restrict userdb to dovecot user/group only instead of world read/write mode = 0660 } unix_listener auth-client { # give Exim access to native authentication mode = 0660 group = exim } } service auth-worker {
no need to run as root
user = $SET:default_internal_user
allow pam passdb to read /etc/shadow
group = shadow } protocols = imap pop3 protocol imap { mail_max_userip_connections = 25 } protocol pop3 { mail_max_userip_connections = 25 }
In addition to help with the noted issues, I'd also love if anyone more knowledgeable on Dovecot 2.4 could give any pointers on my config and if I could improve the translation in any way, given I have been fumbling around with it for weeks and am not really sure if my various decisions are correct.
Best regards.
On 31/05/2025 20:31 EEST via dovecot <dovecot@dovecot.org> wrote:
Hello all,
I am trying to get Dovecot 2.4 running on Alpine Edge x86-64. But a myriad of problems have occurred that make it impossible to do so.
First of all was the recent off-by-one bug that was recently fixed, which caused an unavoidable segfault on start under musl libc, which Alpine uses: https://gitlab.alpinelinux.org/alpine/aports/-/issues/17050
Then, I saw that the shadow driver was removed, which necessitated the installation of PAM and my learning about it... I haven't needed it before, so I'd never used it. I guess I understand why the decision was made to use PAM, though I still don't like it.
After resolving those issue, I've run into new ones:
The passwd userdb does not seem to work correctly. It is able to get the first user and its information correctly, but any attempts to look up any user after that will seemingly always use the information from the first matched user. Setting max auths to 1 so that the auth process is discarded afterwards does not seem to help. For example, if the first user to log in to Dovecot is bob, and then user joe logs in, the information returned from the passwd userdb will all be that of user bob, not joe. This continues for all subsequent userdb lookups until Dovecot is restarted, and a new userdb lookup is persisted. Changing to passwd-file instead works and does not experience this problem.
Why is the only available username field for the passwd driver %{passwd:system_groups_user}? I understand what the purpose of the field is based on the docs, but shouldn't more than this be exposed? The fields available through passwd are also not documented, and I had to read the source to find them out. This should either be documented or a more standard username field be additionally exposed as with other drivers. Can this field even be relied on to use as the username, if the driver otherwise worked?
It seems the Exim auth driver is no longer able to successfully authenticate with Dovecot 2.4 against Exim 4.98.2. It works fine in the last version of 2.3.21.1. Has the authentication framework changed along with the other modifications that 2.4 underwent? Has any effort been made with the Exim team to resolve any incompatibilities? Can I somehow diagnose this issue if it's otherwise supposed to work?
The ldap driver can no longer look up missing fields from LDAP users. Some (but not all) of my users have the rfc2307bis schema posixAccount supplemental objectClass, and as such contain the posixHome, uidNumber, and gidNumber attributes—but even if I specify a default value, if one of the attributes is not queryable for the returned user, the lookup errors out and the login fails. This is a regression/undocumented change from 2.3.x, where instead of erroring out, the default_fields value would be used. Based on my interpretation of the design goals for 2.4, I would imagine the default filter would ignore the missing attribute error and set the default value like default_fields used to, no? Or am I missing something?
passdb_ldap_filter is always required for the ldap passdb driver, even if passdb_ldap_bind is enabled. Not only does this go against the docs, but the value is unused and it doesn't matter what it's set to. This is a regression from 2.3.x.
Overall I am very unimpressed by 2.4. It clearly needed a lot more time in the oven and much more thorough testing.
We unfortunately aren't able to test all possible combinations that our community users, and these problems that you described are are mostly problems with musl-libc, so with Alpine I would probably wait until next release. Testing under every possible combination under the sun is not really possible for us at this moment, so we focus on the things we support with Pro.
- You can't combine things like that unfortunately. You could try instead %{ldap:homeDirectory | default} (this will default to empty string).
Also default needs '' if you want to return exactly mail, so default('mail') would probably work. default(mail) does something totally different.
- We'll look at fixing that.
Aki
On Saturday, 31 May 2025 12:52:32 CDT Aki Tuomi via dovecot wrote:
these problems that you described are are mostly problems with musl-libc, so with Alpine I would probably wait until next release.
Which issues in particular? The segfault was already fixed and it now runs under musl libc. So does e.g. 1 through 3 work on your test systems? Even with my specific configuration? If so that is incredibly strange and quite disheartening... I'll deliver it to the Alpine team however if it's the case.
And next release of what? Will these issues be addressed in Dovecot or do you mean Alpine will need to discover the source and deliver patches downstream?
I will try your suggestion with the default filter, though right now I don't have a lot of time to do so.
Regards.
participants (3)
-
Aki Tuomi
-
sev monster
-
sev+dovecot＠sev.monster