- dovecot - dovecot.org

Lots of problems with Dovecot 2.4
by sev+dovecot＠sev.monster 31 May '25

31 May '25

Hello all, I am trying to get Dovecot 2.4 running on Alpine Edge x86-64. But a myriad of problems have occurred that make it impossible to do so. First of all was the recent off-by-one bug that was recently fixed, which caused an unavoidable segfault on start under musl libc, which Alpine uses: https://gitlab.alpinelinux.org/alpine/aports/-/issues/17050 Then, I saw that the shadow driver was removed, which necessitated the installation of PAM and my learning about it... I haven't needed it before, so I'd never used it. I guess I understand why the decision was made to use PAM, though I still don't like it. After resolving those issue, I've run into new ones: 1. The passwd userdb does not seem to work correctly. It is able to get the first user and its information correctly, but any attempts to look up any user after that will seemingly always use the information from the first matched user. Setting max auths to 1 so that the auth process is discarded afterwards does not seem to help. For example, if the first user to log in to Dovecot is bob, and then user joe logs in, the information returned from the passwd userdb will all be that of user bob, not joe. This continues for all subsequent userdb lookups until Dovecot is restarted, and a new userdb lookup is persisted. Changing to passwd-file instead works and does not experience this problem. 2. Why is the only available username field for the passwd driver %{passwd:system_groups_user}? I understand what the purpose of the field is based on the docs, but shouldn't more than this be exposed? The fields available through passwd are also not documented, and I had to read the source to find them out. This should either be documented or a more standard username field be additionally exposed as with other drivers. Can this field even be relied on to use as the username, if the driver otherwise worked? 3. It seems the Exim auth driver is no longer able to successfully authenticate with Dovecot 2.4 against Exim 4.98.2. It works fine in the last version of 2.3.21.1. Has the authentication framework changed along with the other modifications that 2.4 underwent? Has any effort been made with the Exim team to resolve any incompatibilities? Can I somehow diagnose this issue if it's otherwise supposed to work? 4. The ldap driver can no longer look up missing fields from LDAP users. Some (but not all) of my users have the rfc2307bis schema posixAccount supplemental objectClass, and as such contain the posixHome, uidNumber, and gidNumber attributes—but even if I specify a default value, if one of the attributes is not queryable for the returned user, the lookup errors out and the login fails. This is a regression/undocumented change from 2.3.x, where instead of erroring out, the default_fields value would be used. Based on my interpretation of the design goals for 2.4, I would imagine the default filter would ignore the missing attribute error and set the default value like default_fields used to, no? Or am I missing something? 5. passdb_ldap_filter is always required for the ldap passdb driver, even if passdb_ldap_bind is enabled. Not only does this go against the docs, but the value is unused and it doesn't matter what it's set to. This is a regression from 2.3.x. Overall I am very unimpressed by 2.4. It clearly needed a lot more time in the oven and much more thorough testing. Relevant minimal config from my working 2.3.21.1 install: auth_username_format = %Ln auth_mechanisms = plain login passdb { driver = shadow } userdb { driver = passwd override_fields = gid=mail home=/var/mail/%Ln } passdb { driver = ldap args = /etc/dovecot/ldap-passdb.conf.ext #uris = ldapi://%2frun%2fopenldap%2fslapd.sock #base = ou=accounts,dc=ldap #auth_bind = yes #auth_bind_userdn = uid=%Ln,ou=accounts,dc=ldap } userdb { driver = ldap default_fields = uid=mail gid=mail home=/var/mail/%Ln args = /etc/dovecot/ldap-userdb.conf.ext #uris = ldapi://%2frun%2fopenldap%2fslapd.sock #dn = cn=dovecot,dc=ldap #dnpass = "some pw" #base = ou=accounts,dc=ldap #iterate_filter = (objectClass=inetOrgPerson) #iterate_attrs = uid=username #user_filter = (&(objectClass=inetOrgPerson)(uid=%Ln)) #user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid } mail_location = maildir:/var/mail/%Ln mail_plugins = $mail_plugins virtual namespace inbox { inbox = yes mailbox INBOX { auto = subscribe } } namespace virtual { type = private separator = . prefix = "Search Folders." location = virtual:/etc/dovecot/virtual:INDEX=~/dovecot-virtual.cache:CONTROL=~/dovecot-virtual.cache:VOLATILEDIR=~/dovecot-virtual.cache # virtual/All/dovecot-virtual: #!INBOX #* #-Junk #-Junk* #-Spam #-Spam* #-Trash #-Trash* #-Deleted #-Deleted* #-Deleted Items #-Deleted Items* # all hidden = yes list = children subscriptions = no mailbox All { auto = subscribe special_use = \All comment = All messages, excluding Junk and Trash } } namespace virtual-user { type = private separator = . prefix = "User Search Folders." location = virtual:~/.virtual:INDEX=~/dovecot-virtual-user.cache:CONTROL=~/dovecot-virtual-user.cache:VOLATILEDIR=~/dovecot-virtual-user.cache:LAYOUT=maildir++ hidden = yes list = children subscriptions = no } service auth { unix_listener auth-userdb { # restrict userdb to dovecot user/group only instead of world read/write mode = 0660 user = $default_internal_user group = $default_internal_group } unix_listener auth-client { # give Exim access to native authentication mode = 0660 group = exim } } service auth-worker { # no need to run as root user = $default_internal_user # allow pam passdb to read /etc/shadow group = shadow } !include_try /usr/share/dovecot/protocols.d/*.conf protocol imap { mail_max_userip_connections = 25 } protocol pop3 { mail_max_userip_connections = 25 } Converted config for 2.4: dovecot_config_version = 2.4.1 dovecot_storage_version = 2.4.1 mail_home = /var/mail/%{user | username | lower } mail_driver = maildir mail_path = $SET:mail_home mail_plugins = virtual auth_username_format = %{user | username | lower } auth_mechanisms = plain login passdb pam { failure_show_msg = yes max_requests = 1 } userdb passwd-file { passwd_file_path = /etc/passwd fields { gid = mail } } # BUG: doesn't work right #userdb passwd { # fields { # gid = mail # } #} passdb ldap { ldap_uris = ldapi://%2frun%2fopenldap%2fslapd.sock ldap_connection_group = login ldap_base = ou=accounts,dc=ldap ldap_bind = yes ldap_bind_userdn = uid=%{user | username | lower },ou=accounts,dc=ldap # XXX: thought this wasnt needed with bind? ldap_filter = z } userdb ldap { ldap_uris = ldapi://%2frun%2fopenldap%2fslapd.sock ldap_auth_dn = cn=dovecot,dc=ldap ldap_auth_dn_password = "some pw" ldap_base = ou=accounts,dc=ldap ldap_filter = (&(objectClass=inetOrgPerson)(uid=%{user | username | lower })) fields { username = %{ldap:uid} # BUG: causes errors if rfc2307bis posixAccount objectClass is not set home = %{ldap:homeDirectory | default($SET:mail_home)} uid = %{ldap:uidNumber | default(mail)} gid = %{ldap:gidNumber | default(mail)} } ldap_iterate_filter = (objectClass=inetOrgPerson) iterate_fields { username = %{ldap:uid} } } namespace inbox { inbox = yes mailbox INBOX { auto = subscribe } } namespace virtual { type = private separator = . prefix = "Search Folders." mail_driver = virtual mail_path = /etc/dovecot/virtual # virtual/All/dovecot-virtual: #!INBOX #* #-Junk #-Junk* #-Spam #-Spam* #-Trash #-Trash* #-Deleted #-Deleted* #-Deleted Items #-Deleted Items* # all mail_index_path = ~/dovecot-virtual.cache mail_control_path = ~/dovecot-virtual.cache mail_volatile_path = ~/dovecot-virtual.cache hidden = yes list = children subscriptions = no mailbox All { auto = subscribe special_use = \All comment = All messages, excluding Junk and Trash } } namespace virtual-user { type = private separator = . prefix = "User Search Folders." mail_driver = virtual mail_path = ~/dovecot-virtual-user mail_index_path = ~/dovecot-virtual-user.cache mail_control_path = ~/dovecot-virtual-user.cache mail_volatile_path = ~/dovecot-virtual-user.cache mailbox_list_layout = Maildir++ hidden = yes list = children subscriptions = no } service auth { unix_listener auth-userdb { # restrict userdb to dovecot user/group only instead of world read/write mode = 0660 } unix_listener auth-client { # give Exim access to native authentication mode = 0660 group = exim } } service auth-worker { # no need to run as root user = $SET:default_internal_user # allow pam passdb to read /etc/shadow group = shadow } protocols = imap pop3 protocol imap { mail_max_userip_connections = 25 } protocol pop3 { mail_max_userip_connections = 25 } In addition to help with the noted issues, I'd also love if anyone more knowledgeable on Dovecot 2.4 could give any pointers on my config and if I could improve the translation in any way, given I have been fumbling around with it for weeks and am not really sure if my various decisions are correct. Best regards.

3 2

Fatal: Couldn't load required plugin /usr/lib/dovecot/lib90_sieve_plugin.so: dlopen() failed: Error relocating /usr/lib/dovecot/lib90_sieve_plugin.so: mail_deliver_get_return_address: symbol not found
by paradoor 31 May '25

31 May '25

Hi there, I have a Postfix/Dovecot server which today I upgraded from Alpine Linux 3.21 to 3.22. In the process, Dovecot updated from 2.3.21 to 2.4.1, and I updated my Dovecot config accordingly. At least `doveconf -n` runs without any error, and I'm able to start and run Dovecot. For context, I am using a passwd file where I enter usernames and argon2id passwords manually, since it's just a small mail server where I am the only user. The setup was working fine with Dovecot 2.3.x, and I've sent and received lots of mail before the update. However, I noticed that I'm no longer able to authenticate to the server since doing all the updates. I thought, maybe I should make sure the passwd file is correct with my password, so I tried running `doveadm pw -s argon2id`. I then got the error: `Fatal: Couldn't load required plugin /usr/lib/dovecot/lib90_sieve_plugin.so: dlopen() failed: Error relocating /usr/lib/dovecot/lib90_sieve_plugin.so: mail_deliver_get_return_address: symbol not found` I assume this is the cause of the problem, ie Dovecot isn't able to authenticate as this error occurs when a login attempt is made (I think?). I am not sure if this is an Alpine packaging issue or a Dovecot user issue, but just in case it's a user issue that I can fix myself, I thought to post here and ask if people know if there's something I can do to resolve this. I'm also posting to the Alpine mailing list in the case of it being an Alpine packaging issue. I've attached the output of `doveconf -n`; let me know if it's preferable to put this in the body of the email and I can send a reply with it in the body. `dovecot --version` is `2.4.1-4 (7d8c0e5759)`. Example `/var/log/dovecot.conf` authentication attempt: May 31 21:50:41 auth: Debug: conn unix:login (pid=21015,uid=91) [1]: client in: AUTH 4 PLAIN protocol=imap final-resp-ok secured=tls session=ElBsg3U25ou51Zrs lip=93.113.25.226 rip=185.213.154.236 lport=993 rport=35814 ssl_ja3_hash=422b5eb1ed3e9bdf65e87a42ed96ba1f local_name=mail.revsuine.xyz resp=<hidden> May 31 21:50:41 auth(${user | username},185.213.154.236,sasl:plain)<ElBsg3U25ou51Zrs>: Debug: passwd-file: Performing passdb lookup May 31 21:50:41 auth(${user | username},185.213.154.236,sasl:plain)<ElBsg3U25ou51Zrs>: Debug: passwd-file: lookup: user=${user | username} file=/etc/dovecot/passwd May 31 21:50:41 auth(${user | username},185.213.154.236,sasl:plain)<ElBsg3U25ou51Zrs>: Info: passwd-file: unknown user May 31 21:50:41 auth(${user | username},185.213.154.236,sasl:plain)<ElBsg3U25ou51Zrs>: Debug: passwd-file: Finished passdb lookup May 31 21:50:41 auth(${user | username},185.213.154.236,sasl:plain)<ElBsg3U25ou51Zrs>: Debug: Auth request finished May 31 21:50:41 auth(${user | username},185.213.154.236,sasl:plain)<ElBsg3U25ou51Zrs>: Debug: delaying auth failure May 31 21:50:43 auth: Debug: conn unix:login (pid=21015,uid=91) [1]: client passdb out: FAIL 4 user=${user | username} original_user=pid1 (The IP addresses are my public server IP address and a Mullvad VPN IP address, so not sensitive.) I'm using the same credentials and passwd file as before the update. Thanks in advance, appreciate any responses.

1 0

Crash in imapd
by Peter Chubb 31 May '25

31 May '25

Hi Folks, I'm seeing imapd crash when an evolution user accesses her primary mailbox. She can see the mailbox perfectly well using thunderbird or Apple Mail on an iPad; just not from her evolution instance. Version is 2.4.1-4 (7d8c0e5759) The error is an assertion failure: Panic: file istream-header-filter.c: line 665 (i_stream_header_filter_snapshot_free): assertion failed: (snapshot->mstream->snapshot_pending) I'm running on Debian unstable in an LXC container, using the Debian packaged version of dovecot. Filesystem for INBOX (mbox format) is XFS; home directories are mounted via NFS. The backtrace in gdb is: #0 0x00007f996709e95c in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f9967049cc2 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00007f99670324ac in abort () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x00007f9967265964 in ?? () from /usr/lib/dovecot/libdovecot.so.0 #4 0x00007f9967333cb7 in ?? () from /usr/lib/dovecot/libdovecot.so.0 #5 0x00007f99672656c0 in i_panic () from /usr/lib/dovecot/libdovecot.so.0 #6 0x00007f996725eab1 in ?? () from /usr/lib/dovecot/libdovecot.so.0 #7 0x00007f9967345bdc in i_stream_snapshot_free () from /usr/lib/dovecot/libdovecot.so.0 #8 0x00007f9967345bdc in i_stream_snapshot_free () from /usr/lib/dovecot/libdovecot.so.0 #9 0x00007f9967345c6c in i_stream_unref () from /usr/lib/dovecot/libdovecot.so.0 #10 0x00007f996752bf99 in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 #11 0x00007f996752c391 in ?? () from /usr/lib/dovecot/libdovecot-storage.so.0 #12 0x00007f996752c64c in index_mail_get_special () from /usr/lib/dovecot/libdovecot-storage.so.0 #13 0x00007f99674a570e in mail_get_special () from /usr/lib/dovecot/libdovecot-storage.so.0 #14 0x000055fb1f1b298a in fetch_snippet (ctx=0x55fb4c044168, mail=0x55fb4c070e88, preview=0x55fb4c070230) at ./src/imap/imap-fetch-body.c:610 #15 0x000055fb1f1b7ce3 in imap_fetch_more_int (ctx=ctx@entry=0x55fb4c044168, cancel=false) at ./src/imap/imap-fetch.c:562 #16 0x000055fb1f1b7fbd in imap_fetch_more (ctx=0x55fb4c044168, cmd=0x55fb4c043e98) at ./src/imap/imap-fetch.c:617 #17 0x000055fb1f1ac1c8 in cmd_fetch (cmd=0x55fb4c043e98) at ./src/imap/cmd-fetch.c:382 #18 0x000055fb1f1b5124 in command_exec (cmd=cmd@entry=0x55fb4c043e98) at ./src/imap/imap-commands.c:208 #19 0x000055fb1f1bb650 in client_command_input (cmd=<optimized out>, cmd@entry=0x55fb4c043e98) at ./src/imap/imap-client.c:1271 #20 0x000055fb1f1bb6e6 in client_command_input (cmd=<optimized out>, cmd@entry=0x55fb4c043e98) at ./src/imap/imap-client.c:1341 #21 0x000055fb1f1bb97d in client_command_input (cmd=<optimized out>) at ./src/imap/imap-client.c:1305 #22 0x000055fb1f1bbb55 in client_handle_next_command (client=0x55fb4c042a68, remove_io_r=<synthetic pointer>) at ./src/imap/imap-client.c:1383 #23 client_handle_input (client=client@entry=0x55fb4c042a68) at ./src/imap/imap-client.c:1397 #24 0x000055fb1f1bbf47 in client_input (client=0x55fb4c042a68) at ./src/imap/imap-client.c:1441 #25 0x00007f996735177b in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 #26 0x00007f99673535ea in io_loop_handler_run_internal () from /usr/lib/dovecot/libdovecot.so.0 #27 0x00007f9967353694 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 #28 0x00007f9967353868 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 #29 0x00007f99672a6347 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 #30 0x000055fb1f1a6e27 in main (argc=<optimized out>, argv=<optimized out>) at ./src/imap/main.c:601 dovecot -n === # 2.4.1-4 (7d8c0e5759): /etc/dovecot/dovecot.conf # Pigeonhole version 2.4.1-4 (0a86619f) # OS: Linux 6.12.16-amd64 x86_64 Debian 13.0 # Hostname: wombat.chubb.wattle.id.au # 4 default setting changes since version 2.4.0 dovecot_config_version = 2.4.0 dovecot_storage_version = 2.4.0 first_valid_uid = 130 fts_autoindex = yes fts_autoindex_max_recent_msgs = 999 fts_search_add_missing = yes mail_access_groups = mail mail_driver = mbox mail_full_filesystem_access = yes mail_home = /home/%{user|username} mail_inbox_path = /var/mail/%{user} mail_index_path = /var/indices/%{user} mail_nfs_storage = yes mail_path = ~/Mail/ mail_privileged_group = mail protocols { imap = yes } ssl_cipher_list = EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+ECDHE-RSA-AES256-GCM-SHA384:+AES256:+CAMELLIA128:+AES128:+SSLv3:DES-CBC3-SHA:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA ssl_min_protocol = TLSv1 passdb pam { } userdb passwd { } namespace inbox { inbox = yes mailbox Drafts { special_use = "\\Drafts" } mailbox Junk { special_use = "\\Junk" } mailbox Trash { special_use = "\\Trash" } mailbox Sent { special_use = "\\Sent" } mailbox "Sent Messages" { special_use = "\\Sent" } mailbox Spam { special_use = "\\Junk" } } service imap-login { inet_listener imap { } inet_listener imaps { } } service pop3-login { inet_listener pop3 { } inet_listener pop3s { } } service submission-login { inet_listener submission { } inet_listener submissions { } } service lmtp { unix_listener lmtp { } } service imap { } service submission { } service auth { unix_listener auth-userdb { } } service auth-worker { } service dict { unix_listener dict { } } ssl_server { cert_file = /etc/dovecot/private/dovecot.pem key_file = /etc/dovecot/private/dovecot.key } passdb local { driver = pam } userdb local { driver = passwd } ---

1 0

Help Translating Dovecot 2.3 Proxy Configuration to 2.4
by Brent Clark 30 May '25

30 May '25

Hi all, I'm in the process of upgrading from Dovecot 2.3 to 2.4 and would appreciate some help translating an existing configuration that uses a proxy lookup for user and password databases. Here's the relevant 2.3 configuration: |passdb { driver =dict args =/etc/dovecot/dovecot-dict-auth.conf.ext } userdb { driver =dict args =/etc/dovecot/dovecot-dict-auth.conf.ext } uri= proxy:/var/run/mail_directory_service/socket:somewhere password_key= passdb/%u user_key= userdb/%u iterate_disable= yes | This configuration was working fine in 2.3 to proxy user and password lookups to an external service via a Unix socket. However, Dovecot 2.4 has reworked configuration parsing, and this syntax no longer appears valid. I couldn’t find a direct equivalent in the 2.4 documentation. Could someone advise how this should be adapted for 2.4? What’s the correct way to specify the proxy: URI in 2.4? Are password_key and user_key still supported, or should I be using a different mechanism? Is iterate_disable = yes still relevant or required in this context? Any guidance, examples, or pointers to migration resources would be very helpful. Thanks in advance, Brent Clark Hi all, I'm in the process of upgrading from Dovecot 2.3 to 2.4 and would appreciate some help translating an existing configuration that uses a proxy lookup for user and password databases. Here's the relevant 2.3 configuration: passdb { driver =dict args =/etc/dovecot/dovecot-dict-auth.conf.ext } userdb { driver =dict args =/etc/dovecot/dovecot-dict-auth.conf.ext } uri = proxy:/var/run/mail_directory_service/socket:somewhere password_key = passdb/%u user_key = userdb/%u iterate_disable = yes This configuration was working fine in 2.3 to proxy user and password lookups to an external service via a Unix socket. However, Dovecot 2.4 has reworked configuration parsing, and this syntax no longer appears valid. I couldn’t find a direct equivalent in the 2.4 documentation. Could someone advise how this should be adapted for 2.4? What’s the correct way to specify the proxy: URI in 2.4? Are password_key and user_key still supported, or should I be using a different mechanism? Is iterate_disable = yes still relevant or required in this context? Any guidance, examples, or pointers to migration resources would be very helpful. Thanks in advance, Brent Clark

6 11

Panic with FTS_BACKEND_FLAG_TOKENIZED_INPUT and virtual mailboxes: file index-search-result.c: line 132
by markus-dovecot＠doits.de 30 May '25

30 May '25

Originally posted and debugged here: https://github.com/slusarz/dovecot-fts-flatcurve/issues/66 I have to following panic with fts flatcurve enabled: ``` Panic: file index-search-result.c: line 132 (index_search_result_update_flags): assertion failed: (result->search_args->args == &search_arg) Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x46) [0x74e9890a2846] -> /usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x22) [0x74e9890a2962] -> /usr/lib/dovecot/libdovecot.so.0(+0x10d81b) [0x74e9890af81b] -> /usr/lib/dovecot/libdovecot.so.0(+0x10d8b7) [0x74e9890af8b7] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e2e2) [0x74e9890002e2] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x518a4) [0x74e9891d38a4] -> /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x274e) [0x74e988bf8a9e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x5c) [0x74e9891eff9c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x39) [0x74e9891f0039] -> dovecot/imap(cmd_select_full+0x1df) [0x62354bd8326f] -> dovecot/imap(command_exec+0xa4) [0x62354bd8b444] -> dovecot/imap(+0x22392) [0x62354bd89392] -> dovecot/imap(+0x22444) [0x62354bd89444] -> dovecot/imap(client_handle_input+0x1bd) [0x62354bd8989d] -> dovecot/imap(client_input+0x74) [0x62354bd89e64] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6d) [0x74e9890c603d] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x13a) [0x74e9890c77aa] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x54) [0x74e9890c60e4] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) [0x74e9890c62a0] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x17) [0x74e989036ad7] -> dovecot/imap(main+0x570) [0x62354bd7ad20] -> /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x74e988c29d90] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x74e988c29e40] -> dovecot/imap(_start+0x25) [0x62354bd7ade5] ``` It happens when accessing virtual folder with filters like this: ``` something-*/excep-dev* header x-abc-exception "1" ``` The error happens on subsequent access of the folder: - Removing dovecot.index{,.log} files allows to access the virtual folder again - dovecot.index gets recreated on access and the folder can be accessed multiple times - Marking an e-mail read triggers the error, though! Now each next access raises the error - Removing dovecot.index{,.log} restores access again and the e-mails are still marked as read (so this information is not lost), until the next e-mail is marked as read: then the folder cannot be accessed again Slusarz debugged the backtrace and wrote a detailed description at https://github.com/slusarz/dovecot-fts-flatcurve/issues/66#issuecomment-229… which I'll quote here: > This is not a flatcurve bug - it is a bug within Dovecot's indexing/fts handling. > > (Detailed debug below, so that core team can understand the problem.) > > As discussed above, this bug is triggered with virtual mailboxes on subsequent accesses. The initial access will work fine; the assert crash only happens when the Dovecot lib-storage index code reads the indexes created for the virtual mailboxes. > > `index_search_result_update_appends()` in lib-storage/index/index-search-result.c > > * Inside this function, a "temporary search parameter to limit the search only to the new messages" is added to the mail search args. Later, it is checked that the mail_search args points to this same search parameter after the search is executed. It is this assert check that is failing, because the mail args is changing during the course of the search. > > * `i_assert(result->search_args->args == &search_arg);` > > > The search args are being altered in plugins/fts/fts-search.args.c, `fts_search_args_expand()`. Specifically, in that function, either `fts_search_args_expand_language_top_level()` or `fts_search_args_expand_tree()` is being called, which can replace the current args parameter with a new pointer. > > `fts_search_args_expand()` is being called in plugins//fts/fts-search.c by `fts_search_try_lookup()` when the FTS backend has the FTS_BACKEND_FLAG_TOKENIZED_INPUT flag set. Flatcurve has this flag set. > > This flag is defined in plugins/fts/fts-api-private.h. Comments say: > > ``` > /* Tokenize all the input. update_build_more() will be called a single > directly indexable token at a time. Searching will modify the search > args so that lookup() sees only tokens that can be directly > searched. */ > ``` > > Thus, the comments explicitly say the search args will be modified if this flag is set. This modification is incompatible with the assert check in `index_search_result_update_appends()`, which requires that the search args not be modified. > > The core team will have to determine whether this behavior is correct, or if the assert check is invalid. You can find some more discussion at https://github.com/slusarz/dovecot-fts-flatcurve/issues/66, but I think the most important part (the very detailed analysis from Slusarz, thanks again!) might be enough to understand and maybe fix the problem. Relevant information about the mail system: ``` # 2.3.21 (47349e2482): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.21 (f6cd4b8e) # OS: Linux 6.5.0-45-generic x86_64 Ubuntu 22.04.4 LTS [...] namespace { location = virtual:~/virtual:LAYOUT=maildir++ prefix = _/ separator = / } [...] plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/db/shared fts = flatcurve fts_autoindex = yes fts_filters = normalizer-icu snowball stopwords fts_filters_en = lowercase snowball stopwords fts_languages = de en fts_tokenizer_email_address = maxlen=100 fts_tokenizer_generic = algorithm=simple maxlen=30 fts_tokenizers = generic email-address sieve = file:~/sieve;active=~/.dovecot.sieve } [...] ``` I can provide more information if you need. Thanks for looking into it!

2 2

RE: dragging files into mail folders
by Marc 29 May '25

29 May '25

If I drag files in imap folders in outlook they will show up in eg roundcube/thunderbird as winmail.dat. So I guess this is just a microsoft thing. Would be nice if there were plugins that would add functionality that unwrapped these winmail.dat files. > > > If i Understood correctly you are looking for a software like > > roundcube or something right ?. > > Not necessarily, but would be a good start. Question is if the imap > allows to store these files directly. > > > > that offers ajax based GUI that allows > > you to drap and drop mails into folders. > > Files not mails. .jpg, .odt, .txt etc > > > > > > > > With exchange and outlook I was always used to sometimes drag files > > into mail folders. Is something like this available for imap? Or maybe > a > > dovecot extension? > > > > > > (obviously I am not looking for a solution to store them as > > attachments to mails) > > > _______________________________________________ > > > dovecot mailing list -- dovecot(a)dovecot.org > > > To unsubscribe send an email to dovecot-leave(a)dovecot.org

2 2

dragging files into mail folders
by Marc 28 May '25

28 May '25

With exchange and outlook I was always used to sometimes drag files into mail folders. Is something like this available for imap? Or maybe a dovecot extension? (obviously I am not looking for a solution to store them as attachments to mails)

1 0

Re: NAMESPACE empty when logging via haproxy but populated properly when logging directly
by dk+lists.dovecot.org＠7ns.eu 27 May '25

27 May '25

On 2025-05-22 17:41, dk+lists.dovecot.org(a)7ns.eu wrote: > > On 2025-05-22 16:55, Aki Tuomi via dovecot wrote: >>> >>> I see you a local 10.20.20.20 statement there, which changes how the >>> inbox prefix behaves, and I'm guessing (?) you are testing from >>> 10.20.20.20 as it's local, not remote. >>> >>> Which would explain why. >> >> Also to add a bit that the "local_ip" (not real_local_ip) gets updated >> with haproxy proxy protocol (when enabled), so it will remote >> haproxy's local IP. >> >> Aki >> > Wow. Thank you. It works. I mangled a IPs little bit before I send them > to mailist. All machines are in the same network. Client, Proxy, Server. > All in the same subnet. This is first time I see such problem with > server software and reverse proxy. Never seen similar problem. What > exactly this "local" statement do? Mail client is in the same network as > server, so after PROXY header is stripped, my mail client IP should be > seen by dovecot exactly the same, so shouldn't dovecot treat those > connections equally? oO > > Thank you again for tip Aki. > DK Hmm, now I am struggling with "local_name" filtering in haproxy environment. According to dovecot docs [1] it could potentially work with "send-proxy-v2-ssl" configured on proxy backend. However my dovecot filter does not work. Have the same problem I had with "local" filter. Dovecot does not recognize this connection and does not filter config for it properly. I tried "send-proxy-v2-ssl-cn". Still the same. I even enabled ssl on dovecot globally with "ssl=required" to be sure dovecot does not disable some ssl logic when "ssl=no". Still no go. Seems like dovecot does not get or use TLS/SNI info from haproxy. Does it even work in such environment? Is it supported? Or is this a bug or maybe my error? Any clues please? DK [1] https://doc.dovecot.org/2.4.1/core/config/haproxy.html#tls-forwarding

4 10

Sieve vnd.dovecot.config extension does not work when mail is delivered to lmtp inet_listener but works when delivered to lmtp unix_listener
by DK 27 May '25

27 May '25

Hello. I have a strange problem with sieve scripts vnd.dovecot.config extension processing on newest dovecot 2.3 and 2.4. When I deliver mail via lmtp unix_listener sieve script can get sieve_env_* variable with no problem. However when I deliver mail via lmtp inet_listener sieve script can not get the same sieve_env_* variable. I deliver mails from postfix to lmtp inet_listener directly (not via haproxy). The same postfix is used when testing lmtp unix_listener mail delivery which works. Any clues what could be the cause of such problem? DK

1 2

Thunderbird and Dovecot 2.4.0 Incompatibility
by Amit Chaudhari 26 May '25

26 May '25

I'm trying to setup a dovecot 2.4.0 imap server and have it work with Thunderbird. I'm able to connect/auth fine, however trying to save a message to a mailbox causes the connection to close. ``` Info: Disconnected: Too many invalid IMAP commands. in=744 out=2559 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0 ``` It appears thunderbird's imap APPEND command is using syntax which dovecot doesn't like? I enabled raw logs and pasted them below: Downgrading to 2.3 resolves the issue, so it’s either a regression, or some config issue I’m not able to pinpoint. ``` 1738942430.025029 5 ID ("name" "Thunderbird" "version" "128.7.0") 1738942430.029320 6 ENABLE UTF8=ACCEPT 1738942430.033907 7 select "Drafts" 1738942430.039796 8 append "Drafts" (\Draft) UTF8 (~{726+} 1738942430.039796 Message-ID: <ea714f62-7186-4bcb-b709-007e39c6a9d6(a)example.com <mailto:ea714f62-7186-4bcb-b709-007e39c6a9d6@example.com>> 1738942430.039796 Date: Fri, 7 Feb 2025 10:33:49 -0500 1738942430.039796 MIME-Version: 1.0 1738942430.039796 User-Agent: Mozilla Thunderbird 1738942430.039796 Content-Language: en-US 1738942430.039796 To: sender(a)example.com <mailto:sender@example.com> 1738942430.039796 From: User Name <user(a)example.com <mailto:user@example.com>> 1738942430.039796 Subject: test 1738942430.039796 X-Mozilla-Draft-Info: internal/draft; vcard=0; receipt=0; DSN=0; uuencode=0; 1738942430.039796 attachmentreminder=0; deliveryformat=0 1738942430.039796 X-Identity-Key: id2 1738942430.039796 Fcc: imap://user%40example.com@192.168.115.203/Sent 1738942430.039796 Content-Type: text/html; charset=UTF-8 1738942430.039796 Content-Transfer-Encoding: 7bit 1738942430.039796 1738942430.039796 <!DOCTYPE html> 1738942430.039796 <html> 1738942430.039796 <head> 1738942430.039796 1738942430.039796 <meta http-equiv="content-type" content="text/html; charset=UTF-8"> 1738942430.039796 </head> 1738942430.039796 <body> 1738942430.039796 <p><br> 1738942430.039796 </p> 1738942430.039796 </body> 1738942430.039796 </html> 1738942430.039796 ) ``` ``` 1738942430.025188 * ID ("name" "Dovecot") 1738942430.025234 5 OK ID completed (0.001 + 0.000 secs). 1738942430.029390 * ENABLED UTF8=ACCEPT 1738942430.029418 6 OK Enabled (0.001 + 0.000 secs). 1738942430.035478 * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) 1738942430.035531 * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. 1738942430.035559 * 0 EXISTS 1738942430.035585 * 0 RECENT 1738942430.035611 * OK [UIDVALIDITY 1738898190] UIDs valid 1738942430.035638 * OK [UIDNEXT 1] Predicted next UID 1738942430.035693 7 OK [READ-WRITE] Select completed (0.002 + 0.000 + 0.001 secs). 1738942430.040007 8 BAD Error in IMAP command APPEND: Missing ')' (0.001 + 0.000 secs). 1738942430.040140 * BAD Error in IMAP tag: Invalid tag (0.001 + 0.000 secs). 1738942430.040172 Date: BAD Error in IMAP command FRI,: Unknown command (0.001 + 0.000 secs). 1738942430.040233 MIME-Version: BAD Error in IMAP command 1.0: Unknown command (0.001 + 0.000 secs). 1738942430.040291 User-Agent: BAD Error in IMAP command MOZILLA: Unknown command (0.001 + 0.000 secs). 1738942430.040356 Content-Language: BAD Error in IMAP command EN-US: Unknown command (0.001 + 0.000 secs). 1738942430.040409 To: BAD Error in IMAP command sender(a)example.com <mailto:sender@example.com>: Unknown command (0.001 + 0.000 secs). 1738942430.040467 From: BAD Error in IMAP command USER: Unknown command (0.001 + 0.000 secs). 1738942430.040521 Subject: BAD Error in IMAP command TEST: Unknown command (0.001 + 0.000 secs). 1738942430.040574 X-Mozilla-Draft-Info: BAD Error in IMAP command INTERNAL/DRAFT;: Unknown command (0.001 + 0.000 secs). 1738942430.040638 * BAD Error in IMAP tag: Invalid tag (0.001 + 0.000 secs). 1738942430.040666 X-Identity-Key: BAD Error in IMAP command ID2: Unknown command (0.001 + 0.000 secs). 1738942430.040720 Fcc: BAD Error in IMAP command: Invalid command name (0.001 + 0.000 secs). 1738942430.040746 Content-Type: BAD Error in IMAP command TEXT/HTML;: Unknown command (0.001 + 0.000 secs). 1738942430.040802 Content-Transfer-Encoding: BAD Error in IMAP command 7BIT: Unknown command (0.001 + 0.000 secs). 1738942430.040862 * BAD Error in IMAP tag: Invalid tag (0.001 + 0.000 secs). 1738942430.040889 <!DOCTYPE BAD Error in IMAP command HTML>: Unknown command (0.001 + 0.000 secs). 1738942430.040966 <html> BAD Error in IMAP command: Invalid command name (0.001 + 0.000 secs). 1738942430.040991 * BAD Error in IMAP tag: Invalid tag (0.001 + 0.000 secs). 1738942430.041015 * BAD Error in IMAP tag: Invalid tag (0.001 + 0.000 secs). 1738942430.041032 * BYE Too many invalid IMAP commands. ```

5 7