We are pleased to release v2.3.17 of Dovecot.
Please note that 2.3.17 release will be the last one to support Debian/Stretch since Bullseye is now supported.
https://dovecot.org/releases/2.3/dovecot-2.3.17.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.17.tar.gz.sig Binary packages in https://repo.dovecot.org/ Docker images in https://hub.docker.com/r/dovecot/dovecot
Regards Aki Tuomi Open-Xchange oy
- Dovecot now logs a warning if time seems to jump forward at least 100 milliseconds.
- dict: Lines logged by the dict process now contain the dict name as the prefix.
- lib-index: mail_cache_fields, mail_always_cache_fields and mail_never_cache_fields now verifies that the listed header names are valid. Especially the UTF8 "–" character has sometimes been wrongly used instead of the ASCII "-".
- *-login: Added login_proxy_rawlog_dir setting to capture rawlogs between proxy and backend.
- dict: The server process now keeps the last 10 idle dict backends cached for maximum of 30 seconds. Practically this acts as a connection pool for dict-redis and dict-ldap. Note that this doesn't affect dict-sql, because it already had its own internal cache.
- doveadm: New stats add/remove commands added to support changing the metrics configuration on runtime.
- lazy_expunge: Added lazy_expunge_exclude settings to disable lazy_expunge for specific folders. \Special-use flags can be used as folder names.
- lib-lua: Added a new helper function dovecot.restrict_global_variables() to disable or enable defining new global variables.
- LAYOUT=index List index rebuild was missing.
- LAYOUT=index: Duplicate GUIDs were not detected.
- acl: When using acl_ignore_namespace Dovecot attempted to access or create dovecot-acl-list even when the namespace should have been ignored. For virtual namespaces this could have yielded errors about "Read-only file system" or "Permission denied".
- auth: Setting the "master" passdb field to empty value would cause proxying to fail with an authentication error. Now an empty "master" field is ignored.
- doveadm-server: Duplicate error lines were sent for failed commands. This didn't normally cause visible problems, except when using wildcards in usernames or -A parameter to go through multiple users.
- doveadm-server: Logs written by doveadm-server were often missing log prefixes, especially mail_log_prefix for mail commands. Logs sent to doveadm TCP client were also missing log prefixes.
- doveadm: v2.3 regression: batch command always crashes.
- doveadm: v2.3.11 regression: Commands failed if ssl_cert or ssl_key files weren't readable by the user running doveadm, even though doveadm didn't actually use these settings
- imap-hibernate: Process may crash at deinit: Panic: file ioloop.c: line 928 (io_loop_destroy): assertion failed: (ioloop->cur_ctx == NULL).
- imap: Using imap_fetch_failure=no-after can cause assert-crash with some IMAP commands if reading the mail fails (e.g. wrong cached mail size). Fixes: Panic: file index-mail-headers.c: line 198 (index_mail_parse_header_init): assertion failed: (!mail->data.header_parser_initialized)
- imap: v2.3.10 regression: When using INDEXPVT to enable private \Seen flags (for shared or public namespaces) the STORE command did not send untagged replies for the \Seen flag changes.
- imap: v2.3.15 regression: If PREVIEW/SNIPPET is not the final FETCH option in the command, the IMAP FETCH response is broken.
- imap: v2.3.15 regression: MOVE command leaks mailbox if it can't be opened and crashes at deinit: Panic: file mail-user.c: line 229 (mail_user_deinit): assertion failed: ((*user)->refcount == 1).
- imapc: Copying nonexistent mail via imapc could have crashed. Fixes: Panic: file mail-storage.c: line 2385 (mailbox_transaction_commit_get_changes): assertion failed: (ret < 0 || seq_range_count(&changes_r->saved_uids) == save_count || array_count(&changes_r->saved_uids) == 0).
- indexer: v2.3.15 regression: Process crashes if indexer-client disconnects while it's waiting for command reply. This happened for example if IMAP SEARCH triggered long fts indexing and the IMAP client disconnected while waiting for the reply.
- indexer: v2.3.15 regression: Process may have crashed in some situations.
- indexer: v2.3.15 regression: indexer-worker processes may not have reached the process_limit in some situations, possibly even using just one indexer-worker process even though there were many indexing requests queued.
- lib-compression: Reading lz4 compressed mdbox mails may crash. Fixes: Panic: file istream.c: line 345 (i_stream_read_memarea): assertion failed: (!stream->blocking).
- lib-compression: bench-compress crashes due to xz being read-only.
- lib-lua: Fix linking libdict_lua for non-GNU linkers when Lua support is disabled.
- lib-mail: There was no limit on how large an email header name could be. Processable header names are now limited to 1000 bytes.
- lib-oauth2: Dovecot disallowed JWT tokens if their validity time was older than token creation time (nbf < iat).
- lib-storage: Reduce memory footprint of certain storage operations.
- lib-storage: When listing mailboxes with storage name escape characters (^ or .) as part of the mailbox name, the listing could show corrupted mailbox names. Due to an issue in handling escaped parent folders, the listing of other mailbox names would become corrupted by prepending parts of the previously listed mailboxes parent folder as prefix to the actual mailbox names. The corruption can occur when using LAYOUT=INDEX and maildir or obox, or when using the listescape plugin.
- mail-crypt: Fix "-O" argument for "doveadm mailbox cryptokey password" command to be a boolean, and not expect a string.
- submission-login: Add support for not authenticating to next hop in submission proxying.
- submission-login: EHLO was not sent again after XCLIENT when doing submission proxying.
- virtual: Mailboxes do not correctly detect underlying mailboxes getting re-created even though they have a different UIDVALIDITY or GUID.
On Thu, 28 Oct 2021 12:12:53 +0300 (EEST) Aki Tuomi wrote:
We are pleased to release v2.3.17 of Dovecot.
Hello,
on my private fully updated Fedora 34 system I've replaced version 2.3.16 by version 2.3.17 without touching the configuration.
Whereas version 2.3.16 worked the new 2.3.17 says Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, TLS Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): ID sent: name=imapsync, version=1.977, os=linux, vendor=Gilles LAMIRAL, support-url=https://imapsync.lamiral.info/, date=23-Dec-2019 20:18:02 +0000, side=host2 Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: service(imap): child 216943 killed with signal 11 (core dumped) Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, TLS Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: service(imap): child 216952 killed with signal 11 (core dumped) Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, TLS Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: service(imap): child 216964 killed with signal 11 (core dumped)
For comparison the old version
Oct 28 11:36:58 christo dovecot[279524]: imap-login: Login: frank, 127.0.0.1, TLS Oct 28 11:36:58 christo dovecot[279524]: IMAP(frank,127.0.0.1): ID sent: name=imapsync, version=1.977, os=linux, vendor=Gilles LAMIRAL, support-url=https://imapsync.lamiral.info/, date=23-Dec-2019 20:18:02 +0000, side=host2 Oct 28 11:37:37 christo dovecot[279524]: IMAP(frank,127.0.0.1): Disconnected: Logged out [79451/32883563]
If required here is the configuration which is rather old but worked until 2.3.16
# 2.3.16 (7e2e900c1a): /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 5.14.13-200.fc34.x86_64 x86_64 Fedora release 34 (Thirty Four) ext3 # Hostname: christo auth_mechanisms = plain login default_client_limit = 1024 default_process_limit = 256 default_vsz_limit = 512 M first_valid_uid = 200 last_valid_uid = 65534 listen = * lmtp_save_to_detail_mailbox = yes login_greeting = m28a.ddns.net - IMAPs Service (dovecot) ready. login_log_format_elements = %u %r %c mail_location = maildir:/var/spool/mail/%u:LAYOUT=fs mail_log_prefix = "%Us(%u,%r): " mail_plugin_dir = /usr/dovecot/lib/dovecot/ mail_plugins = notify quota fts fts_squat acl namespace inbox { inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Gesendet { special_use = \Sent } mailbox SPAM { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = subscriptions = yes type = private } passdb { args = dovecot driver = pam } plugin { fts = squat fts_squat = partial=4 full=10 mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box from subject msgid size flags mail_log_group_events = yes } postmaster_address = mailonator@mailbox.org protocols = imap service anvil { client_limit = 1024 } service auth { unix_listener auth-client { group = exim mode = 0660 user = exim } } service imap-login { inet_listener imap { port = 143 } process_limit = 512 process_min_avail = 3 } service imap-postlogin { executable = script-login /usr/local/sbin/dovecot-imap-post-login } service imap { executable = imap process_limit = 128 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3 { process_limit = 128 } ssl_cert =
What's going on? Any pointer to solution welcome.
Kind regards, Frank
Hi!
Can you provide gdb bt full output for this crash?
Install debug symbols (if necessary) and
gdb /usr/libexec/dovecot/imap /path/to/core bt full
Aki
On 28 October 2021 11.27.07 UTC, Frank Elsner frank.elsner@mailbox.org wrote:
On Thu, 28 Oct 2021 12:12:53 +0300 (EEST) Aki Tuomi wrote:
We are pleased to release v2.3.17 of Dovecot.
Hello,
on my private fully updated Fedora 34 system I've replaced version 2.3.16 by version 2.3.17 without touching the configuration.
Whereas version 2.3.16 worked the new 2.3.17 says Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, TLS Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): ID sent: name=imapsync, version=1.977, os=linux, vendor=Gilles LAMIRAL, support-url=https://imapsync.lamiral.info/, date=23-Dec-2019 20:18:02 +0000, side=host2 Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: service(imap): child 216943 killed with signal 11 (core dumped) Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, TLS Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: service(imap): child 216952 killed with signal 11 (core dumped) Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, TLS Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: service(imap): child 216964 killed with signal 11 (core dumped)
For comparison the old version
Oct 28 11:36:58 christo dovecot[279524]: imap-login: Login: frank, 127.0.0.1, TLS Oct 28 11:36:58 christo dovecot[279524]: IMAP(frank,127.0.0.1): ID sent: name=imapsync, version=1.977, os=linux, vendor=Gilles LAMIRAL, support-url=https://imapsync.lamiral.info/, date=23-Dec-2019 20:18:02 +0000, side=host2 Oct 28 11:37:37 christo dovecot[279524]: IMAP(frank,127.0.0.1): Disconnected: Logged out [79451/32883563]
If required here is the configuration which is rather old but worked until 2.3.16
# 2.3.16 (7e2e900c1a): /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 5.14.13-200.fc34.x86_64 x86_64 Fedora release 34 (Thirty Four) ext3 # Hostname: christo auth_mechanisms = plain login default_client_limit = 1024 default_process_limit = 256 default_vsz_limit = 512 M first_valid_uid = 200 last_valid_uid = 65534 listen = * lmtp_save_to_detail_mailbox = yes login_greeting = m28a.ddns.net - IMAPs Service (dovecot) ready. login_log_format_elements = %u %r %c mail_location = maildir:/var/spool/mail/%u:LAYOUT=fs mail_log_prefix = "%Us(%u,%r): " mail_plugin_dir = /usr/dovecot/lib/dovecot/ mail_plugins = notify quota fts fts_squat acl namespace inbox { inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Gesendet { special_use = \Sent } mailbox SPAM { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = subscriptions = yes type = private } passdb { args = dovecot driver = pam } plugin { fts = squat fts_squat = partial=4 full=10 mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box from subject msgid size flags mail_log_group_events = yes } postmaster_address = mailonator@mailbox.org protocols = imap service anvil { client_limit = 1024 } service auth { unix_listener auth-client { group = exim mode = 0660 user = exim } } service imap-login { inet_listener imap { port = 143 } process_limit = 512 process_min_avail = 3 } service imap-postlogin { executable = script-login /usr/local/sbin/dovecot-imap-post-login } service imap { executable = imap process_limit = 128 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3 { process_limit = 128 } ssl_cert =
What's going on? Any pointer to solution welcome.
Kind regards, Frank
On 28/10/2021 15:39 Frank Elsner frank.elsner@mailbox.org wrote:
On Thu, 28 Oct 2021 11:39:42 +0000 Aki Tuomi wrote:
Hi!
Can you provide gdb bt full output for this crash?
I fear this is far beyond my knowledge :-( but I will try.
Install debug symbols (if necessary) and
gdb /usr/libexec/dovecot/imap /path/to/core
What core?
--Frank
I guess you are using systemd-coredump, so try
coredumpctl dump -o core /usr/libexec/dovecot/imap
If this does not work, or you are not using systemd-coredump, please see https://www.dovecot.org/bugreport-mail/
Aki
On Thu, 28 Oct 2021 17:32:28 +0300 (EEST) Aki Tuomi wrote:
[ ... ]
I guess you are using systemd-coredump, so try
coredumpctl dump -o core /usr/libexec/dovecot/imap
That worked so I can provide
# coredumpctl dump -o core /usr/local/dovecot/libexec/dovecot/imap PID: 309338 (imap) UID: 1953 (frank) GID: 12203 (elsner) Signal: 11 (SEGV) Timestamp: Thu 2021-10-28 15:48:43 CEST (2h 19min ago) Command Line: dovecot/imap [frank 127.0.0.1 STATUS] Executable: /usr/local/dovecot/libexec/dovecot/imap Control Group: /system.slice/dovecot.service Unit: dovecot.service Slice: system.slice Boot ID: c97b6e2b6f464589b7a50fbf4009d2bb Machine ID: 5367c967725543b39d46d1b5b5b90fa8 Hostname: christo Storage: /var/lib/systemd/coredump/core.imap.1953.c97b6e2b6f464589b7a50fbf4009d2bb.309338.1635428923000000.zst (present) Disk Size: 252.6K Message: Process 309338 (imap) of user 1953 dumped core. Stack trace of thread 309338: #0 0x00007ff2193782cb fts_user_autoindex_exclude (lib20_fts_plugin.so + 0xa2cb) #1 0x00007ff2193813b6 fts_mailbox_allocated (lib20_fts_plugin.so + 0x133b6) #2 0x00007ff21984ad1c hook_mailbox_allocated (libdovecot-storage.so.0 + 0x62d1c) #3 0x00007ff219845f11 mailbox_alloc (libdovecot-storage.so.0 + 0x5df11) #4 0x000055d49e2003e5 imap_status_get (imap + 0x2b3e5) #5 0x000055d49e1f0c6e cmd_status (imap + 0x1bc6e) #6 0x000055d49e1f7494 command_exec (imap + 0x22494) #7 0x000055d49e1f54c2 client_command_input (imap + 0x204c2) #8 0x000055d49e1f5574 client_command_input (imap + 0x20574) #9 0x000055d49e1f597d client_handle_next_command (imap + 0x2097d) #10 0x000055d49e1f5f30 client_input (imap + 0x20f30) #11 0x00007ff219733249 io_loop_call_io (libdovecot.so.0 + 0x116249) #12 0x00007ff2197348f2 io_loop_handler_run_internal (libdovecot.so.0 + 0x1178f2) #13 0x00007ff2197332f0 io_loop_handler_run (libdovecot.so.0 + 0x1162f0) #14 0x00007ff2197334b0 io_loop_run (libdovecot.so.0 + 0x1164b0) #15 0x00007ff2196a74f3 master_service_run (libdovecot.so.0 + 0x8a4f3) #16 0x000055d49e1e6f85 main (imap + 0x11f85) #17 0x00007ff219453b75 __libc_start_main (libc.so.6 + 0x27b75) #18 0x000055d49e1e704e _start (imap + 0x1204e)
HTH, Frank
On Oct 28, 2021, at 5:12 AM, Aki Tuomi
We are pleased to release v2.3.17 of Dovecot.
This patch is still needed to build on newer MacOS -
--- src/lib/ioloop-notify-kqueue.c.orig 2021-06-14 07:56:46.000000000 -0400
+++ src/lib/ioloop-notify-kqueue.c 2021-06-21 12:10:16.000000000 -0400
@@ -11,6 +11,7 @@
#include "ioloop-private.h"
#include "llist.h"
+#include "time-util.h"
#include
Hi
THis is the coredump detail on 2.3.17
Returning to 2.3.16 resolves the issue
Oct 30 10:40:26 gjserver systemd-coredump[219812]: [🡕] Process 219810 (imap) of user 1003 dumped core.
Found module linux-vdso.so.1 with build-id: f076f68f712d35c37d7f60f9e2d7eaf4151ca1cc
Found module
libzstd.so.1 with build-id: 4b10444c1560ebc574af4d5f488b7408b22d450e
Found module
liblz4.so.1 with build-id: e63600ab23b2f6997f42fac2fa56e1f02ce159a1
Found module
libbz2.so.1.0 with build-id: 919597c477c9b2cb9cdbb7745ed6494ac0e6da60
Found module
lib30_imap_zlib_plugin.so with build-id:
e5787cb12d099c47090e0b24dd4f3521a5f7b551
Found module
libicuio.so.69 with build-id: dc3e5944cf30ba4aa85f1b88b7acaf886fd6ca10
Found module
libuuid.so.1 with build-id: 832a09e0d9568cc6dbe699472218ea2f79dc0f34
Found module
libz.so.1 with build-id: 81bf6e728a6d6f5b105b0f8b25f6c614ce10452a
Found module
librt.so.1 with build-id: 75484da2d6f1515189eefa076e0a40328834cd16
Found module
libxapian.so.30 with build-id: 254e2426aaf66d1055cd495518cad3f7610f165c
Found module
lib21_fts_xapian_plugin.so with build-id:
ee657eb1bc2813c5a465717eb35a64031d1b9953
Found module
libgcc_s.so.1 with build-id: 7f8508bb914546ada778809b64b99d234337d835
Found module
libm.so.6 with build-id: 2b8fd1f869ecab4e0b55e92f2f151897f6818acf
Found module
libstdc++.so.6 with build-id: 8ab0e57054dd1dcba681f217016afc6a4e639783
Found module
libicudata.so.69 with build-id: 0ab994a49ef1848499c4af333b3266f28432a922
Found module
libicuuc.so.69 with build-id: 5cf18c56e2f64efdac32cf61fb9c0c48e9bb1797
Found module
libicui18n.so.69 with build-id: 9cdecde5b2e47a2bd81dc14915cbfefcade76c12
Found module
libexttextcat-2.0.so.0 with build-id:
9c7e50b434ef8c70e32466ddf97c8c2499ca86ad
Found module
lib20_fts_plugin.so with build-id:
6a667a8c8d822d218e20805fcf5cf4b825013daa
Found module
libresolv.so.2 with build-id: c915c72668282861a813f7ea3c0780f37b681dc0
Found module
libkeyutils.so.1 with build-id: ac405ddd17be10ce538da3211415ee50c8f8df79
Found module
libkrb5support.so.0 with build-id:
adf65240a4d2aba772d7a0772b4d015469934113
Found module
libcom_err.so.2 with build-id: eb61ef71c8b97846db759fb89a115405cff6dd30
Found module
libk5crypto.so.3 with build-id: eb8220b8f36675aac769450be4cb6bb7f97ec38a
Found module
libkrb5.so.3 with build-id: 72d26767c5cb1097db75a5f5bff88860233c902b
Found module
liblzma.so.5 with build-id: 8b615460aa230708c5183f16bede67aa0437d95e
Found module
libpthread.so.0 with build-id: 07c8f95b4f3251d08550217ad8a1f31066229996
Found module
libgssapi_krb5.so.2 with build-id:
e6e098ad51ce7bdd3dbe902d7b0f69a90f8a9e08
Found module
ld-linux-x86-64.so.2 with build-id:
040cc3dd10461562f177df39e3be2f3704258c3c
Found module
libdl.so.2 with build-id: 5abc547e7b0949f89f3c0e21ab0c8331a7440a8a
Found module
libunwind.so.8 with build-id: de4aafba062ffac19b3a0cedace42d0616ef9c1e
Found module
libunwind-x86_64.so.8 with build-id:
4a26ca7953613d0ce48dbb5e609dcdec3d605671
Found module
libtirpc.so.3 with build-id: 5bef2adfdee3df283f593b3e2d37b6dac405256a
Found module
libc.so.6 with build-id: 4b406737057708c0e4c642345a703c47a61c73dc
Found module
libdovecot.so.0 with build-id: d71e0f8502f886f63db11138da1319862eba9731
Found module
libdovecot-storage.so.0 with build-id:
54ced32e49745ad3304af3342fcdf7aa8e43ea58
Found module imap
with build-id: ceb1dba5737de9ccb213eba33c65e613826bebf1
Stack trace of thread
219810:
#0
0x00007f602afc332b fts_user_autoindex_exclude (lib20_fts_plugin.so +
0xa32b)
#1
0x00007f602afcc496 fts_mailbox_allocated (lib20_fts_plugin.so + 0x13496)
#2
0x00007f602b659ccc hook_mailbox_allocated (libdovecot-storage.so.0 +
0x62ccc)
#3
0x00007f602b654ec1 mailbox_alloc (libdovecot-storage.so.0 + 0x5dec1)
#4
0x0000564d71cbd3e5 imap_status_get (imap + 0x2b3e5)
#5
0x0000564d71cadc6e cmd_status (imap + 0x1bc6e)
#6
0x0000564d71cb4494 command_exec (imap + 0x22494)
#7
0x0000564d71cb24c2 client_command_input (imap + 0x204c2)
#8
0x0000564d71cb2574 client_command_input (imap + 0x20574)
#9
0x0000564d71cb297d client_handle_next_command (imap + 0x2097d)
#10
0x0000564d71cb2f30 client_input (imap + 0x20f30)
#11
0x00007f602b5421a9 io_loop_call_io (libdovecot.so.0 + 0x1161a9)
#12
0x00007f602b543852 io_loop_handler_run_internal (libdovecot.so.0 +
0x117852)
#13
0x00007f602b542250 io_loop_handler_run (libdovecot.so.0 + 0x116250)
#14
0x00007f602b542410 io_loop_run (libdovecot.so.0 + 0x116410)
#15
0x00007f602b4b6453 master_service_run (libdovecot.so.0 + 0x8a453)
#16
0x0000564d71ca3f75 main (imap + 0x11f75)
#17
0x00007f602b287b25 __libc_start_main (libc.so.6 + 0x27b25)
#18
0x0000564d71ca403e _start (imap + 0x1203e)
Oct 30 10:40:26 gjserver dovecot[219074]:
imap(jom@grosjo.net)<219810>
(gdb) bt full
#0 fts_user_autoindex_exclude (box=<optimized out>,
box@entry=0x55cf79d865c8) at fts-user.c:347
fuser = <optimized out>
#1 0x00007f1f31198496 in fts_mailbox_allocated (box=0x55cf79d865c8) at
fts-storage.c:806
flist = <optimized out>
v = 0x55cf79d865f0
fbox = 0x55cf79d86bf8
#2 0x00007f1f31825ccc in hook_mailbox_allocated
(box=box@entry=0x55cf79d865c8) at mail-storage-hooks.c:256
_data_stack_cur_id = 5
_foreach_end = 0x55cf79d77088
_foreach_ptr = 0x55cf79d77078
hooks = 0x7f1f311c6ba0
On 2021-10-28 10:12, Aki Tuomi wrote:
We are pleased to release v2.3.17 of Dovecot.
Please note that 2.3.17 release will be the last one to support Debian/Stretch since Bullseye is now supported.
https://dovecot.org/releases/2.3/dovecot-2.3.17.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.17.tar.gz.sig Binary packages in https://repo.dovecot.org/ Docker images in https://hub.docker.com/r/dovecot/dovecot
Regards Aki Tuomi Open-Xchange oy
- Dovecot now logs a warning if time seems to jump forward at least 100 milliseconds.
- dict: Lines logged by the dict process now contain the dict name as the prefix.
- lib-index: mail_cache_fields, mail_always_cache_fields and mail_never_cache_fields now verifies that the listed header names are valid. Especially the UTF8 "-" character has sometimes been wrongly used instead of the ASCII "-".
- *-login: Added login_proxy_rawlog_dir setting to capture rawlogs between proxy and backend.
- dict: The server process now keeps the last 10 idle dict backends cached for maximum of 30 seconds. Practically this acts as a connection pool for dict-redis and dict-ldap. Note that this doesn't affect dict-sql, because it already had its own internal cache.
- doveadm: New stats add/remove commands added to support changing the metrics configuration on runtime.
- lazy_expunge: Added lazy_expunge_exclude settings to disable lazy_expunge for specific folders. \Special-use flags can be used as folder names.
- lib-lua: Added a new helper function dovecot.restrict_global_variables() to disable or enable defining new global variables.
- LAYOUT=index List index rebuild was missing.
- LAYOUT=index: Duplicate GUIDs were not detected.
- acl: When using acl_ignore_namespace Dovecot attempted to access or create dovecot-acl-list even when the namespace should have been ignored. For virtual namespaces this could have yielded errors about "Read-only file system" or "Permission denied".
- auth: Setting the "master" passdb field to empty value would cause proxying to fail with an authentication error. Now an empty "master" field is ignored.
- doveadm-server: Duplicate error lines were sent for failed commands. This didn't normally cause visible problems, except when using wildcards in usernames or -A parameter to go through multiple users.
- doveadm-server: Logs written by doveadm-server were often missing log prefixes, especially mail_log_prefix for mail commands. Logs sent to doveadm TCP client were also missing log prefixes.
- doveadm: v2.3 regression: batch command always crashes.
- doveadm: v2.3.11 regression: Commands failed if ssl_cert or ssl_key files weren't readable by the user running doveadm, even though doveadm didn't actually use these settings
- imap-hibernate: Process may crash at deinit: Panic: file ioloop.c: line 928 (io_loop_destroy): assertion failed: (ioloop->cur_ctx == NULL).
- imap: Using imap_fetch_failure=no-after can cause assert-crash with some IMAP commands if reading the mail fails (e.g. wrong cached mail size). Fixes: Panic: file index-mail-headers.c: line 198 (index_mail_parse_header_init): assertion failed: (!mail->data.header_parser_initialized)
- imap: v2.3.10 regression: When using INDEXPVT to enable private \Seen flags (for shared or public namespaces) the STORE command did not send untagged replies for the \Seen flag changes.
- imap: v2.3.15 regression: If PREVIEW/SNIPPET is not the final FETCH option in the command, the IMAP FETCH response is broken.
- imap: v2.3.15 regression: MOVE command leaks mailbox if it can't be opened and crashes at deinit: Panic: file mail-user.c: line 229 (mail_user_deinit): assertion failed: ((*user)->refcount == 1).
- imapc: Copying nonexistent mail via imapc could have crashed. Fixes: Panic: file mail-storage.c: line 2385 (mailbox_transaction_commit_get_changes): assertion failed: (ret < 0 || seq_range_count(&changes_r->saved_uids) == save_count || array_count(&changes_r->saved_uids) == 0).
- indexer: v2.3.15 regression: Process crashes if indexer-client disconnects while it's waiting for command reply. This happened for example if IMAP SEARCH triggered long fts indexing and the IMAP client disconnected while waiting for the reply.
- indexer: v2.3.15 regression: Process may have crashed in some situations.
- indexer: v2.3.15 regression: indexer-worker processes may not have reached the process_limit in some situations, possibly even using just one indexer-worker process even though there were many indexing requests queued.
- lib-compression: Reading lz4 compressed mdbox mails may crash. Fixes: Panic: file istream.c: line 345 (i_stream_read_memarea): assertion failed: (!stream->blocking).
- lib-compression: bench-compress crashes due to xz being read-only.
- lib-lua: Fix linking libdict_lua for non-GNU linkers when Lua support is disabled.
- lib-mail: There was no limit on how large an email header name could be. Processable header names are now limited to 1000 bytes.
- lib-oauth2: Dovecot disallowed JWT tokens if their validity time was older than token creation time (nbf < iat).
- lib-storage: Reduce memory footprint of certain storage operations.
- lib-storage: When listing mailboxes with storage name escape characters (^ or .) as part of the mailbox name, the listing could show corrupted mailbox names. Due to an issue in handling escaped parent folders, the listing of other mailbox names would become corrupted by prepending parts of the previously listed mailboxes parent folder as prefix to the actual mailbox names. The corruption can occur when using LAYOUT=INDEX and maildir or obox, or when using the listescape plugin.
- mail-crypt: Fix "-O" argument for "doveadm mailbox cryptokey password" command to be a boolean, and not expect a string.
- submission-login: Add support for not authenticating to next hop in submission proxying.
- submission-login: EHLO was not sent again after XCLIENT when doing submission proxying.
- virtual: Mailboxes do not correctly detect underlying mailboxes getting re-created even though they have a different UIDVALIDITY or GUID.
Hi
THis is the coredump detail on 2.3.17
Returning to 2.3.16 resolves the issue
(gdb) bt full
#0 fts_user_autoindex_exclude (box=<optimized out>,
box@entry=0x55cf79d865c8) at fts-user.c:347
fuser = <optimized out>
#1 0x00007f1f31198496 in fts_mailbox_allocated (box=0x55cf79d865c8) at
fts-storage.c:806
flist = <optimized out>
v = 0x55cf79d865f0
fbox = 0x55cf79d86bf8
#2 0x00007f1f31825ccc in hook_mailbox_allocated
(box=box@entry=0x55cf79d865c8) at mail-storage-hooks.c:256
_data_stack_cur_id = 5
_foreach_end = 0x55cf79d77088
_foreach_ptr = 0x55cf79d77078
hooks = 0x7f1f311c6ba0
Please convert all source code to ASCII. If it fails to compile, then it may have a trojan hiding in Unicode clothing.
-------- Original Message -------- On Oct 28, 2021, 11:12, Aki Tuomi wrote:
We are pleased to release v2.3.17 of Dovecot. Please note that 2.3.17 release will be the last one to support Debian/Stretch since Bullseye is now supported. https://dovecot.org/releases/2.3/dovecot-2.3.17.tar.gz https://dovecot.org/releases/2.3/dovecot-2.3.17.tar.gz.sig Binary packages in https://repo.dovecot.org/ Docker images in https://hub.docker.com/r/dovecot/dovecot Regards Aki Tuomi Open-Xchange oy --- * Dovecot now logs a warning if time seems to jump forward at least 100 milliseconds. * dict: Lines logged by the dict process now contain the dict name as the prefix. * lib-index: mail_cache_fields, mail_always_cache_fields and mail_never_cache_fields now verifies that the listed header names are valid. Especially the UTF8 "–" character has sometimes been wrongly used instead of the ASCII "-". + *-login: Added login_proxy_rawlog_dir setting to capture rawlogs between proxy and backend. + dict: The server process now keeps the last 10 idle dict backends cached for maximum of 30 seconds. Practically this acts as a connection pool for dict-redis and dict-ldap. Note that this doesn't affect dict-sql, because it already had its own internal cache. + doveadm: New stats add/remove commands added to support changing the metrics configuration on runtime. + lazy_expunge: Added lazy_expunge_exclude settings to disable lazy_expunge for specific folders. \Special-use flags can be used as folder names. + lib-lua: Added a new helper function dovecot.restrict_global_variables() to disable or enable defining new global variables. - LAYOUT=index List index rebuild was missing. - LAYOUT=index: Duplicate GUIDs were not detected. - acl: When using acl_ignore_namespace Dovecot attempted to access or create dovecot-acl-list even when the namespace should have been ignored. For virtual namespaces this could have yielded errors about "Read-only file system" or "Permission denied". - auth: Setting the "master" passdb field to empty value would cause proxying to fail with an authentication error. Now an empty "master" field is ignored. - doveadm-server: Duplicate error lines were sent for failed commands. This didn't normally cause visible problems, except when using wildcards in usernames or -A parameter to go through multiple users. - doveadm-server: Logs written by doveadm-server were often missing log prefixes, especially mail_log_prefix for mail commands. Logs sent to doveadm TCP client were also missing log prefixes. - doveadm: v2.3 regression: batch command always crashes. - doveadm: v2.3.11 regression: Commands failed if ssl_cert or ssl_key files weren't readable by the user running doveadm, even though doveadm didn't actually use these settings - imap-hibernate: Process may crash at deinit: Panic: file ioloop.c: line 928 (io_loop_destroy): assertion failed: (ioloop->cur_ctx == NULL). - imap: Using imap_fetch_failure=no-after can cause assert-crash with some IMAP commands if reading the mail fails (e.g. wrong cached mail size). Fixes: Panic: file index-mail-headers.c: line 198 (index_mail_parse_header_init): assertion failed: (!mail->data.header_parser_initialized) - imap: v2.3.10 regression: When using INDEXPVT to enable private \Seen flags (for shared or public namespaces) the STORE command did not send untagged replies for the \Seen flag changes. - imap: v2.3.15 regression: If PREVIEW/SNIPPET is not the final FETCH option in the command, the IMAP FETCH response is broken. - imap: v2.3.15 regression: MOVE command leaks mailbox if it can't be opened and crashes at deinit: Panic: file mail-user.c: line 229 (mail_user_deinit): assertion failed: ((*user)->refcount == 1). - imapc: Copying nonexistent mail via imapc could have crashed. Fixes: Panic: file mail-storage.c: line 2385 (mailbox_transaction_commit_get_changes): assertion failed: (ret saved_uids) == save_count || array_count(&changes_r->saved_uids) == 0). - indexer: v2.3.15 regression: Process crashes if indexer-client disconnects while it's waiting for command reply. This happened for example if IMAP SEARCH triggered long fts indexing and the IMAP client disconnected while waiting for the reply. - indexer: v2.3.15 regression: Process may have crashed in some situations. - indexer: v2.3.15 regression: indexer-worker processes may not have reached the process_limit in some situations, possibly even using just one indexer-worker process even though there were many indexing requests queued. - lib-compression: Reading lz4 compressed mdbox mails may crash. Fixes: Panic: file istream.c: line 345 (i_stream_read_memarea): assertion failed: (!stream->blocking). - lib-compression: bench-compress crashes due to xz being read-only. - lib-lua: Fix linking libdict_lua for non-GNU linkers when Lua support is disabled. - lib-mail: There was no limit on how large an email header name could be. Processable header names are now limited to 1000 bytes. - lib-oauth2: Dovecot disallowed JWT tokens if their validity time was older than token creation time (nbf
On Thu, 4 Nov 2021, Rupert Gallagher wrote:
Please convert all source code to ASCII. If it fails to compile, then it may have a trojan hiding in Unicode clothing.
Did you check yourself?
The only source code files which contain non-7-bit-ASCII characters are
- src/lib-storage/list/mailbox-list-index-status.c
- Opportunistic function to see ïf we can extract guid from mailbox path */
i.e. in a /* comment */, and it's 8-bit ASCII not even UTF-anything.
- src/lib-mail/test-qp-encoder.c which defines binary data.
I don't think any C compiler allows Unicode in the code itself (instructions, variables names, etc.)
Cheers.
The unicode hack is in the comments. Google "Trojan Source". Having never dealt with Hebrew and Arabic, it was news to me there is a right to left feature in Unicode.
TWIT Security Now (MP3): SN 843: Trojan Source - Chrome 0-days, Windows 11 confusion, VoIP DDos attacks, Dune https://pdst.fm/e/chtbl.com/track/E91833/cdn.twit.tv/audio/sn/sn0843/sn0843.... [01:19:28]
Or look for the paper. Hopefully this isn't too off topic.
Original Message
From: reinob@bbmk.org Sent: November 4, 2021 2:16 AM To: dovecot@dovecot.org Reply-to: dovecot@dovecot.org Subject: Re: Dovecot v2.3.17 released
On Thu, 4 Nov 2021, Rupert Gallagher wrote:
Please convert all source code to ASCII. If it fails to compile, then it may have a trojan hiding in Unicode clothing.
Did you check yourself?
The only source code files which contain non-7-bit-ASCII characters are
- src/lib-storage/list/mailbox-list-index-status.c * Opportunistic function to see ïf we can extract guid from mailbox path */
i.e. in a /* comment */, and it's 8-bit ASCII not even UTF-anything.
- src/lib-mail/test-qp-encoder.c which defines binary data.
I don't think any C compiler allows Unicode in the code itself (instructions, variables names, etc.)
Cheers.
just for reference https://trojansource.codes/
On 04.11.21 10:50, lists wrote:
The unicode hack is in the comments. Google "Trojan Source". Having never dealt with Hebrew and Arabic, it was news to me there is a right to left feature in Unicode.
TWIT Security Now (MP3): SN 843: Trojan Source - Chrome 0-days, Windows 11 confusion, VoIP DDos attacks, Dune https://pdst.fm/e/chtbl.com/track/E91833/cdn.twit.tv/audio/sn/sn0843/sn0843.... [01:19:28]
Or look for the paper. Hopefully this isn't too off topic.
Original Message
From: reinob@bbmk.org Sent: November 4, 2021 2:16 AM To: dovecot@dovecot.org Reply-to: dovecot@dovecot.org Subject: Re: Dovecot v2.3.17 released
On Thu, 4 Nov 2021, Rupert Gallagher wrote:
Please convert all source code to ASCII. If it fails to compile, then it may have a trojan hiding in Unicode clothing. Did you check yourself?
The only source code files which contain non-7-bit-ASCII characters are
- src/lib-storage/list/mailbox-list-index-status.c * Opportunistic function to see ïf we can extract guid from mailbox path */
i.e. in a /* comment */, and it's 8-bit ASCII not even UTF-anything.
- src/lib-mail/test-qp-encoder.c which defines binary data.
I don't think any C compiler allows Unicode in the code itself (instructions, variables names, etc.)
Cheers.
On Thu, 4 Nov 2021, infoomatic wrote:
just for reference https://trojansource.codes/
Yup. But as I wrote before, at least in the downloadable .tar.gz for version 2.3.17 there is not a single Unicode-but-not-ASCII character (and therefore no Right-to-Left override/isolate control characters), and even the only non-7-bit-ASCII characters are in harmless places.
[ rest deleted due to top-posting mess.. ]
participants (9)
-
Aki Tuomi
-
Aki Tuomi
-
Bernardo Reino
-
Daniel J. Luke
-
Frank Elsner
-
infoomatic
-
Joan Moreau
-
lists
-
Rupert Gallagher