Sorry, the provided patch link was wrong, it's already in 2.3.21, my bad. Anyways, it is still fixed in main, since it does not happen there.

Aki

On 19/01/2024 09:13 EET Aki Tuomi via dovecot dovecot@dovecot.org wrote:

Hi!

I was able to reproduce this issue with 2.3.21, but it seems to have been fixed in main. I think https://github.com/dovecot/core/commit/1c1b77dbf9a548aac788efb76973ce2d0fa6c... will fix this.

Aki

...

On 18/01/2024 22:51 EET John van der Kamp via dovecot dovecot@dovecot.org wrote:

Hello,

I've found a crash in a very specific setup. A dovecot server with imapc connection needs to receive an email with no body contents for the intent of generating a preview/snippet. It crashes somewhere deep in the jungle of istream and snapshots. I've included a script which sets up the systems to reproduce the crash.

I've tested this with several versions. 2.3.16 doesn't seem to be affected, but 2.3.20 and 2.3.21 are affect.

For me it produces a traceback like this, using the ubuntu version from here: https://packages.ubuntu.com/noble/dovecot-core

(gdb) bt #0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:44 #1  __pthread_kill_internal (signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:78 #2  __GI___pthread_kill (threadid=140530132887360, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 #3  0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #4  0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79 #5  0x00007fcfb8b37fe5 in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at ../lib/failures.c:465 #6  fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:477 #7  0x00007fcfb8be50d7 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:879 #8  0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530 #9  0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:663 #10 i_stream_header_filter_snapshot_free (_snapshot=0x55dabe297a60) at ../lib-mail/istream-header-filter.c:655 #11 0x00007fcfb8bf25ac in i_stream_snapshot_free (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253 #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0) at ../lib/istream.c:66 #13 0x00007fcfb8d96baa in index_mail_write_body_snippet (mail=0x55dabe292058) at index/index-mail.c:1151 #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551 #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:1602 #16 index_mail_get_special (_mail=0x55dabe292058, field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:1730 #17 0x00007fcfb8d16ffe in mail_get_special (mail=mail@entry=0x55dabe292058, field=field@entry=MAIL_FETCH_BODY_SNIPPET, value_r=value_r@entry=0x7ffc16cc8050)     at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-storage/mail.c:418 #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050, mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-fetch-body.c:615 #19 0x000055dabc52b5cc in imap_fetch_more_int (ctx=0x55dabe26e050, cancel=false) at ./src/imap/imap-fetch.c:562 #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050, cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617 #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./src/imap/cmd-fetch.c:382 #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./src/imap/imap-commands.c:201 #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized out>) at ./src/imap/imap-client.c:1237 #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized out>) at ./src/imap/imap-client.c:1307 #25 0x000055dabc52eeed in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1349 #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1363 #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1407 #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at ../lib/ioloop.c:737 #29 0x00007fcfb8bff81a in io_loop_handler_run_internal (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222 #30 0x00007fcfb8bff8d4 in io_loop_handler_run (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:789 #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:762 #32 0x00007fcfb8b6ce57 in master_service_run (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210 <client_connected>) at ../lib-master/master-service.c:878 #33 0x000055dabc51ad37 in main (argc=<optimized out>, argv=<optimized out>) at ./src/imap/main.c:575

John

Hello,   I've found a crash in a very specific setup. A dovecot server with imapc connection needs to receive an email with no body contents for the intent of generating a preview/snippet. It crashes somewhere deep in the jungle of istream and snapshots. I've included a script which sets up the systems to reproduce the crash.   I've tested this with several versions. 2.3.16 doesn't seem to be affected, but 2.3.20 and 2.3.21 are affect.   For me it produces a traceback like this, using the ubuntu version from here: https://packages.ubuntu.com/noble/dovecot-core   (gdb) bt #0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:44 #1  __pthread_kill_internal (signo=6, threadid=140530132887360) at ./nptl/ pthread_kill.c:78 #2  __GI___pthread_kill (threadid=140530132887360, signo=signo@entry=6) at ./ nptl/pthread_kill.c:89 #3  0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/ raise.c:26 #4  0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79 #5  0x00007fcfb8b37fe5 in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at ../lib/failures.c:465 #6  fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:477 #7  0x00007fcfb8be50d7 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:879 #8  0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530 #9  0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:663 #10 i_stream_header_filter_snapshot_free (_snapshot=0x55dabe297a60) at ../lib- mail/istream-header-filter.c:655 #11 0x00007fcfb8bf25ac in i_stream_snapshot_free (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253 #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0) at ../lib/ istream.c:66 #13 0x00007fcfb8d96baa in index_mail_write_body_snippet (mail=0x55dabe292058) at index/index-mail.c:1151 #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551 #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:1602 #16 index_mail_get_special (_mail=0x55dabe292058, field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:1730 #17 0x00007fcfb8d16ffe in mail_get_special (mail=mail@entry=0x55dabe292058, field=field@entry=MAIL_FETCH_BODY_SNIPPET, value_r=value_r@entry=0x7ffc16cc8050)     at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-storage/mail.c:418 #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050, mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-fetch-body.c: 615 #19 0x000055dabc52b5cc in imap_fetch_more_int (ctx=0x55dabe26e050, cancel=false) at ./src/imap/imap-fetch.c:562 #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050, cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617 #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./src/imap/cmd- fetch.c:382 #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./src/imap/imap- commands.c:201 #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized out>) at ./src/ imap/imap-client.c:1237 #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized out>) at ./src/ imap/imap-client.c:1307 #25 0x000055dabc52eeed in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1349 #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c: 1363 #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at ./src/imap/ imap-client.c:1407 #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at ../lib/ ioloop.c:737 #29 0x00007fcfb8bff81a in io_loop_handler_run_internal (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222 #30 0x00007fcfb8bff8d4 in io_loop_handler_run (ioloop=0x55dabe243fd0) at ../ lib/ioloop.c:789 #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at ../lib/ ioloop.c:762 #32 0x00007fcfb8b6ce57 in master_service_run (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210 <client_connected>) at ../lib-master/ master-service.c:878 #33 0x000055dabc51ad37 in main (argc=<optimized out>, argv=<optimized out>) at ./src/imap/main.c:575   John   _______________________________________________ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

---

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

Could you provide some simple way to reproduce this, minimal config etc?

Aki

On 19/03/2024 17:44 EET John van der Kamp via dovecot dovecot@dovecot.org wrote:

Hi, sorry for the late reply.

The commit you've pointed at before is the commit introducing code for the snippets.

Your claim that main is fixed is incorrect: I've bisected through the git history, and the commit that "fixes" it, is the one flipping imapc features to negatives: https://github.com/dovecot/core/commit/7810b38d30b7dbb2155f78873fe760bc9e2e6... https://github.com/dovecot/core/commit/7810b38d30b7dbb2155f78873fe760bc9e2e6212 However, the default imapc_features value stays the same, so all the "negative" features are suddenly enabled.

I've reset the defaults in the frontend config to what it was before:

imapc_features = no-fetch-size no-fetch-headers no-search no-modseq no-delay-login no-fetch-bodystructure no-acl

and then dovecot starts crashing again in the described scenario. It is the "no-fetch-size" flag, and if I use "rfc822.size" feature on a 2.3 branch it stops crashing.

Turns out this same feature adds some filter that seems to be meant for some exchange email side-effect: https://github.com/dovecot/core/blob/main/src/lib-storage/index/imapc/imapc-... https://github.com/dovecot/core/blob/main/src/lib-storage/index/imapc/imapc-mail-fetch.c#L596   where this filter tries to remove any X-Message-Flag header. This is weird, because it could have been an normally received header as well as something that was tacked on later by exchange.

The main bug is not fixed by just removing that filter: chaining filters is probably very broken when using the imapc backend, and it might be broken in other unknown scenarios.

Regards,

John

-----Original message----- From: Aki Tuomi via dovecot dovecot@dovecot.org Sent: Friday, 19th January 2024, 8:37 To: Aki Tuomi via dovecot dovecot@dovecot.org; John van der Kamp jkamp@amazon.nl Subject: RE: Crash in dovecot snippet when using imapc   CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

Sorry, the provided patch link was wrong, it's already in 2.3.21, my bad. Anyways, it is still fixed in main, since it does not happen there.

Aki

...

On 19/01/2024 09:13 EET Aki Tuomi via dovecot dovecot@dovecot.org wrote:

Hi!

I was able to reproduce this issue with 2.3.21, but it seems to have been fixed in main. I think https://github.com/dovecot/core/commit/1c1b77dbf9a548aac788efb76973ce2d0fa6c... will fix this.

Aki

...

On 18/01/2024 22:51 EET John van der Kamp via dovecot dovecot@dovecot.org wrote:

Hello,

I've found a crash in a very specific setup. A dovecot server with imapc connection needs to receive an email with no body contents for the intent of generating a preview/snippet. It crashes somewhere deep in the jungle of istream and snapshots. I've included a script which sets up the systems to reproduce the crash.

I've tested this with several versions. 2.3.16 doesn't seem to be affected, but 2.3.20 and 2.3.21 are affect.

For me it produces a traceback like this, using the ubuntu version from here: https://packages.ubuntu.com/noble/dovecot-core

(gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=140530132887360, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 #3 0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #4 0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79 #5 0x00007fcfb8b37fe5 in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at ../lib/failures.c:465 #6 fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:477 #7 0x00007fcfb8be50d7 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:879 #8 0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530 #9 0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:663 #10 i_stream_header_filter_snapshot_free (_snapshot=0x55dabe297a60) at ../lib-mail/istream-header-filter.c:655 #11 0x00007fcfb8bf25ac in i_stream_snapshot_free (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253 #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0) at ../lib/istream.c:66 #13 0x00007fcfb8d96baa in index_mail_write_body_snippet (mail=0x55dabe292058) at index/index-mail.c:1151 #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551 #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:1602 #16 index_mail_get_special (_mail=0x55dabe292058, field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:1730 #17 0x00007fcfb8d16ffe in mail_get_special (mail=mail@entry=0x55dabe292058, field=field@entry=MAIL_FETCH_BODY_SNIPPET, value_r=value_r@entry=0x7ffc16cc8050) at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-storage/mail.c:418 #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050, mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-fetch-body.c:615 #19 0x000055dabc52b5cc in imap_fetch_more_int (ctx=0x55dabe26e050, cancel=false) at ./src/imap/imap-fetch.c:562 #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050, cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617 #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./src/imap/cmd-fetch.c:382 #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./src/imap/imap-commands.c:201 #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized out>) at ./src/imap/imap-client.c:1237 #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized out>) at ./src/imap/imap-client.c:1307 #25 0x000055dabc52eeed in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1349 #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1363 #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1407 #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at ../lib/ioloop.c:737 #29 0x00007fcfb8bff81a in io_loop_handler_run_internal (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222 #30 0x00007fcfb8bff8d4 in io_loop_handler_run (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:789 #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:762 #32 0x00007fcfb8b6ce57 in master_service_run (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210 <client_connected>) at ../lib-master/master-service.c:878 #33 0x000055dabc51ad37 in main (argc=<optimized out>, argv=<optimized out>) at ./src/imap/main.c:575

John

Hello,

I've found a crash in a very specific setup. A dovecot server with imapc connection needs to receive an email with no body contents for the intent of generating a preview/snippet. It crashes somewhere deep in the jungle of istream and snapshots. I've included a script which sets up the systems to reproduce the crash.

I've tested this with several versions. 2.3.16 doesn't seem to be affected, but 2.3.20 and 2.3.21 are affect.

For me it produces a traceback like this, using the ubuntu version from here: https://packages.ubuntu.com/noble/dovecot-core

(gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=140530132887360) at ./nptl/ pthread_kill.c:78 #2 __GI___pthread_kill (threadid=140530132887360, signo=signo@entry=6) at ./ nptl/pthread_kill.c:89 #3 0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/ raise.c:26 #4 0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79 #5 0x00007fcfb8b37fe5 in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at ../lib/failures.c:465 #6 fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:477 #7 0x00007fcfb8be50d7 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:879 #8 0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530 #9 0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:663 #10 i_stream_header_filter_snapshot_free (_snapshot=0x55dabe297a60) at ../lib- mail/istream-header-filter.c:655 #11 0x00007fcfb8bf25ac in i_stream_snapshot_free (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253 #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0) at ../lib/ istream.c:66 #13 0x00007fcfb8d96baa in index_mail_write_body_snippet (mail=0x55dabe292058) at index/index-mail.c:1151 #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551 #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:1602 #16 index_mail_get_special (_mail=0x55dabe292058, field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:1730 #17 0x00007fcfb8d16ffe in mail_get_special (mail=mail@entry=0x55dabe292058, field=field@entry=MAIL_FETCH_BODY_SNIPPET, value_r=value_r@entry=0x7ffc16cc8050) at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-storage/mail.c:418 #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050, mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-fetch-body.c: 615 #19 0x000055dabc52b5cc in imap_fetch_more_int (ctx=0x55dabe26e050,

...

...

cancel=false) at ./src/imap/imap-fetch.c:562 #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050, cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617 #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./src/imap/cmd- fetch.c:382 #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./src/imap/imap- commands.c:201 #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized out>) at ./src/ imap/imap-client.c:1237 #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized out>) at ./src/ imap/imap-client.c:1307 #25 0x000055dabc52eeed in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1349 #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c: 1363 #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at ./src/imap/ imap-client.c:1407 #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at ../lib/ ioloop.c:737 #29 0x00007fcfb8bff81a in io_loop_handler_run_internal (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222 #30 0x00007fcfb8bff8d4 in io_loop_handler_run (ioloop=0x55dabe243fd0) at ../ lib/ioloop.c:789 #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at ../lib/ ioloop.c:762 #32 0x00007fcfb8b6ce57 in master_service_run (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210 <client_connected>) at ../lib-master/ master-service.c:878 #33 0x000055dabc51ad37 in main (argc=<optimized out>, argv=<optimized out>) at ./src/imap/main.c:575

John

---

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

---

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

---

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

Amazon Development Center (Netherlands) B.V., Johanna Westerdijkplein 1, NL-2521 EN The Hague, Registration No. Chamber of Commerce 56869649, VAT: NL 852339859B01

Hi, sorry for the late reply. The commit you've pointed at before is the commit introducing code for the snippets. Your claim that main is fixed is incorrect: I've bisected through the git history, and the commit that "fixes" it, is the one flipping imapc features to negatives: https://github.com/dovecot/core/commit/ 7810b38d30b7dbb2155f78873fe760bc9e2e6212 However, the default imapc_features value stays the same, so all the "negative" features are suddenly enabled. I've reset the defaults in the frontend config to what it was before: imapc_features = no-fetch-size no-fetch-headers no-search no-modseq no-delay- login no-fetch-bodystructure no-acl and then dovecot starts crashing again in the described scenario. It is the "no-fetch-size" flag, and if I use "rfc822.size" feature on a 2.3 branch it stops crashing.   Turns out this same feature adds some filter that seems to be meant for some exchange email side-effect: https://github.com/dovecot/core/blob/main/src/lib- storage/index/imapc/imapc-mail-fetch.c#L596  where this filter tries to remove any X-Message-Flag header. This is weird, because it could have been an normally received header as well as something that was tacked on later by exchange.   The main bug is not fixed by just removing that filter: chaining filters is probably very broken when using the imapc backend, and it might be broken in other unknown scenarios.   Regards,   John     -----Original message----- From: Aki Tuomi via dovecot dovecot@dovecot.org Sent: Friday, 19th January 2024, 8:37 To: Aki Tuomi via dovecot dovecot@dovecot.org; John van der Kamp jkamp@amazon.nl Subject: RE: Crash in dovecot snippet when using imapc   CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 Sorry, the provided patch link was wrong, it's already in 2.3.21, my
 bad. Anyways, it is still fixed in main, since it does not happen
 there.

 Aki

 > On 19/01/2024 09:13 EET Aki Tuomi via dovecot <dovecot@dovecot.org>
 wrote:
 >
 >
 > Hi!
 >
 > I was able to reproduce this issue with 2.3.21, but it seems to
 have been fixed in main. I think https://github.com/dovecot/core/
 commit/1c1b77dbf9a548aac788efb76973ce2d0fa6c732.patch will fix this.
 >
 > Aki
 >
 > > On 18/01/2024 22:51 EET John van der Kamp via dovecot
 <dovecot@dovecot.org> wrote:
 > >
 > >
 > > Hello,
 > >
 > >
 > > I've found a crash in a very specific setup. A dovecot server
 with imapc connection needs to receive an email with no body contents
 for the intent of generating a preview/snippet. It crashes somewhere
 deep in the jungle of istream and snapshots. I've included a script
 which sets up the systems to reproduce the crash.
 > >
 > >
 > > I've tested this with several versions. 2.3.16 doesn't seem to be
 affected, but 2.3.20 and 2.3.21 are affect.
 > >
 > >
 > > For me it produces a traceback like this, using the ubuntu
 version from here: https://packages.ubuntu.com/noble/dovecot-core
 > >
 > >
 > > (gdb) bt
 > > #0  __pthread_kill_implementation (no_tid=0, signo=6,
 threadid=140530132887360) at ./nptl/pthread_kill.c:44
 > > #1  __pthread_kill_internal (signo=6, threadid=140530132887360)
 at ./nptl/pthread_kill.c:78
 > > #2  __GI___pthread_kill (threadid=140530132887360,
 signo=signo@entry=6) at ./nptl/pthread_kill.c:89
 > > #3  0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../
 sysdeps/posix/raise.c:26
 > > #4  0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79
 > > #5  0x00007fcfb8b37fe5 in default_fatal_finish (status=0,
 type=LOG_TYPE_PANIC) at ../lib/failures.c:465
 > > #6  fatal_handler_real (ctx=<optimized out>, format=<optimized
 out>, args=<optimized out>) at ../lib/failures.c:477
 > > #7  0x00007fcfb8be50d7 in i_internal_fatal_handler
 (ctx=<optimized out>, format=<optimized out>, args=<optimized out>)
 at ../lib/failures.c:879
 > > #8  0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file
 %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530
 > > #9  0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free
 (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:
 663
 > > #10 i_stream_header_filter_snapshot_free
 (_snapshot=0x55dabe297a60) at ../lib-mail/istream-header-filter.c:655
 > > #11 0x00007fcfb8bf25ac in i_stream_snapshot_free
 (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253
 > > #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0)
 at ../lib/istream.c:66
 > > #13 0x00007fcfb8d96baa in index_mail_write_body_snippet
 (mail=0x55dabe292058) at index/index-mail.c:1151
 > > #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure
 (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-
 mail.c:1551
 > > #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet
 (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:
 1602
 > > #16 index_mail_get_special (_mail=0x55dabe292058,
 field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:
 1730
 > > #17 0x00007fcfb8d16ffe in mail_get_special
 (mail=mail@entry=0x55dabe292058,
 field=field@entry=MAIL_FETCH_BODY_SNIPPET,
 value_r=value_r@entry=0x7ffc16cc8050)
 > >     at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-
 storage/mail.c:418
 > > #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050,
 mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-
 fetch-body.c:615
 > > #19 0x000055dabc52b5cc in imap_fetch_more_int
 (ctx=0x55dabe26e050, cancel=false) at ./src/imap/imap-fetch.c:562
 > > #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050,
 cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617
 > > #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./
 src/imap/cmd-fetch.c:382
 > > #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./
 src/imap/imap-commands.c:201
 > > #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized
 out>) at ./src/imap/imap-client.c:1237
 > > #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized
 out>) at ./src/imap/imap-client.c:1307
 > > #25 0x000055dabc52eeed in client_handle_next_command
 (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/
 imap/imap-client.c:1349
 > > #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/
 imap-client.c:1363
 > > #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at
 ./src/imap/imap-client.c:1407
 > > #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at
 ../lib/ioloop.c:737
 > > #29 0x00007fcfb8bff81a in io_loop_handler_run_internal
 (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222
 > > #30 0x00007fcfb8bff8d4 in io_loop_handler_run
 (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:789
 > > #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at
 ../lib/ioloop.c:762
 > > #32 0x00007fcfb8b6ce57 in master_service_run
 (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210
 <client_connected>) at ../lib-master/master-service.c:878
 > > #33 0x000055dabc51ad37 in main (argc=<optimized out>,
 argv=<optimized out>) at ./src/imap/main.c:575
 > >
 > >
 > > John
 > >
 > >
 > >
 > > Hello,
 > >
 > > I've found a crash in a very specific setup. A dovecot server
 with imapc
 > > connection needs to receive an email with no body contents for
 the intent of
 > > generating a preview/snippet. It crashes somewhere deep in the
 jungle of
 > > istream and snapshots. I've included a script which sets up the
 systems to
 > > reproduce the crash.
 > >
 > > I've tested this with several versions. 2.3.16 doesn't seem to be
 affected, but
 > > 2.3.20 and 2.3.21 are affect.
 > >
 > > For me it produces a traceback like this, using the ubuntu
 version from here:
 > > https://packages.ubuntu.com/noble/dovecot-core
 > >
 > > (gdb) bt
 > > #0  __pthread_kill_implementation (no_tid=0, signo=6,
 threadid=140530132887360)
 > > at ./nptl/pthread_kill.c:44
 > > #1  __pthread_kill_internal (signo=6, threadid=140530132887360)
 at ./nptl/
 > > pthread_kill.c:78
 > > #2  __GI___pthread_kill (threadid=140530132887360,
 signo=signo@entry=6) at ./
 > > nptl/pthread_kill.c:89
 > > #3  0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../
 sysdeps/posix/
 > > raise.c:26
 > > #4  0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79
 > > #5  0x00007fcfb8b37fe5 in default_fatal_finish (status=0,
 type=LOG_TYPE_PANIC)
 > > at ../lib/failures.c:465
 > > #6  fatal_handler_real (ctx=<optimized out>, format=<optimized
 out>,
 > > args=<optimized out>) at ../lib/failures.c:477
 > > #7  0x00007fcfb8be50d7 in i_internal_fatal_handler
 (ctx=<optimized out>,
 > > format=<optimized out>, args=<optimized out>) at ../lib/
 failures.c:879
 > > #8  0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file
 %s: line %d
 > > (%s): assertion failed: (%s)") at ../lib/failures.c:530
 > > #9  0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free
 > > (_snapshot=<optimized out>) at ../lib-mail/istream-header-
 filter.c:663
 > > #10 i_stream_header_filter_snapshot_free
 (_snapshot=0x55dabe297a60) at ../lib-
 > > mail/istream-header-filter.c:655
 > > #11 0x00007fcfb8bf25ac in i_stream_snapshot_free
 > > (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:
 253
 > > #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0)
 at ../lib/
 > > istream.c:66
 > > #13 0x00007fcfb8d96baa in index_mail_write_body_snippet
 (mail=0x55dabe292058)
 > > at index/index-mail.c:1151
 > > #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure
 (mail=0x55dabe292058,
 > > field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551
 > > #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet
 > > (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-
 mail.c:1602
 > > #16 index_mail_get_special (_mail=0x55dabe292058,
 field=<optimized out>,
 > > value_r=0x7ffc16cc8050) at index/index-mail.c:1730
 > > #17 0x00007fcfb8d16ffe in mail_get_special
 (mail=mail@entry=0x55dabe292058,
 > > field=field@entry=MAIL_FETCH_BODY_SNIPPET,
 > > value_r=value_r@entry=0x7ffc16cc8050)
 > >     at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-
 storage/mail.c:418
 > > #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050,
 > > mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-
 fetch-body.c:
 > > 615
 > > #19 0x000055dabc52b5cc in imap_fetch_more_int
 (ctx=0x55dabe26e050,
 > > cancel=false) at ./src/imap/imap-fetch.c:562
 > > #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050,
 > > cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617
 > > #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./
 src/imap/cmd-
 > > fetch.c:382
 > > #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./
 src/imap/imap-
 > > commands.c:201
 > > #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized
 out>) at ./src/
 > > imap/imap-client.c:1237
 > > #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized
 out>) at ./src/
 > > imap/imap-client.c:1307
 > > #25 0x000055dabc52eeed in client_handle_next_command
 (remove_io_r=<synthetic
 > > pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1349
 > > #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/
 imap-client.c:
 > > 1363
 > > #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at
 ./src/imap/
 > > imap-client.c:1407
 > > #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at
 ../lib/
 > > ioloop.c:737
 > > #29 0x00007fcfb8bff81a in io_loop_handler_run_internal
 (ioloop=0x55dabe243fd0)
 > > at ../lib/ioloop-epoll.c:222
 > > #30 0x00007fcfb8bff8d4 in io_loop_handler_run
 (ioloop=0x55dabe243fd0) at ../
 > > lib/ioloop.c:789
 > > #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at
 ../lib/
 > > ioloop.c:762
 > > #32 0x00007fcfb8b6ce57 in master_service_run
 (service=0x55dabe243e20,
 > > callback=callback@entry=0x55dabc533210 <client_connected>) at ../
 lib-master/
 > > master-service.c:878
 > > #33 0x000055dabc51ad37 in main (argc=<optimized out>,
 argv=<optimized out>) at
 > > ./src/imap/main.c:575
 > >
 > > John
 > >
 > > _______________________________________________
 > > dovecot mailing list -- dovecot@dovecot.org
 > > To unsubscribe send an email to dovecot-leave@dovecot.org
 > _______________________________________________
 > dovecot mailing list -- dovecot@dovecot.org
 > To unsubscribe send an email to dovecot-leave@dovecot.org
 _______________________________________________
 dovecot mailing list -- dovecot@dovecot.org
 To unsubscribe send an email to dovecot-leave@dovecot.org



 Amazon Development Center (Netherlands) B.V., Johanna Westerdijkplein
 1, NL-2521 EN The Hague, Registration No. Chamber of Commerce
 56869649, VAT: NL 852339859B01

---

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

Hi!

I can see the crash now, thanks.

For 2.3, I would suggest as workaround to enable the fetch-size imapc_feature.

Aki

On 20/03/2024 09:58 EET John van der Kamp via dovecot dovecot@dovecot.org wrote:

See attached script I used. If you get EOF then you've hit the crash.

John

-----Original message----- From: Aki Tuomi aki.tuomi@open-xchange.com Sent: Wednesday, 20th March 2024, 8:40 To: John van der Kamp jkamp@amazon.nl; John van der Kamp via dovecot dovecot@dovecot.org Subject: RE: Crash in dovecot snippet when using imapc   CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

Could you provide some simple way to reproduce this, minimal config etc?

Aki

...

On 19/03/2024 17:44 EET John van der Kamp via dovecot dovecot@dovecot.org wrote:

Hi, sorry for the late reply.

The commit you've pointed at before is the commit introducing code for the snippets.

Your claim that main is fixed is incorrect: I've bisected through the git history, and the commit that "fixes" it, is the one flipping imapc features to negatives: https://github.com/dovecot/core/commit/7810b38d30b7dbb2155f78873fe760bc9e2e6... https://github.com/dovecot/core/commit/7810b38d30b7dbb2155f78873fe760bc9e2e6212 However, the default imapc_features value stays the same, so all the "negative" features are suddenly enabled.

I've reset the defaults in the frontend config to what it was before:

imapc_features = no-fetch-size no-fetch-headers no-search no-modseq no-delay-login no-fetch-bodystructure no-acl

and then dovecot starts crashing again in the described scenario. It is the "no-fetch-size" flag, and if I use "rfc822.size" feature on a 2.3 branch it stops crashing.

Turns out this same feature adds some filter that seems to be meant for some exchange email side-effect: https://github.com/dovecot/core/blob/main/src/lib-storage/index/imapc/imapc-... https://github.com/dovecot/core/blob/main/src/lib-storage/index/imapc/imapc-mail-fetch.c#L596 where this filter tries to remove any X-Message-Flag header. This is weird, because it could have been an normally received header as well as something that was tacked on later by exchange.

The main bug is not fixed by just removing that filter: chaining filters is probably very broken when using the imapc backend, and it might be broken in other unknown scenarios.

Regards,

John

-----Original message----- From: Aki Tuomi via dovecot dovecot@dovecot.org Sent: Friday, 19th January 2024, 8:37 To: Aki Tuomi via dovecot dovecot@dovecot.org; John van der Kamp jkamp@amazon.nl Subject: RE: Crash in dovecot snippet when using imapc

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

Sorry, the provided patch link was wrong, it's already in 2.3.21, my bad. Anyways, it is still fixed in main, since it does not happen there.

Aki

...

On 19/01/2024 09:13 EET Aki Tuomi via dovecot dovecot@dovecot.org wrote:

Hi!

I was able to reproduce this issue with 2.3.21, but it seems to have been fixed in main. I think https://github.com/dovecot/core/commit/1c1b77dbf9a548aac788efb76973ce2d0fa6c... will fix this.

Aki

...

On 18/01/2024 22:51 EET John van der Kamp via dovecot dovecot@dovecot.org wrote:

Hello,

I've found a crash in a very specific setup. A dovecot server with imapc connection needs to receive an email with no body contents for the intent of generating a preview/snippet. It crashes somewhere deep in the jungle of istream and snapshots. I've included a script which sets up the systems to reproduce the crash.

I've tested this with several versions. 2.3.16 doesn't seem to be affected, but 2.3.20 and 2.3.21 are affect.

For me it produces a traceback like this, using the ubuntu version from here: https://packages.ubuntu.com/noble/dovecot-core

(gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=140530132887360, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 #3 0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 #4 0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79 #5 0x00007fcfb8b37fe5 in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at ../lib/failures.c:465 #6 fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:477 #7 0x00007fcfb8be50d7 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:879 #8 0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530 #9 0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:663 #10 i_stream_header_filter_snapshot_free (_snapshot=0x55dabe297a60) at ../lib-mail/istream-header-filter.c:655 #11 0x00007fcfb8bf25ac in i_stream_snapshot_free (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253 #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0) at ../lib/istream.c:66 #13 0x00007fcfb8d96baa in index_mail_write_body_snippet (mail=0x55dabe292058) at index/index-mail.c:1151 #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551 #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:1602 #16 index_mail_get_special (_mail=0x55dabe292058, field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:1730 #17 0x00007fcfb8d16ffe in mail_get_special (mail=mail@entry=0x55dabe292058, field=field@entry=MAIL_FETCH_BODY_SNIPPET, value_r=value_r@entry=0x7ffc16cc8050) at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-storage/mail.c:418 #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050, mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-fetch-body.c:615 #19 0x000055dabc52b5cc in imap_fetch_more_int (ctx=0x55dabe26e050, cancel=false) at ./src/imap/imap-fetch.c:562 #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050, cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617 #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./src/imap/cmd-fetch.c:382 #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./src/imap/imap-commands.c:201 #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized out>) at ./src/imap/imap-client.c:1237 #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized out>) at ./src/imap/imap-client.c:1307 #25 0x000055dabc52eeed in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1349 #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1363 #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1407 #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at ../lib/ioloop.c:737 #29 0x00007fcfb8bff81a in io_loop_handler_run_internal (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222 #30 0x00007fcfb8bff8d4 in io_loop_handler_run (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:789 #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:762 #32 0x00007fcfb8b6ce57 in master_service_run (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210 <client_connected>) at ../lib-master/master-service.c:878 #33 0x000055dabc51ad37 in main (argc=<optimized out>, argv=<optimized out>) at ./src/imap/main.c:575

John

Hello,

I've found a crash in a very specific setup. A dovecot server with imapc connection needs to receive an email with no body contents for the intent of generating a preview/snippet. It crashes somewhere deep in the jungle of istream and snapshots. I've included a script which sets up the systems to reproduce the crash.

I've tested this with several versions. 2.3.16 doesn't seem to be affected, but 2.3.20 and 2.3.21 are affect.

For me it produces a traceback like this, using the ubuntu version from here: https://packages.ubuntu.com/noble/dovecot-core

(gdb) bt #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140530132887360) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=140530132887360) at ./nptl/ pthread_kill.c:78 #2 __GI___pthread_kill (threadid=140530132887360, signo=signo@entry=6) at ./ nptl/pthread_kill.c:89 #3 0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/ raise.c:26 #4 0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79 #5 0x00007fcfb8b37fe5 in default_fatal_finish (status=0, type=LOG_TYPE_PANIC) at ../lib/failures.c:465 #6 fatal_handler_real (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:477 #7 0x00007fcfb8be50d7 in i_internal_fatal_handler (ctx=<optimized out>, format=<optimized out>, args=<optimized out>) at ../lib/failures.c:879 #8 0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530 #9 0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:663 #10 i_stream_header_filter_snapshot_free (_snapshot=0x55dabe297a60) at ../lib- mail/istream-header-filter.c:655 #11 0x00007fcfb8bf25ac in i_stream_snapshot_free (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253 #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0) at ../lib/ istream.c:66 #13 0x00007fcfb8d96baa in index_mail_write_body_snippet (mail=0x55dabe292058) at index/index-mail.c:1151 #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551 #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:1602 #16 index_mail_get_special (_mail=0x55dabe292058, field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:1730 #17 0x00007fcfb8d16ffe in mail_get_special (mail=mail@entry=0x55dabe292058, field=field@entry=MAIL_FETCH_BODY_SNIPPET, value_r=value_r@entry=0x7ffc16cc8050) at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-storage/mail.c:418 #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050, mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-fetch-body.c: 615 #19 0x000055dabc52b5cc in imap_fetch_more_int (ctx=0x55dabe26e050,

...

...

cancel=false) at ./src/imap/imap-fetch.c:562 #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050, cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617 #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./src/imap/cmd- fetch.c:382 #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./src/imap/imap- commands.c:201 #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized out>) at ./src/ imap/imap-client.c:1237 #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized out>) at ./src/ imap/imap-client.c:1307 #25 0x000055dabc52eeed in client_handle_next_command (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1349 #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/imap-client.c: 1363 #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at ./src/imap/ imap-client.c:1407 #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at ../lib/ ioloop.c:737 #29 0x00007fcfb8bff81a in io_loop_handler_run_internal (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222 #30 0x00007fcfb8bff8d4 in io_loop_handler_run (ioloop=0x55dabe243fd0) at ../ lib/ioloop.c:789 #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at ../lib/ ioloop.c:762 #32 0x00007fcfb8b6ce57 in master_service_run (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210 <client_connected>) at ../lib-master/ master-service.c:878 #33 0x000055dabc51ad37 in main (argc=<optimized out>, argv=<optimized out>) at ./src/imap/main.c:575

John

---

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

---

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

---

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

Amazon Development Center (Netherlands) B.V., Johanna Westerdijkplein 1, NL-2521 EN The Hague, Registration No. Chamber of Commerce 56869649, VAT: NL 852339859B01

Hi, sorry for the late reply. The commit you've pointed at before is the commit introducing code for the snippets. Your claim that main is fixed is incorrect: I've bisected through the git history, and the commit that "fixes" it, is the one flipping imapc features to negatives: https://github.com/dovecot/core/commit/ 7810b38d30b7dbb2155f78873fe760bc9e2e6212 However, the default imapc_features value stays the same, so all the "negative" features are suddenly enabled. I've reset the defaults in the frontend config to what it was before: imapc_features = no-fetch-size no-fetch-headers no-search no-modseq no-delay- login no-fetch-bodystructure no-acl and then dovecot starts crashing again in the described scenario. It is the "no-fetch-size" flag, and if I use "rfc822.size" feature on a 2.3 branch it stops crashing.

Turns out this same feature adds some filter that seems to be meant for some exchange email side-effect: https://github.com/dovecot/core/blob/main/src/lib- storage/index/imapc/imapc-mail-fetch.c#L596 where this filter tries to remove any X-Message-Flag header. This is weird, because it could have been an normally received header as well as something that was tacked on later by exchange.

The main bug is not fixed by just removing that filter: chaining filters is probably very broken when using the imapc backend, and it might be broken in other unknown scenarios.

Regards,

John

 -----Original message-----
 From: Aki Tuomi via dovecot <dovecot@dovecot.org>
 Sent: Friday, 19th January 2024, 8:37
 To: Aki Tuomi via dovecot <dovecot@dovecot.org>; John van der Kamp
 <jkamp@amazon.nl>
 Subject: RE: Crash in dovecot snippet when using imapc

 CAUTION: This email originated from outside of the organization. Do
 not click links or open attachments unless you can confirm the sender
 and know the content is safe.



 Sorry, the provided patch link was wrong, it's already in 2.3.21, my
 bad. Anyways, it is still fixed in main, since it does not happen

...

 there.

 Aki

 > On 19/01/2024 09:13 EET Aki Tuomi via dovecot <dovecot@dovecot.org>
 wrote:
 >
 >
 > Hi!
 >
 > I was able to reproduce this issue with 2.3.21, but it seems to

...

 have been fixed in main. I think https://github.com/dovecot/core/

...

 commit/1c1b77dbf9a548aac788efb76973ce2d0fa6c732.patch will fix this.
 >
 > Aki
 >
 > > On 18/01/2024 22:51 EET John van der Kamp via dovecot
 <dovecot@dovecot.org> wrote:
 > >
 > >
 > > Hello,
 > >
 > >
 > > I've found a crash in a very specific setup. A dovecot server

...

 with imapc connection needs to receive an email with no body contents
 for the intent of generating a preview/snippet. It crashes somewhere
 deep in the jungle of istream and snapshots. I've included a script
 which sets up the systems to reproduce the crash.
 > >
 > >
 > > I've tested this with several versions. 2.3.16 doesn't seem to be
 affected, but 2.3.20 and 2.3.21 are affect.
 > >
 > >
 > > For me it produces a traceback like this, using the ubuntu
 version from here: https://packages.ubuntu.com/noble/dovecot-core

...

 > >
 > >
 > > (gdb) bt
 > > #0  __pthread_kill_implementation (no_tid=0, signo=6,
 threadid=140530132887360) at ./nptl/pthread_kill.c:44
 > > #1  __pthread_kill_internal (signo=6, threadid=140530132887360)
 at ./nptl/pthread_kill.c:78
 > > #2  __GI___pthread_kill (threadid=140530132887360,
 signo=signo@entry=6) at ./nptl/pthread_kill.c:89
 > > #3  0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../
 sysdeps/posix/raise.c:26
 > > #4  0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79
 > > #5  0x00007fcfb8b37fe5 in default_fatal_finish (status=0,
 type=LOG_TYPE_PANIC) at ../lib/failures.c:465
 > > #6  fatal_handler_real (ctx=<optimized out>, format=<optimized
 out>, args=<optimized out>) at ../lib/failures.c:477
 > > #7  0x00007fcfb8be50d7 in i_internal_fatal_handler
 (ctx=<optimized out>, format=<optimized out>, args=<optimized out>)
 at ../lib/failures.c:879
 > > #8  0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file
 %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530
 > > #9  0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free
 (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:
 663
 > > #10 i_stream_header_filter_snapshot_free
 (_snapshot=0x55dabe297a60) at ../lib-mail/istream-header-filter.c:655
 > > #11 0x00007fcfb8bf25ac in i_stream_snapshot_free
 (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253
 > > #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0)
 at ../lib/istream.c:66
 > > #13 0x00007fcfb8d96baa in index_mail_write_body_snippet
 (mail=0x55dabe292058) at index/index-mail.c:1151
 > > #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure
 (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-
 mail.c:1551
 > > #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet
 (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:
 1602
 > > #16 index_mail_get_special (_mail=0x55dabe292058,
 field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:
 1730
 > > #17 0x00007fcfb8d16ffe in mail_get_special
 (mail=mail@entry=0x55dabe292058,
 field=field@entry=MAIL_FETCH_BODY_SNIPPET,
 value_r=value_r@entry=0x7ffc16cc8050)
 > >     at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-

...

 storage/mail.c:418
 > > #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050,
 mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-
 fetch-body.c:615
 > > #19 0x000055dabc52b5cc in imap_fetch_more_int
 (ctx=0x55dabe26e050, cancel=false) at ./src/imap/imap-fetch.c:562
 > > #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050,
 cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617
 > > #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./
 src/imap/cmd-fetch.c:382
 > > #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./
 src/imap/imap-commands.c:201
 > > #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized
 out>) at ./src/imap/imap-client.c:1237
 > > #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized
 out>) at ./src/imap/imap-client.c:1307
 > > #25 0x000055dabc52eeed in client_handle_next_command
 (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/
 imap/imap-client.c:1349
 > > #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/
 imap-client.c:1363
 > > #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at
 ./src/imap/imap-client.c:1407
 > > #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at
 ../lib/ioloop.c:737
 > > #29 0x00007fcfb8bff81a in io_loop_handler_run_internal
 (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222
 > > #30 0x00007fcfb8bff8d4 in io_loop_handler_run
 (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:789
 > > #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at
 ../lib/ioloop.c:762
 > > #32 0x00007fcfb8b6ce57 in master_service_run
 (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210
 <client_connected>) at ../lib-master/master-service.c:878
 > > #33 0x000055dabc51ad37 in main (argc=<optimized out>,
 argv=<optimized out>) at ./src/imap/main.c:575
 > >
 > >
 > > John
 > >
 > >
 > >
 > > Hello,
 > >
 > > I've found a crash in a very specific setup. A dovecot server

...

 with imapc
 > > connection needs to receive an email with no body contents for
 the intent of
 > > generating a preview/snippet. It crashes somewhere deep in the
 jungle of
 > > istream and snapshots. I've included a script which sets up the
 systems to
 > > reproduce the crash.
 > >
 > > I've tested this with several versions. 2.3.16 doesn't seem to be
 affected, but
 > > 2.3.20 and 2.3.21 are affect.
 > >
 > > For me it produces a traceback like this, using the ubuntu
 version from here:
 > > https://packages.ubuntu.com/noble/dovecot-core
 > >
 > > (gdb) bt
 > > #0  __pthread_kill_implementation (no_tid=0, signo=6,
 threadid=140530132887360)
 > > at ./nptl/pthread_kill.c:44
 > > #1  __pthread_kill_internal (signo=6, threadid=140530132887360)
 at ./nptl/
 > > pthread_kill.c:78
 > > #2  __GI___pthread_kill (threadid=140530132887360,
 signo=signo@entry=6) at ./
 > > nptl/pthread_kill.c:89
 > > #3  0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../
 sysdeps/posix/
 > > raise.c:26
 > > #4  0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79
 > > #5  0x00007fcfb8b37fe5 in default_fatal_finish (status=0,
 type=LOG_TYPE_PANIC)
 > > at ../lib/failures.c:465
 > > #6  fatal_handler_real (ctx=<optimized out>, format=<optimized
 out>,
 > > args=<optimized out>) at ../lib/failures.c:477
 > > #7  0x00007fcfb8be50d7 in i_internal_fatal_handler
 (ctx=<optimized out>,
 > > format=<optimized out>, args=<optimized out>) at ../lib/
 failures.c:879
 > > #8  0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file
 %s: line %d
 > > (%s): assertion failed: (%s)") at ../lib/failures.c:530
 > > #9  0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free
 > > (_snapshot=<optimized out>) at ../lib-mail/istream-header-
 filter.c:663
 > > #10 i_stream_header_filter_snapshot_free
 (_snapshot=0x55dabe297a60) at ../lib-
 > > mail/istream-header-filter.c:655
 > > #11 0x00007fcfb8bf25ac in i_stream_snapshot_free
 > > (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:
 253
 > > #12 0x00007fcfb8bf2654 in i_stream_unref (stream=0x7ffc16cc7fa0)
 at ../lib/
 > > istream.c:66
 > > #13 0x00007fcfb8d96baa in index_mail_write_body_snippet
 (mail=0x55dabe292058)
 > > at index/index-mail.c:1151
 > > #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure
 (mail=0x55dabe292058,
 > > field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551
 > > #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet
 > > (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-
 mail.c:1602
 > > #16 index_mail_get_special (_mail=0x55dabe292058,
 field=<optimized out>,
 > > value_r=0x7ffc16cc8050) at index/index-mail.c:1730
 > > #17 0x00007fcfb8d16ffe in mail_get_special
 (mail=mail@entry=0x55dabe292058,
 > > field=field@entry=MAIL_FETCH_BODY_SNIPPET,
 > > value_r=value_r@entry=0x7ffc16cc8050)
 > >     at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-

...

 storage/mail.c:418
 > > #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050,
 > > mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-
 fetch-body.c:
 > > 615
 > > #19 0x000055dabc52b5cc in imap_fetch_more_int
 (ctx=0x55dabe26e050,
 > > cancel=false) at ./src/imap/imap-fetch.c:562
 > > #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050,
 > > cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617
 > > #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./
 src/imap/cmd-
 > > fetch.c:382
 > > #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at ./
 src/imap/imap-
 > > commands.c:201
 > > #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized
 out>) at ./src/
 > > imap/imap-client.c:1237
 > > #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized
 out>) at ./src/
 > > imap/imap-client.c:1307
 > > #25 0x000055dabc52eeed in client_handle_next_command
 (remove_io_r=<synthetic
 > > pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:1349
 > > #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/
 imap-client.c:
 > > 1363
 > > #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8) at
 ./src/imap/
 > > imap-client.c:1407
 > > #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660) at
 ../lib/
 > > ioloop.c:737
 > > #29 0x00007fcfb8bff81a in io_loop_handler_run_internal
 (ioloop=0x55dabe243fd0)
 > > at ../lib/ioloop-epoll.c:222
 > > #30 0x00007fcfb8bff8d4 in io_loop_handler_run
 (ioloop=0x55dabe243fd0) at ../
 > > lib/ioloop.c:789
 > > #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0) at
 ../lib/
 > > ioloop.c:762
 > > #32 0x00007fcfb8b6ce57 in master_service_run
 (service=0x55dabe243e20,
 > > callback=callback@entry=0x55dabc533210 <client_connected>) at ../
 lib-master/
 > > master-service.c:878
 > > #33 0x000055dabc51ad37 in main (argc=<optimized out>,
 argv=<optimized out>) at
 > > ./src/imap/main.c:575
 > >
 > > John
 > >
 > > _______________________________________________
 > > dovecot mailing list -- dovecot@dovecot.org
 > > To unsubscribe send an email to dovecot-leave@dovecot.org
 > _______________________________________________
 > dovecot mailing list -- dovecot@dovecot.org
 > To unsubscribe send an email to dovecot-leave@dovecot.org
 _______________________________________________
 dovecot mailing list -- dovecot@dovecot.org
 To unsubscribe send an email to dovecot-leave@dovecot.org



 Amazon Development Center (Netherlands) B.V., Johanna Westerdijkplein
 1, NL-2521 EN The Hague, Registration No. Chamber of Commerce
 56869649, VAT: NL 852339859B01

---

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org

Amazon Development Center (Netherlands) B.V., Johanna Westerdijkplein 1, NL-2521 EN The Hague, Registration No. Chamber of Commerce 56869649, VAT: NL 852339859B01

See attached script I used. If you get EOF then you've hit the crash.   John     -----Original message----- From: Aki Tuomi aki.tuomi@open-xchange.com Sent: Wednesday, 20th March 2024, 8:40 To: John van der Kamp jkamp@amazon.nl; John van der Kamp via dovecot dovecot@dovecot.org Subject: RE: Crash in dovecot snippet when using imapc   CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 Could you provide some simple way to reproduce this, minimal config
 etc?

 Aki

 > On 19/03/2024 17:44 EET John van der Kamp via dovecot
 <dovecot@dovecot.org> wrote:
 >
 >
 > Hi, sorry for the late reply.
 >
 > The commit you've pointed at before is the commit introducing code
 for the snippets.
 >
 > Your claim that main is fixed is incorrect: I've bisected through
 the git history, and the commit that "fixes" it, is the one flipping
 imapc features to negatives: https://github.com/dovecot/core/commit/
 7810b38d30b7dbb2155f78873fe760bc9e2e6212 <https://github.com/dovecot/
 core/commit/7810b38d30b7dbb2155f78873fe760bc9e2e6212> However, the
 default imapc_features value stays the same, so all the "negative"
 features are suddenly enabled.
 >
 > I've reset the defaults in the frontend config to what it was
 before:
 >
 > imapc_features = no-fetch-size no-fetch-headers no-search no-modseq
 no-delay-login no-fetch-bodystructure no-acl
 >
 > and then dovecot starts crashing again in the described scenario.
 It is the "no-fetch-size" flag, and if I use "rfc822.size" feature on
 a 2.3 branch it stops crashing.
 >
 >
 > Turns out this same feature adds some filter that seems to be meant
 for some exchange email side-effect: https://github.com/dovecot/core/
 blob/main/src/lib-storage/index/imapc/imapc-mail-fetch.c#L596 <https:
 //github.com/dovecot/core/blob/main/src/lib-storage/index/imapc/
 imapc-mail-fetch.c#L596>   where this filter tries to remove any X-
 Message-Flag header. This is weird, because it could have been an
 normally received header as well as something that was tacked on
 later by exchange.
 >
 >
 > The main bug is not fixed by just removing that filter: chaining
 filters is probably very broken when using the imapc backend, and it
 might be broken in other unknown scenarios.
 >
 >
 > Regards,
 >
 >
 > John
 >
 >
 >
 > -----Original message-----
 > From: Aki Tuomi via dovecot <dovecot@dovecot.org>
 > Sent: Friday, 19th January 2024, 8:37
 > To: Aki Tuomi via dovecot <dovecot@dovecot.org>; John van der Kamp
 <jkamp@amazon.nl>
 > Subject: RE: Crash in dovecot snippet when using imapc
 >
 > CAUTION: This email originated from outside of the organization. Do
 not click links or open attachments unless you can confirm the sender
 and know the content is safe.
 >
 >
 >
 > Sorry, the provided patch link was wrong, it's already in 2.3.21,
 my bad. Anyways, it is still fixed in main, since it does not happen
 there.
 >
 > Aki
 >
 > > On 19/01/2024 09:13 EET Aki Tuomi via dovecot
 <dovecot@dovecot.org> wrote:
 > >
 > >
 > > Hi!
 > >
 > > I was able to reproduce this issue with 2.3.21, but it seems to
 have been fixed in main. I think https://github.com/dovecot/core/
 commit/1c1b77dbf9a548aac788efb76973ce2d0fa6c732.patch will fix this.
 > >
 > > Aki
 > >
 > > > On 18/01/2024 22:51 EET John van der Kamp via dovecot
 <dovecot@dovecot.org> wrote:
 > > >
 > > >
 > > > Hello,
 > > >
 > > >
 > > > I've found a crash in a very specific setup. A dovecot server
 with imapc connection needs to receive an email with no body contents
 for the intent of generating a preview/snippet. It crashes somewhere
 deep in the jungle of istream and snapshots. I've included a script
 which sets up the systems to reproduce the crash.
 > > >
 > > >
 > > > I've tested this with several versions. 2.3.16 doesn't seem to
 be affected, but 2.3.20 and 2.3.21 are affect.
 > > >
 > > >
 > > > For me it produces a traceback like this, using the ubuntu
 version from here: https://packages.ubuntu.com/noble/dovecot-core
 > > >
 > > >
 > > > (gdb) bt
 > > > #0  __pthread_kill_implementation (no_tid=0, signo=6,
 threadid=140530132887360) at ./nptl/pthread_kill.c:44
 > > > #1  __pthread_kill_internal (signo=6, threadid=140530132887360)
 at ./nptl/pthread_kill.c:78
 > > > #2  __GI___pthread_kill (threadid=140530132887360,
 signo=signo@entry=6) at ./nptl/pthread_kill.c:89
 > > > #3  0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../
 sysdeps/posix/raise.c:26
 > > > #4  0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79
 > > > #5  0x00007fcfb8b37fe5 in default_fatal_finish (status=0,
 type=LOG_TYPE_PANIC) at ../lib/failures.c:465
 > > > #6  fatal_handler_real (ctx=<optimized out>, format=<optimized
 out>, args=<optimized out>) at ../lib/failures.c:477
 > > > #7  0x00007fcfb8be50d7 in i_internal_fatal_handler
 (ctx=<optimized out>, format=<optimized out>, args=<optimized out>)
 at ../lib/failures.c:879
 > > > #8  0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file
 %s: line %d (%s): assertion failed: (%s)") at ../lib/failures.c:530
 > > > #9  0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free
 (_snapshot=<optimized out>) at ../lib-mail/istream-header-filter.c:
 663
 > > > #10 i_stream_header_filter_snapshot_free
 (_snapshot=0x55dabe297a60) at ../lib-mail/istream-header-filter.c:655
 > > > #11 0x00007fcfb8bf25ac in i_stream_snapshot_free
 (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:253
 > > > #12 0x00007fcfb8bf2654 in i_stream_unref
 (stream=0x7ffc16cc7fa0) at ../lib/istream.c:66
 > > > #13 0x00007fcfb8d96baa in index_mail_write_body_snippet
 (mail=0x55dabe292058) at index/index-mail.c:1151
 > > > #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure
 (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/index-
 mail.c:1551
 > > > #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet
 (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-mail.c:
 1602
 > > > #16 index_mail_get_special (_mail=0x55dabe292058,
 field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-mail.c:
 1730
 > > > #17 0x00007fcfb8d16ffe in mail_get_special
 (mail=mail@entry=0x55dabe292058,
 field=field@entry=MAIL_FETCH_BODY_SNIPPET,
 value_r=value_r@entry=0x7ffc16cc8050)
 > > >     at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-
 storage/mail.c:418
 > > > #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050,
 mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/imap-
 fetch-body.c:615
 > > > #19 0x000055dabc52b5cc in imap_fetch_more_int
 (ctx=0x55dabe26e050, cancel=false) at ./src/imap/imap-fetch.c:562
 > > > #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050,
 cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617
 > > > #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./
 src/imap/cmd-fetch.c:382
 > > > #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at
 ./src/imap/imap-commands.c:201
 > > > #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized
 out>) at ./src/imap/imap-client.c:1237
 > > > #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized
 out>) at ./src/imap/imap-client.c:1307
 > > > #25 0x000055dabc52eeed in client_handle_next_command
 (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./src/
 imap/imap-client.c:1349
 > > > #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/
 imap-client.c:1363
 > > > #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8)
 at ./src/imap/imap-client.c:1407
 > > > #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660)
 at ../lib/ioloop.c:737
 > > > #29 0x00007fcfb8bff81a in io_loop_handler_run_internal
 (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222
 > > > #30 0x00007fcfb8bff8d4 in io_loop_handler_run
 (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:789
 > > > #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0)
 at ../lib/ioloop.c:762
 > > > #32 0x00007fcfb8b6ce57 in master_service_run
 (service=0x55dabe243e20, callback=callback@entry=0x55dabc533210
 <client_connected>) at ../lib-master/master-service.c:878
 > > > #33 0x000055dabc51ad37 in main (argc=<optimized out>,
 argv=<optimized out>) at ./src/imap/main.c:575
 > > >
 > > >
 > > > John
 > > >
 > > >
 > > >
 > > > Hello,
 > > >
 > > > I've found a crash in a very specific setup. A dovecot server
 with imapc
 > > > connection needs to receive an email with no body contents for
 the intent of
 > > > generating a preview/snippet. It crashes somewhere deep in the
 jungle of
 > > > istream and snapshots. I've included a script which sets up the
 systems to
 > > > reproduce the crash.
 > > >
 > > > I've tested this with several versions. 2.3.16 doesn't seem to
 be affected, but
 > > > 2.3.20 and 2.3.21 are affect.
 > > >
 > > > For me it produces a traceback like this, using the ubuntu
 version from here:
 > > > https://packages.ubuntu.com/noble/dovecot-core
 > > >
 > > > (gdb) bt
 > > > #0  __pthread_kill_implementation (no_tid=0, signo=6,
 threadid=140530132887360)
 > > > at ./nptl/pthread_kill.c:44
 > > > #1  __pthread_kill_internal (signo=6, threadid=140530132887360)
 at ./nptl/
 > > > pthread_kill.c:78
 > > > #2  __GI___pthread_kill (threadid=140530132887360,
 signo=signo@entry=6) at ./
 > > > nptl/pthread_kill.c:89
 > > > #3  0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at ../
 sysdeps/posix/
 > > > raise.c:26
 > > > #4  0x00007fcfb88287f3 in __GI_abort () at ./stdlib/abort.c:79
 > > > #5  0x00007fcfb8b37fe5 in default_fatal_finish (status=0,
 type=LOG_TYPE_PANIC)
 > > > at ../lib/failures.c:465
 > > > #6  fatal_handler_real (ctx=<optimized out>, format=<optimized
 out>,
 > > > args=<optimized out>) at ../lib/failures.c:477
 > > > #7  0x00007fcfb8be50d7 in i_internal_fatal_handler
 (ctx=<optimized out>,
 > > > format=<optimized out>, args=<optimized out>) at ../lib/
 failures.c:879
 > > > #8  0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020 "file
 %s: line %d
 > > > (%s): assertion failed: (%s)") at ../lib/failures.c:530
 > > > #9  0x00007fcfb8b3387b in i_stream_header_filter_snapshot_free
 > > > (_snapshot=<optimized out>) at ../lib-mail/istream-header-
 filter.c:663
 > > > #10 i_stream_header_filter_snapshot_free
 (_snapshot=0x55dabe297a60) at ../lib-
 > > > mail/istream-header-filter.c:655
 > > > #11 0x00007fcfb8bf25ac in i_stream_snapshot_free
 > > > (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/istream.c:
 253
 > > > #12 0x00007fcfb8bf2654 in i_stream_unref
 (stream=0x7ffc16cc7fa0) at ../lib/
 > > > istream.c:66
 > > > #13 0x00007fcfb8d96baa in index_mail_write_body_snippet
 (mail=0x55dabe292058)
 > > > at index/index-mail.c:1151
 > > > #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure
 (mail=0x55dabe292058,
 > > > field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551
 > > > #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet
 > > > (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-
 mail.c:1602
 > > > #16 index_mail_get_special (_mail=0x55dabe292058,
 field=<optimized out>,
 > > > value_r=0x7ffc16cc8050) at index/index-mail.c:1730
 > > > #17 0x00007fcfb8d16ffe in mail_get_special
 (mail=mail@entry=0x55dabe292058,
 > > > field=field@entry=MAIL_FETCH_BODY_SNIPPET,
 > > > value_r=value_r@entry=0x7ffc16cc8050)
 > > >     at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/lib-
 storage/mail.c:418
 > > > #18 0x000055dabc52645c in fetch_snippet (ctx=0x55dabe26e050,
 > > > mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/
 imap-fetch-body.c:
 > > > 615
 > > > #19 0x000055dabc52b5cc in imap_fetch_more_int
 (ctx=0x55dabe26e050,
 >
 > > > cancel=false) at ./src/imap/imap-fetch.c:562
 > > > #20 0x000055dabc52b8ad in imap_fetch_more (ctx=0x55dabe26e050,
 > > > cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617
 > > > #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98) at ./
 src/imap/cmd-
 > > > fetch.c:382
 > > > #22 0x000055dabc528af4 in command_exec (cmd=0x55dabe26de98) at
 ./src/imap/imap-
 > > > commands.c:201
 > > > #23 0x000055dabc52e9e2 in client_command_input (cmd=<optimized
 out>) at ./src/
 > > > imap/imap-client.c:1237
 > > > #24 0x000055dabc52ea96 in client_command_input (cmd=<optimized
 out>) at ./src/
 > > > imap/imap-client.c:1307
 > > > #25 0x000055dabc52eeed in client_handle_next_command
 (remove_io_r=<synthetic
 > > > pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-client.c:
 1349
 > > > #26 client_handle_input (client=0x55dabe26d2c8) at ./src/imap/
 imap-client.c:
 > > > 1363
 > > > #27 0x000055dabc52f2c4 in client_input (client=0x55dabe26d2c8)
 at ./src/imap/
 > > > imap-client.c:1407
 > > > #28 0x00007fcfb8bfe27d in io_loop_call_io (io=0x55dabe26e660)
 at ../lib/
 > > > ioloop.c:737
 > > > #29 0x00007fcfb8bff81a in io_loop_handler_run_internal
 (ioloop=0x55dabe243fd0)
 > > > at ../lib/ioloop-epoll.c:222
 > > > #30 0x00007fcfb8bff8d4 in io_loop_handler_run
 (ioloop=0x55dabe243fd0) at ../
 > > > lib/ioloop.c:789
 > > > #31 0x00007fcfb8bffa90 in io_loop_run (ioloop=0x55dabe243fd0)
 at ../lib/
 > > > ioloop.c:762
 > > > #32 0x00007fcfb8b6ce57 in master_service_run
 (service=0x55dabe243e20,
 > > > callback=callback@entry=0x55dabc533210 <client_connected>) at
 ../lib-master/
 > > > master-service.c:878
 > > > #33 0x000055dabc51ad37 in main (argc=<optimized out>,
 argv=<optimized out>) at
 > > > ./src/imap/main.c:575
 > > >
 > > > John
 > > >
 > > > _______________________________________________
 > > > dovecot mailing list -- dovecot@dovecot.org
 > > > To unsubscribe send an email to dovecot-leave@dovecot.org
 > > _______________________________________________
 > > dovecot mailing list -- dovecot@dovecot.org
 > > To unsubscribe send an email to dovecot-leave@dovecot.org
 > _______________________________________________
 > dovecot mailing list -- dovecot@dovecot.org
 > To unsubscribe send an email to dovecot-leave@dovecot.org
 >
 >
 >
 > Amazon Development Center (Netherlands) B.V., Johanna
 Westerdijkplein 1, NL-2521 EN The Hague, Registration No. Chamber of
 Commerce 56869649, VAT: NL 852339859B01
 >
 >
 >
 > Hi, sorry for the late reply.
 > The commit you've pointed at before is the commit introducing code
 for the
 > snippets.
 > Your claim that main is fixed is incorrect: I've bisected through
 the git
 > history, and the commit that "fixes" it, is the one flipping imapc
 features to
 > negatives: https://github.com/dovecot/core/commit/
 > 7810b38d30b7dbb2155f78873fe760bc9e2e6212 However, the default
 imapc_features
 > value stays the same, so all the "negative" features are suddenly
 enabled.
 > I've reset the defaults in the frontend config to what it was
 before:
 > imapc_features = no-fetch-size no-fetch-headers no-search no-modseq
 no-delay-
 > login no-fetch-bodystructure no-acl
 > and then dovecot starts crashing again in the described scenario.
 It is the
 > "no-fetch-size" flag, and if I use "rfc822.size" feature on a 2.3
 branch it
 > stops crashing.
 >
 > Turns out this same feature adds some filter that seems to be meant
 for some
 > exchange email side-effect: https://github.com/dovecot/core/blob/
 main/src/lib-
 > storage/index/imapc/imapc-mail-fetch.c#L596  where this filter
 tries to remove
 > any X-Message-Flag header. This is weird, because it could have
 been an
 > normally received header as well as something that was tacked on
 later by
 > exchange.
 >
 > The main bug is not fixed by just removing that filter: chaining
 filters is
 > probably very broken when using the imapc backend, and it might be
 broken in
 > other unknown scenarios.
 >
 > Regards,
 >
 > John
 >
 >
 >      -----Original message-----
 >      From: Aki Tuomi via dovecot <dovecot@dovecot.org>
 >      Sent: Friday, 19th January 2024, 8:37
 >      To: Aki Tuomi via dovecot <dovecot@dovecot.org>; John van der
 Kamp
 >      <jkamp@amazon.nl>
 >      Subject: RE: Crash in dovecot snippet when using imapc
 >
 >      CAUTION: This email originated from outside of the
 organization. Do
 >      not click links or open attachments unless you can confirm the
 sender
 >      and know the content is safe.
 >
 >
 >
 >      Sorry, the provided patch link was wrong, it's already in
 2.3.21, my
 >      bad. Anyways, it is still fixed in main, since it does not
 happen
 >      there.
 >
 >      Aki
 >
 >      > On 19/01/2024 09:13 EET Aki Tuomi via dovecot
 <dovecot@dovecot.org>
 >      wrote:
 >      >
 >      >
 >      > Hi!
 >      >
 >      > I was able to reproduce this issue with 2.3.21, but it seems
 to
 >      have been fixed in main. I think https://github.com/dovecot/
 core/
 >      commit/1c1b77dbf9a548aac788efb76973ce2d0fa6c732.patch will fix
 this.
 >      >
 >      > Aki
 >      >
 >      > > On 18/01/2024 22:51 EET John van der Kamp via dovecot
 >      <dovecot@dovecot.org> wrote:
 >      > >
 >      > >
 >      > > Hello,
 >      > >
 >      > >
 >      > > I've found a crash in a very specific setup. A dovecot
 server
 >      with imapc connection needs to receive an email with no body
 contents
 >      for the intent of generating a preview/snippet. It crashes
 somewhere
 >      deep in the jungle of istream and snapshots. I've included a
 script
 >      which sets up the systems to reproduce the crash.
 >      > >
 >      > >
 >      > > I've tested this with several versions. 2.3.16 doesn't
 seem to be
 >      affected, but 2.3.20 and 2.3.21 are affect.
 >      > >
 >      > >
 >      > > For me it produces a traceback like this, using the ubuntu
 >      version from here: https://packages.ubuntu.com/noble/dovecot-
 core
 >      > >
 >      > >
 >      > > (gdb) bt
 >      > > #0  __pthread_kill_implementation (no_tid=0, signo=6,
 >      threadid=140530132887360) at ./nptl/pthread_kill.c:44
 >      > > #1  __pthread_kill_internal (signo=6,
 threadid=140530132887360)
 >      at ./nptl/pthread_kill.c:78
 >      > > #2  __GI___pthread_kill (threadid=140530132887360,
 >      signo=signo@entry=6) at ./nptl/pthread_kill.c:89
 >      > > #3  0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at
 ../
 >      sysdeps/posix/raise.c:26
 >      > > #4  0x00007fcfb88287f3 in __GI_abort () at ./stdlib/
 abort.c:79
 >      > > #5  0x00007fcfb8b37fe5 in default_fatal_finish (status=0,
 >      type=LOG_TYPE_PANIC) at ../lib/failures.c:465
 >      > > #6  fatal_handler_real (ctx=<optimized out>,
 format=<optimized
 >      out>, args=<optimized out>) at ../lib/failures.c:477
 >      > > #7  0x00007fcfb8be50d7 in i_internal_fatal_handler
 >      (ctx=<optimized out>, format=<optimized out>, args=<optimized
 out>)
 >      at ../lib/failures.c:879
 >      > > #8  0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020
 "file
 >      %s: line %d (%s): assertion failed: (%s)") at ../lib/
 failures.c:530
 >      > > #9  0x00007fcfb8b3387b in
 i_stream_header_filter_snapshot_free
 >      (_snapshot=<optimized out>) at ../lib-mail/istream-header-
 filter.c:
 >      663
 >      > > #10 i_stream_header_filter_snapshot_free
 >      (_snapshot=0x55dabe297a60) at ../lib-mail/istream-header-
 filter.c:655
 >      > > #11 0x00007fcfb8bf25ac in i_stream_snapshot_free
 >      (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/
 istream.c:253
 >      > > #12 0x00007fcfb8bf2654 in i_stream_unref
 (stream=0x7ffc16cc7fa0)
 >      at ../lib/istream.c:66
 >      > > #13 0x00007fcfb8d96baa in index_mail_write_body_snippet
 >      (mail=0x55dabe292058) at index/index-mail.c:1151
 >      > > #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure
 >      (mail=0x55dabe292058, field=MAIL_CACHE_BODY_SNIPPET) at index/
 index-
 >      mail.c:1551
 >      > > #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet
 >      (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/index-
 mail.c:
 >      1602
 >      > > #16 index_mail_get_special (_mail=0x55dabe292058,
 >      field=<optimized out>, value_r=0x7ffc16cc8050) at index/index-
 mail.c:
 >      1730
 >      > > #17 0x00007fcfb8d16ffe in mail_get_special
 >      (mail=mail@entry=0x55dabe292058,
 >      field=field@entry=MAIL_FETCH_BODY_SNIPPET,
 >      value_r=value_r@entry=0x7ffc16cc8050)
 >      > >     at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/
 lib-
 >      storage/mail.c:418
 >      > > #18 0x000055dabc52645c in fetch_snippet
 (ctx=0x55dabe26e050,
 >      mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/imap/
 imap-
 >      fetch-body.c:615
 >      > > #19 0x000055dabc52b5cc in imap_fetch_more_int
 >      (ctx=0x55dabe26e050, cancel=false) at ./src/imap/imap-fetch.c:
 562
 >      > > #20 0x000055dabc52b8ad in imap_fetch_more
 (ctx=0x55dabe26e050,
 >      cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617
 >      > > #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98)
 at ./
 >      src/imap/cmd-fetch.c:382
 >      > > #22 0x000055dabc528af4 in command_exec
 (cmd=0x55dabe26de98) at ./
 >      src/imap/imap-commands.c:201
 >      > > #23 0x000055dabc52e9e2 in client_command_input
 (cmd=<optimized
 >      out>) at ./src/imap/imap-client.c:1237
 >      > > #24 0x000055dabc52ea96 in client_command_input
 (cmd=<optimized
 >      out>) at ./src/imap/imap-client.c:1307
 >      > > #25 0x000055dabc52eeed in client_handle_next_command
 >      (remove_io_r=<synthetic pointer>, client=0x55dabe26d2c8) at ./
 src/
 >      imap/imap-client.c:1349
 >      > > #26 client_handle_input (client=0x55dabe26d2c8) at ./src/
 imap/
 >      imap-client.c:1363
 >      > > #27 0x000055dabc52f2c4 in client_input
 (client=0x55dabe26d2c8) at
 >      ./src/imap/imap-client.c:1407
 >      > > #28 0x00007fcfb8bfe27d in io_loop_call_io
 (io=0x55dabe26e660) at
 >      ../lib/ioloop.c:737
 >      > > #29 0x00007fcfb8bff81a in io_loop_handler_run_internal
 >      (ioloop=0x55dabe243fd0) at ../lib/ioloop-epoll.c:222
 >      > > #30 0x00007fcfb8bff8d4 in io_loop_handler_run
 >      (ioloop=0x55dabe243fd0) at ../lib/ioloop.c:789
 >      > > #31 0x00007fcfb8bffa90 in io_loop_run
 (ioloop=0x55dabe243fd0) at
 >      ../lib/ioloop.c:762
 >      > > #32 0x00007fcfb8b6ce57 in master_service_run
 >      (service=0x55dabe243e20,
 callback=callback@entry=0x55dabc533210
 >      <client_connected>) at ../lib-master/master-service.c:878
 >      > > #33 0x000055dabc51ad37 in main (argc=<optimized out>,
 >      argv=<optimized out>) at ./src/imap/main.c:575
 >      > >
 >      > >
 >      > > John
 >      > >
 >      > >
 >      > >
 >      > > Hello,
 >      > >
 >      > > I've found a crash in a very specific setup. A dovecot
 server
 >      with imapc
 >      > > connection needs to receive an email with no body contents
 for
 >      the intent of
 >      > > generating a preview/snippet. It crashes somewhere deep in
 the
 >      jungle of
 >      > > istream and snapshots. I've included a script which sets
 up the
 >      systems to
 >      > > reproduce the crash.
 >      > >
 >      > > I've tested this with several versions. 2.3.16 doesn't
 seem to be
 >      affected, but
 >      > > 2.3.20 and 2.3.21 are affect.
 >      > >
 >      > > For me it produces a traceback like this, using the ubuntu
 >      version from here:
 >      > > https://packages.ubuntu.com/noble/dovecot-core
 >      > >
 >      > > (gdb) bt
 >      > > #0  __pthread_kill_implementation (no_tid=0, signo=6,
 >      threadid=140530132887360)
 >      > > at ./nptl/pthread_kill.c:44
 >      > > #1  __pthread_kill_internal (signo=6,
 threadid=140530132887360)
 >      at ./nptl/
 >      > > pthread_kill.c:78
 >      > > #2  __GI___pthread_kill (threadid=140530132887360,
 >      signo=signo@entry=6) at ./
 >      > > nptl/pthread_kill.c:89
 >      > > #3  0x00007fcfb8842476 in __GI_raise (sig=sig@entry=6) at
 ../
 >      sysdeps/posix/
 >      > > raise.c:26
 >      > > #4  0x00007fcfb88287f3 in __GI_abort () at ./stdlib/
 abort.c:79
 >      > > #5  0x00007fcfb8b37fe5 in default_fatal_finish (status=0,
 >      type=LOG_TYPE_PANIC)
 >      > > at ../lib/failures.c:465
 >      > > #6  fatal_handler_real (ctx=<optimized out>,
 format=<optimized
 >      out>,
 >      > > args=<optimized out>) at ../lib/failures.c:477
 >      > > #7  0x00007fcfb8be50d7 in i_internal_fatal_handler
 >      (ctx=<optimized out>,
 >      > > format=<optimized out>, args=<optimized out>) at ../lib/
 >      failures.c:879
 >      > > #8  0x00007fcfb8b37eea in i_panic (format=0x7fcfb8c29020
 "file
 >      %s: line %d
 >      > > (%s): assertion failed: (%s)") at ../lib/failures.c:530
 >      > > #9  0x00007fcfb8b3387b in
 i_stream_header_filter_snapshot_free
 >      > > (_snapshot=<optimized out>) at ../lib-mail/istream-header-
 >      filter.c:663
 >      > > #10 i_stream_header_filter_snapshot_free
 >      (_snapshot=0x55dabe297a60) at ../lib-
 >      > > mail/istream-header-filter.c:655
 >      > > #11 0x00007fcfb8bf25ac in i_stream_snapshot_free
 >      > > (_snapshot=_snapshot@entry=0x55dabe29b0d0) at ../lib/
 istream.c:
 >      253
 >      > > #12 0x00007fcfb8bf2654 in i_stream_unref
 (stream=0x7ffc16cc7fa0)
 >      at ../lib/
 >      > > istream.c:66
 >      > > #13 0x00007fcfb8d96baa in index_mail_write_body_snippet
 >      (mail=0x55dabe292058)
 >      > > at index/index-mail.c:1151
 >      > > #14 0x00007fcfb8d97e48 in index_mail_parse_bodystructure
 >      (mail=0x55dabe292058,
 >      > > field=MAIL_CACHE_BODY_SNIPPET) at index/index-mail.c:1551
 >      > > #15 0x00007fcfb8d97fe2 in index_mail_fetch_body_snippet
 >      > > (value_r=0x7ffc16cc8050, mail=0x55dabe292058) at index/
 index-
 >      mail.c:1602
 >      > > #16 index_mail_get_special (_mail=0x55dabe292058,
 >      field=<optimized out>,
 >      > > value_r=0x7ffc16cc8050) at index/index-mail.c:1730
 >      > > #17 0x00007fcfb8d16ffe in mail_get_special
 >      (mail=mail@entry=0x55dabe292058,
 >      > > field=field@entry=MAIL_FETCH_BODY_SNIPPET,
 >      > > value_r=value_r@entry=0x7ffc16cc8050)
 >      > >     at /home/ubuntu/dovecot/new/dovecot-2.3.21+dfsg1/src/
 lib-
 >      storage/mail.c:418
 >      > > #18 0x000055dabc52645c in fetch_snippet
 (ctx=0x55dabe26e050,
 >      > > mail=0x55dabe292058, preview=0x55dabe28f1f8) at ./src/
 imap/imap-
 >      fetch-body.c:
 >      > > 615
 >      > > #19 0x000055dabc52b5cc in imap_fetch_more_int
 >      (ctx=0x55dabe26e050,
 >      > > cancel=false) at ./src/imap/imap-fetch.c:562
 >      > > #20 0x000055dabc52b8ad in imap_fetch_more
 (ctx=0x55dabe26e050,
 >      > > cmd=0x55dabe26de98) at ./src/imap/imap-fetch.c:617
 >      > > #21 0x000055dabc51fd07 in cmd_fetch (cmd=0x55dabe26de98)
 at ./
 >      src/imap/cmd-
 >      > > fetch.c:382
 >      > > #22 0x000055dabc528af4 in command_exec
 (cmd=0x55dabe26de98) at ./
 >      src/imap/imap-
 >      > > commands.c:201
 >      > > #23 0x000055dabc52e9e2 in client_command_input
 (cmd=<optimized
 >      out>) at ./src/
 >      > > imap/imap-client.c:1237
 >      > > #24 0x000055dabc52ea96 in client_command_input
 (cmd=<optimized
 >      out>) at ./src/
 >      > > imap/imap-client.c:1307
 >      > > #25 0x000055dabc52eeed in client_handle_next_command
 >      (remove_io_r=<synthetic
 >      > > pointer>, client=0x55dabe26d2c8) at ./src/imap/imap-
 client.c:1349
 >      > > #26 client_handle_input (client=0x55dabe26d2c8) at ./src/
 imap/
 >      imap-client.c:
 >      > > 1363
 >      > > #27 0x000055dabc52f2c4 in client_input
 (client=0x55dabe26d2c8) at
 >      ./src/imap/
 >      > > imap-client.c:1407
 >      > > #28 0x00007fcfb8bfe27d in io_loop_call_io
 (io=0x55dabe26e660) at
 >      ../lib/
 >      > > ioloop.c:737
 >      > > #29 0x00007fcfb8bff81a in io_loop_handler_run_internal
 >      (ioloop=0x55dabe243fd0)
 >      > > at ../lib/ioloop-epoll.c:222
 >      > > #30 0x00007fcfb8bff8d4 in io_loop_handler_run
 >      (ioloop=0x55dabe243fd0) at ../
 >      > > lib/ioloop.c:789
 >      > > #31 0x00007fcfb8bffa90 in io_loop_run
 (ioloop=0x55dabe243fd0) at
 >      ../lib/
 >      > > ioloop.c:762
 >      > > #32 0x00007fcfb8b6ce57 in master_service_run
 >      (service=0x55dabe243e20,
 >      > > callback=callback@entry=0x55dabc533210 <client_connected>)
 at ../
 >      lib-master/
 >      > > master-service.c:878
 >      > > #33 0x000055dabc51ad37 in main (argc=<optimized out>,
 >      argv=<optimized out>) at
 >      > > ./src/imap/main.c:575
 >      > >
 >      > > John
 >      > >
 >      > > _______________________________________________
 >      > > dovecot mailing list -- dovecot@dovecot.org
 >      > > To unsubscribe send an email to dovecot-leave@dovecot.org
 >      > _______________________________________________
 >      > dovecot mailing list -- dovecot@dovecot.org
 >      > To unsubscribe send an email to dovecot-leave@dovecot.org
 >      _______________________________________________
 >      dovecot mailing list -- dovecot@dovecot.org
 >      To unsubscribe send an email to dovecot-leave@dovecot.org
 >
 >
 >
 >      Amazon Development Center (Netherlands) B.V., Johanna
 Westerdijkplein
 >      1, NL-2521 EN The Hague, Registration No. Chamber of Commerce
 >      56869649, VAT: NL 852339859B01
 > _______________________________________________
 > dovecot mailing list -- dovecot@dovecot.org
 > To unsubscribe send an email to dovecot-leave@dovecot.org



 Amazon Development Center (Netherlands) B.V., Johanna Westerdijkplein
 1, NL-2521 EN The Hague, Registration No. Chamber of Commerce
 56869649, VAT: NL 852339859B01

---

dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org