Hello Dovecot maintainers,
Following up on the master crash during configuration reload, we traced the defect to commit 7e35059dc07013fd8e0612333c87fe789b1314b9, dated May 12, 2025. This commit introduced the block-name cache involved in the crash. GitHub confirms it is an ancestor of the stable 2.4.2 release.
Introducing commit
https://github.com/dovecot/core/commit/7e35059dc07013fd8e0612333c87fe789b131...
Comparison against the 2.4.2 release commit
https://github.com/dovecot/core/compare/7e35059dc07013fd8e0612333c87fe789b13...
The optimization in settings_override_set_blocks() stores pointers into the binary configuration mapping in set->block_names. Commit 713538a2ca840923f40e11556739b2e457ff1611 later moved the pointer array from mmap->pool to set->pool to fix a memory leak. The array then belongs to the override's pool, while its entries still point into the configuration mapping. Both commits are included in 2.4.2.
Allocation change
https://github.com/dovecot/core/commit/713538a2ca840923f40e11556739b2e457ff1...
On reload, settings_read() replaces the root configuration mapping. settings_mmap_unref() unmaps the old mapping when its reference count reaches zero. The root overrides survive the replacement. When a previously parsed override is reused, setting_override_match_info() can pass one of the old block-name pointers to strcmp().
Configuration mapping replacement
https://github.com/dovecot/core/blob/0cbb641e3e88d1616c2e8c1ff109c67c0985ddc...
Cached block-name comparison
https://github.com/dovecot/core/blob/0cbb641e3e88d1616c2e8c1ff109c67c0985ddc...
Runtime confirmation
We reproduced the crash twice with the vendor package 2:2.4.5-2+debian13 on Debian 13 amd64, using native systemd reloads. The first reload succeeded and the same master served the expected TLS certificate over IPv4 and IPv6. A subsequent reload of the unchanged configuration crashed that master.
With GDB 16.3 and matching debug symbols, we confirmed that set->block_names[0] for the */verbose_proctitle override still pointed into the retired mapping. It matched both the first argument to strcmp() and the kernel-reported fault address. The address was absent from the process's current mappings.
Backtrace excerpt
__strcmp_avx2 strcmp-avx2.S:283
setting_override_match_info settings.c:1911
settings_override_filter_match settings.c:1931
settings_apply_add_override settings.c:2209
settings_instance_override_init settings.c:2270
...
master_service_settings_read_int master-service-settings.c:642
sig_settings_reload master/main.c:421
Release scope
The cache is absent from upstream 2.4.1 and from the patch series of Debian's 1:2.4.1+dfsg1-6+deb13u7. The same mapping replacement and cache reuse path is present in the 2.4.2, 2.4.3, 2.4.4 and 2.4.5 source tags. Our evidence for 2.4.2 through 2.4.4 is from source inspection. The crash is confirmed at runtime on 2.4.5.
2.4.2 source
https://github.com/dovecot/core/blob/0962ed2104800eae2ea00899f106638f8654c8e...
2.4.3 source
https://github.com/dovecot/core/blob/c1b22ef978ca72999e254b2f3964b3d32d1a497...
2.4.4 source
https://github.com/dovecot/core/blob/8b687aa65c719e2a0d773eccea84f67c5770fc9...
Could you review the lifetime of the mapping-dependent root override cache when settings_read() replaces the mapping? A regression check would verify that the override remains effective across successive reloads and that its cached names refer to the current configuration. We can validate a proposed fix on the same test host.
Thank you.
Hello Dovecot maintainers,
Following up on the master crash during configuration reload, we traced
the defect to commit 7e35059dc07013fd8e0612333c87fe789b1314b9, dated May 12, 2025. This commit introduced the block-name cache involved in the crash. GitHub confirms it is an ancestor of the stable 2.4.2 release.
Introducing commit
[1]https://github.com/dovecot/core/commit/7e35059dc07013fd8e0612333c87fe789b131...
Comparison against the 2.4.2 release commit
[2]https://github.com/dovecot/core/compare/7e35059dc07013fd8e0612333c87fe789b13...
The optimization in settings_override_set_blocks() stores pointers into
the binary configuration mapping in set->block_names. Commit 713538a2ca840923f40e11556739b2e457ff1611 later moved the pointer array from mmap->pool to set->pool to fix a memory leak. The array then belongs to the override's pool, while its entries still point into the configuration mapping. Both commits are included in 2.4.2.
Allocation change
[3]https://github.com/dovecot/core/commit/713538a2ca840923f40e11556739b2e457ff1...
On reload, settings_read() replaces the root configuration mapping.
settings_mmap_unref() unmaps the old mapping when its reference count reaches zero. The root overrides survive the replacement. When a previously parsed override is reused, setting_override_match_info() can pass one of the old block-name pointers to strcmp().
Configuration mapping replacement
[4]https://github.com/dovecot/core/blob/0cbb641e3e88d1616c2e8c1ff109c67c0985ddc...
Cached block-name comparison
[5]https://github.com/dovecot/core/blob/0cbb641e3e88d1616c2e8c1ff109c67c0985ddc...
Runtime confirmation
We reproduced the crash twice with the vendor package 2:2.4.5-2+debian13
on Debian 13 amd64, using native systemd reloads. The first reload succeeded and the same master served the expected TLS certificate over IPv4 and IPv6. A subsequent reload of the unchanged configuration crashed that master.
With GDB 16.3 and matching debug symbols, we confirmed that
set->block_names[0] for the */verbose_proctitle override still pointed into the retired mapping. It matched both the first argument to strcmp() and the kernel-reported fault address. The address was absent from the process's current mappings.
Backtrace excerpt
__strcmp_avx2 strcmp-avx2.S:283
setting_override_match_info settings.c:1911
settings_override_filter_match settings.c:1931
settings_apply_add_override settings.c:2209
settings_instance_override_init settings.c:2270
...
master_service_settings_read_int master-service-settings.c:642
sig_settings_reload master/main.c:421
Release scope
The cache is absent from upstream 2.4.1 and from the patch series of
Debian's 1:2.4.1+dfsg1-6+deb13u7. The same mapping replacement and cache reuse path is present in the 2.4.2, 2.4.3, 2.4.4 and 2.4.5 source tags. Our evidence for 2.4.2 through 2.4.4 is from source inspection. The crash is confirmed at runtime on 2.4.5.
2.4.2 source
[6]https://github.com/dovecot/core/blob/0962ed2104800eae2ea00899f106638f8654c8e...
2.4.3 source
[7]https://github.com/dovecot/core/blob/c1b22ef978ca72999e254b2f3964b3d32d1a497...
2.4.4 source
[8]https://github.com/dovecot/core/blob/8b687aa65c719e2a0d773eccea84f67c5770fc9...
Could you review the lifetime of the mapping-dependent root override
cache when settings_read() replaces the mapping? A regression check would verify that the override remains effective across successive reloads and that its cached names refer to the current configuration. We can validate a proposed fix on the same test host.
Thank you.
References
Visible links
- https://github.com/dovecot/core/commit/7e35059dc07013fd8e0612333c87fe789b131...
- https://github.com/dovecot/core/compare/7e35059dc07013fd8e0612333c87fe789b13...
- https://github.com/dovecot/core/commit/713538a2ca840923f40e11556739b2e457ff1...
- https://github.com/dovecot/core/blob/0cbb641e3e88d1616c2e8c1ff109c67c0985ddc...
- https://github.com/dovecot/core/blob/0cbb641e3e88d1616c2e8c1ff109c67c0985ddc...
- https://github.com/dovecot/core/blob/0962ed2104800eae2ea00899f106638f8654c8e...
- https://github.com/dovecot/core/blob/c1b22ef978ca72999e254b2f3964b3d32d1a497...
- https://github.com/dovecot/core/blob/8b687aa65c719e2a0d773eccea84f67c5770fc9...