json_parse_number broken by compiler optimization
Hi, the recent Ubuntu (re)builds uncovered an issue with dovecot 1:2.3.13+dfsg1-1 build log: https://launchpadlibrarian.net/529849650/buildlog_ubuntu-hirsute-amd64.dovec... A coworker tried 2.3.14 but got the same result. What fails is the json_parser build time test like: test-json-parser.c:161: Assert(#25) failed: null_strcmp(json_output[pos].value, value) == 0 I was looking into that a bit more and what I found is that it is dependent on the new toolchain of gcc 10.2.0-1. Not all calls to json_parse_* fail, e.g. the first one looks all good and passes I was iterating the tests using a report function defined like (gdb) define repcon
c p pos p json_output[pos].type p type p json_output[pos].value p value call null_strcmp(json_output[pos].value, value) end
The first one to be bad was: Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $84 = 25 $85 = JSON_TYPE_NUMBER $86 = JSON_TYPE_NUMBER $87 = 0x555555633b25 "-12.456" $88 = 0x555555693110 "" $89 = 45 Earlier and later parsing was happy, for example Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $90 = 27 $91 = JSON_TYPE_NUMBER $92 = JSON_TYPE_NUMBER $93 = 0x555555633b32 "12.456e9" $94 = 0x555555693110 "12.456e9" $95 = 0 (gdb) We have two things we compare here. 1. json_output[] which is a static define and for this value is 67 »···{ JSON_TYPE_NUMBER, "-12.456" }, 2. the return value that json_parse_next returns. 25 »···" \"sub2\":-12.456,\n" 148 »···»···»···»···ret = json_parse_next(parser, &type, &value); I tried a non negative number and got success which is suspicious Breakpoint 5, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:164 164 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $122 = 25 $123 = JSON_TYPE_NUMBER $124 = JSON_TYPE_NUMBER $125 = 0x555555633b2c "12.456" $126 = 0x555555693110 "12.456" $127 = 0 (gdb) Also the return value otherwise LGTM, it is recognized as a number: 540 } else if ((ret = json_parse_number(parser, value_r)) >= 0) { (gdb) n 541 *type_r = JSON_TYPE_NUMBER; So parsing worked one might think, but the value was not set. (gdb) p *value_r $833 = 0x555555693110 "" It turns out json_parse_number is too optimized. Too optimized for debugging further into json_parse_number. But also optimization is causing the issue. With -O0 or -O1 it works fine, even putting the lower optimization level just around json_parse_number it is enough to avoid the issue. Not a fix but "avoidance" for now: --- a/src/lib/json-parser.c +++ b/src/lib/json-parser.c @@ -386,6 +386,8 @@ static int json_parse_int(struct json_pa return 1; } +#pragma GCC push_options +#pragma GCC optimize ("-O0") static int json_parse_number(struct json_parser *parser, const char **value_r) { int ret; @@ -415,6 +417,7 @@ static int json_parse_number(struct json *value_r = str_c(parser->value); return 1; } +#pragma GCC pop_options static int json_parse_atom(struct json_parser *parser, const char *atom) { I was trying to see what happened and tracked each str_append_c that happened inside json_parse_number. A breakpoint on str_append_c and backtraces showed that we clearly do the "right" parsing. From the backtraces I've seen it does (just from positions): For example (gdb) bt #0 str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 #1 json_parse_digits (parser=<optimized out>) at json-parser.c:366 => that is a digit It does "- digit digit dot digit digit digit" This perfectly matches the input value, but still the result stays empty: (gdb) p (char*)(str->data) $738 = 0x555555693110 "" In comparison - when parsing the latter 12.456e9 which works I see two odd things: 1. the value isn't cleared despite str_truncate(parser->value, 0); at the beginning of json_parse_number 2. the value (this time) is read exactly as I assumed Breakpoint 17, json_parse_number (value_r=0x7fffffffe1a0, parser=0x555555692ee0) at json-parser.c:394 394 str_truncate(parser->value, 0); Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $741 = 0x555555693110 "sub3" ^^ not cleared, still has "sub3" even later on Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $742 = 0x555555693110 "1ub3" ^^ reading left to right one char at a time "over the old value", which didn't update the data. So it seems in json_parse_number the ordering of a) str_truncate(parser->value, 0); b) the parsing itself is messed up by optimization. We already know from the above that reducing just this function to -O1 or less avoids the issue. But what is it really - I don't know? Maybe you have a better idea what is going on, the issue should be reproducible when building dovecot with the toolchain present on the latest Ubuntu being 21.04 (Hirsute) right now. -- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd
On 30/03/2021 18:53 Christian Ehrhardt
wrote: Hi, the recent Ubuntu (re)builds uncovered an issue with dovecot 1:2.3.13+dfsg1-1 build log: https://launchpadlibrarian.net/529849650/buildlog_ubuntu-hirsute-amd64.dovec... A coworker tried 2.3.14 but got the same result.
What fails is the json_parser build time test like: test-json-parser.c:161: Assert(#25) failed: null_strcmp(json_output[pos].value, value) == 0
I was looking into that a bit more and what I found is that it is dependent on the new toolchain of gcc 10.2.0-1.
Not all calls to json_parse_* fail, e.g. the first one looks all good and passes I was iterating the tests using a report function defined like
(gdb) define repcon
c p pos p json_output[pos].type p type p json_output[pos].value p value call null_strcmp(json_output[pos].value, value) end
The first one to be bad was: Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $84 = 25 $85 = JSON_TYPE_NUMBER $86 = JSON_TYPE_NUMBER $87 = 0x555555633b25 "-12.456" $88 = 0x555555693110 "" $89 = 45
Earlier and later parsing was happy, for example
Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $90 = 27 $91 = JSON_TYPE_NUMBER $92 = JSON_TYPE_NUMBER $93 = 0x555555633b32 "12.456e9" $94 = 0x555555693110 "12.456e9" $95 = 0 (gdb)
We have two things we compare here. 1. json_output[] which is a static define and for this value is 67 »···{ JSON_TYPE_NUMBER, "-12.456" }, 2. the return value that json_parse_next returns. 25 »···" \"sub2\":-12.456,\n" 148 »···»···»···»···ret = json_parse_next(parser, &type, &value);
I tried a non negative number and got success which is suspicious
Breakpoint 5, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:164 164 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $122 = 25 $123 = JSON_TYPE_NUMBER $124 = JSON_TYPE_NUMBER $125 = 0x555555633b2c "12.456" $126 = 0x555555693110 "12.456" $127 = 0 (gdb)
Also the return value otherwise LGTM, it is recognized as a number: 540 } else if ((ret = json_parse_number(parser, value_r)) >= 0) { (gdb) n 541 *type_r = JSON_TYPE_NUMBER;
So parsing worked one might think, but the value was not set.
(gdb) p *value_r $833 = 0x555555693110 ""
It turns out json_parse_number is too optimized. Too optimized for debugging further into json_parse_number. But also optimization is causing the issue. With -O0 or -O1 it works fine, even putting the lower optimization level just around json_parse_number it is enough to avoid the issue.
Not a fix but "avoidance" for now:
--- a/src/lib/json-parser.c +++ b/src/lib/json-parser.c @@ -386,6 +386,8 @@ static int json_parse_int(struct json_pa return 1; }
+#pragma GCC push_options +#pragma GCC optimize ("-O0") static int json_parse_number(struct json_parser *parser, const char **value_r) { int ret; @@ -415,6 +417,7 @@ static int json_parse_number(struct json *value_r = str_c(parser->value); return 1; } +#pragma GCC pop_options
static int json_parse_atom(struct json_parser *parser, const char *atom) {
I was trying to see what happened and tracked each str_append_c that happened inside json_parse_number. A breakpoint on str_append_c and backtraces showed that we clearly do the "right" parsing. From the backtraces I've seen it does (just from positions):
For example (gdb) bt #0 str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 #1 json_parse_digits (parser=<optimized out>) at json-parser.c:366 => that is a digit
It does "- digit digit dot digit digit digit"
This perfectly matches the input value, but still the result stays empty: (gdb) p (char*)(str->data) $738 = 0x555555693110 ""
In comparison - when parsing the latter 12.456e9 which works I see two odd things: 1. the value isn't cleared despite str_truncate(parser->value, 0); at the beginning of json_parse_number 2. the value (this time) is read exactly as I assumed
Breakpoint 17, json_parse_number (value_r=0x7fffffffe1a0, parser=0x555555692ee0) at json-parser.c:394 394 str_truncate(parser->value, 0); Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $741 = 0x555555693110 "sub3"
^^ not cleared, still has "sub3" even later on
Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $742 = 0x555555693110 "1ub3"
^^ reading left to right one char at a time "over the old value", which didn't update the data.
So it seems in json_parse_number the ordering of a) str_truncate(parser->value, 0); b) the parsing itself is messed up by optimization.
We already know from the above that reducing just this function to -O1 or less avoids the issue. But what is it really - I don't know?
Maybe you have a better idea what is going on, the issue should be reproducible when building dovecot with the toolchain present on the latest Ubuntu being 21.04 (Hirsute) right now.
-- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd
Thanks! We'll look into this. Aki
On Tue, Mar 30, 2021 at 17:53:27 +0200, Christian Ehrhardt wrote:
Hi, the recent Ubuntu (re)builds uncovered an issue with dovecot 1:2.3.13+dfsg1-1 build log: https://launchpadlibrarian.net/529849650/buildlog_ubuntu-hirsute-amd64.dovec... A coworker tried 2.3.14 but got the same result.
What fails is the json_parser build time test like: test-json-parser.c:161: Assert(#25) failed: null_strcmp(json_output[pos].value, value) == 0
I was looking into that a bit more and what I found is that it is dependent on the new toolchain of gcc 10.2.0-1.
FWIW, I managed to reproduce it on FreeBSD with gcc 11, so the good news for you is that it isn't Ubuntu specific :) I'll debug further. Thanks again for the report, Jeff.
Not all calls to json_parse_* fail, e.g. the first one looks all good and passes I was iterating the tests using a report function defined like
(gdb) define repcon
c p pos p json_output[pos].type p type p json_output[pos].value p value call null_strcmp(json_output[pos].value, value) end
The first one to be bad was: Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $84 = 25 $85 = JSON_TYPE_NUMBER $86 = JSON_TYPE_NUMBER $87 = 0x555555633b25 "-12.456" $88 = 0x555555693110 "" $89 = 45
Earlier and later parsing was happy, for example
Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $90 = 27 $91 = JSON_TYPE_NUMBER $92 = JSON_TYPE_NUMBER $93 = 0x555555633b32 "12.456e9" $94 = 0x555555693110 "12.456e9" $95 = 0 (gdb)
We have two things we compare here. 1. json_output[] which is a static define and for this value is 67 »···{ JSON_TYPE_NUMBER, "-12.456" }, 2. the return value that json_parse_next returns. 25 »···" \"sub2\":-12.456,\n" 148 »···»···»···»···ret = json_parse_next(parser, &type, &value);
I tried a non negative number and got success which is suspicious
Breakpoint 5, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:164 164 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $122 = 25 $123 = JSON_TYPE_NUMBER $124 = JSON_TYPE_NUMBER $125 = 0x555555633b2c "12.456" $126 = 0x555555693110 "12.456" $127 = 0 (gdb)
Also the return value otherwise LGTM, it is recognized as a number: 540 } else if ((ret = json_parse_number(parser, value_r)) >= 0) { (gdb) n 541 *type_r = JSON_TYPE_NUMBER;
So parsing worked one might think, but the value was not set.
(gdb) p *value_r $833 = 0x555555693110 ""
It turns out json_parse_number is too optimized. Too optimized for debugging further into json_parse_number. But also optimization is causing the issue. With -O0 or -O1 it works fine, even putting the lower optimization level just around json_parse_number it is enough to avoid the issue.
Not a fix but "avoidance" for now:
--- a/src/lib/json-parser.c +++ b/src/lib/json-parser.c @@ -386,6 +386,8 @@ static int json_parse_int(struct json_pa return 1; }
+#pragma GCC push_options +#pragma GCC optimize ("-O0") static int json_parse_number(struct json_parser *parser, const char **value_r) { int ret; @@ -415,6 +417,7 @@ static int json_parse_number(struct json *value_r = str_c(parser->value); return 1; } +#pragma GCC pop_options
static int json_parse_atom(struct json_parser *parser, const char *atom) {
I was trying to see what happened and tracked each str_append_c that happened inside json_parse_number. A breakpoint on str_append_c and backtraces showed that we clearly do the "right" parsing. From the backtraces I've seen it does (just from positions):
For example (gdb) bt #0 str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 #1 json_parse_digits (parser=<optimized out>) at json-parser.c:366 => that is a digit
It does "- digit digit dot digit digit digit"
This perfectly matches the input value, but still the result stays empty: (gdb) p (char*)(str->data) $738 = 0x555555693110 ""
In comparison - when parsing the latter 12.456e9 which works I see two odd things: 1. the value isn't cleared despite str_truncate(parser->value, 0); at the beginning of json_parse_number 2. the value (this time) is read exactly as I assumed
Breakpoint 17, json_parse_number (value_r=0x7fffffffe1a0, parser=0x555555692ee0) at json-parser.c:394 394 str_truncate(parser->value, 0); Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $741 = 0x555555693110 "sub3"
^^ not cleared, still has "sub3" even later on
Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $742 = 0x555555693110 "1ub3"
^^ reading left to right one char at a time "over the old value", which didn't update the data.
So it seems in json_parse_number the ordering of a) str_truncate(parser->value, 0); b) the parsing itself is messed up by optimization.
We already know from the above that reducing just this function to -O1 or less avoids the issue. But what is it really - I don't know?
Maybe you have a better idea what is going on, the issue should be reproducible when building dovecot with the toolchain present on the latest Ubuntu being 21.04 (Hirsute) right now.
-- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd
On Tue, Mar 30, 2021 at 13:34:54 -0400, Josef 'Jeff' Sipek wrote:
On Tue, Mar 30, 2021 at 17:53:27 +0200, Christian Ehrhardt wrote:
Hi, the recent Ubuntu (re)builds uncovered an issue with dovecot 1:2.3.13+dfsg1-1 build log: https://launchpadlibrarian.net/529849650/buildlog_ubuntu-hirsute-amd64.dovec... A coworker tried 2.3.14 but got the same result.
What fails is the json_parser build time test like: test-json-parser.c:161: Assert(#25) failed: null_strcmp(json_output[pos].value, value) == 0
I was looking into that a bit more and what I found is that it is dependent on the new toolchain of gcc 10.2.0-1.
FWIW, I managed to reproduce it on FreeBSD with gcc 11, so the good news for you is that it isn't Ubuntu specific :)
I'll debug further.
The culprit seems to be LTO. If you disable LTO, everything should work just fine. So, I think that'll be the "official" workaround - and a much better one than disabling optimization completely. Now, the big question is, is something in the test breaking or is the parser itself somehow triggering this. Jeff.
Thanks again for the report,
Jeff.
Not all calls to json_parse_* fail, e.g. the first one looks all good and passes I was iterating the tests using a report function defined like
(gdb) define repcon
c p pos p json_output[pos].type p type p json_output[pos].value p value call null_strcmp(json_output[pos].value, value) end
The first one to be bad was: Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $84 = 25 $85 = JSON_TYPE_NUMBER $86 = JSON_TYPE_NUMBER $87 = 0x555555633b25 "-12.456" $88 = 0x555555693110 "" $89 = 45
Earlier and later parsing was happy, for example
Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $90 = 27 $91 = JSON_TYPE_NUMBER $92 = JSON_TYPE_NUMBER $93 = 0x555555633b32 "12.456e9" $94 = 0x555555693110 "12.456e9" $95 = 0 (gdb)
We have two things we compare here. 1. json_output[] which is a static define and for this value is 67 »···{ JSON_TYPE_NUMBER, "-12.456" }, 2. the return value that json_parse_next returns. 25 »···" \"sub2\":-12.456,\n" 148 »···»···»···»···ret = json_parse_next(parser, &type, &value);
I tried a non negative number and got success which is suspicious
Breakpoint 5, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:164 164 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $122 = 25 $123 = JSON_TYPE_NUMBER $124 = JSON_TYPE_NUMBER $125 = 0x555555633b2c "12.456" $126 = 0x555555693110 "12.456" $127 = 0 (gdb)
Also the return value otherwise LGTM, it is recognized as a number: 540 } else if ((ret = json_parse_number(parser, value_r)) >= 0) { (gdb) n 541 *type_r = JSON_TYPE_NUMBER;
So parsing worked one might think, but the value was not set.
(gdb) p *value_r $833 = 0x555555693110 ""
It turns out json_parse_number is too optimized. Too optimized for debugging further into json_parse_number. But also optimization is causing the issue. With -O0 or -O1 it works fine, even putting the lower optimization level just around json_parse_number it is enough to avoid the issue.
Not a fix but "avoidance" for now:
--- a/src/lib/json-parser.c +++ b/src/lib/json-parser.c @@ -386,6 +386,8 @@ static int json_parse_int(struct json_pa return 1; }
+#pragma GCC push_options +#pragma GCC optimize ("-O0") static int json_parse_number(struct json_parser *parser, const char **value_r) { int ret; @@ -415,6 +417,7 @@ static int json_parse_number(struct json *value_r = str_c(parser->value); return 1; } +#pragma GCC pop_options
static int json_parse_atom(struct json_parser *parser, const char *atom) {
I was trying to see what happened and tracked each str_append_c that happened inside json_parse_number. A breakpoint on str_append_c and backtraces showed that we clearly do the "right" parsing. From the backtraces I've seen it does (just from positions):
For example (gdb) bt #0 str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 #1 json_parse_digits (parser=<optimized out>) at json-parser.c:366 => that is a digit
It does "- digit digit dot digit digit digit"
This perfectly matches the input value, but still the result stays empty: (gdb) p (char*)(str->data) $738 = 0x555555693110 ""
In comparison - when parsing the latter 12.456e9 which works I see two odd things: 1. the value isn't cleared despite str_truncate(parser->value, 0); at the beginning of json_parse_number 2. the value (this time) is read exactly as I assumed
Breakpoint 17, json_parse_number (value_r=0x7fffffffe1a0, parser=0x555555692ee0) at json-parser.c:394 394 str_truncate(parser->value, 0); Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $741 = 0x555555693110 "sub3"
^^ not cleared, still has "sub3" even later on
Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $742 = 0x555555693110 "1ub3"
^^ reading left to right one char at a time "over the old value", which didn't update the data.
So it seems in json_parse_number the ordering of a) str_truncate(parser->value, 0); b) the parsing itself is messed up by optimization.
We already know from the above that reducing just this function to -O1 or less avoids the issue. But what is it really - I don't know?
Maybe you have a better idea what is going on, the issue should be reproducible when building dovecot with the toolchain present on the latest Ubuntu being 21.04 (Hirsute) right now.
-- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd
On Tue, Mar 30, 2021 at 9:21 PM Josef 'Jeff' Sipek
On Tue, Mar 30, 2021 at 13:34:54 -0400, Josef 'Jeff' Sipek wrote:
On Tue, Mar 30, 2021 at 17:53:27 +0200, Christian Ehrhardt wrote:
Hi, the recent Ubuntu (re)builds uncovered an issue with dovecot 1:2.3.13+dfsg1-1 build log: https://launchpadlibrarian.net/529849650/buildlog_ubuntu-hirsute-amd64.dovec... A coworker tried 2.3.14 but got the same result.
What fails is the json_parser build time test like: test-json-parser.c:161: Assert(#25) failed: null_strcmp(json_output[pos].value, value) == 0
I was looking into that a bit more and what I found is that it is dependent on the new toolchain of gcc 10.2.0-1.
FWIW, I managed to reproduce it on FreeBSD with gcc 11, so the good news for you is that it isn't Ubuntu specific :)
I'll debug further.
The culprit seems to be LTO. If you disable LTO, everything should work just fine.
I've had LTO disabled and it has still shown the same effect (with my gcc 10.2.0-1). I'll give it a non-LTO retry and double check if it really changed the compile options accordingly. I'll let you know about that later on.
So, I think that'll be the "official" workaround - and a much better one than disabling optimization completely.
Well, "completely" is a bit hard, as I only disabled it on a single function and not the full build :-) But yeah if it really turns out to be LTO then disabling that will be fine as an avoidance until we've found the underlying root cause.
Now, the big question is, is something in the test breaking or is the parser itself somehow triggering this.
Jeff.
Thanks again for the report,
Jeff.
Not all calls to json_parse_* fail, e.g. the first one looks all good and passes I was iterating the tests using a report function defined like
(gdb) define repcon
c p pos p json_output[pos].type p type p json_output[pos].value p value call null_strcmp(json_output[pos].value, value) end
The first one to be bad was: Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $84 = 25 $85 = JSON_TYPE_NUMBER $86 = JSON_TYPE_NUMBER $87 = 0x555555633b25 "-12.456" $88 = 0x555555693110 "" $89 = 45
Earlier and later parsing was happy, for example
Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $90 = 27 $91 = JSON_TYPE_NUMBER $92 = JSON_TYPE_NUMBER $93 = 0x555555633b32 "12.456e9" $94 = 0x555555693110 "12.456e9" $95 = 0 (gdb)
We have two things we compare here. 1. json_output[] which is a static define and for this value is 67 »···{ JSON_TYPE_NUMBER, "-12.456" }, 2. the return value that json_parse_next returns. 25 »···" \"sub2\":-12.456,\n" 148 »···»···»···»···ret = json_parse_next(parser, &type, &value);
I tried a non negative number and got success which is suspicious
Breakpoint 5, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:164 164 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $122 = 25 $123 = JSON_TYPE_NUMBER $124 = JSON_TYPE_NUMBER $125 = 0x555555633b2c "12.456" $126 = 0x555555693110 "12.456" $127 = 0 (gdb)
Also the return value otherwise LGTM, it is recognized as a number: 540 } else if ((ret = json_parse_number(parser, value_r)) >= 0) { (gdb) n 541 *type_r = JSON_TYPE_NUMBER;
So parsing worked one might think, but the value was not set.
(gdb) p *value_r $833 = 0x555555693110 ""
It turns out json_parse_number is too optimized. Too optimized for debugging further into json_parse_number. But also optimization is causing the issue. With -O0 or -O1 it works fine, even putting the lower optimization level just around json_parse_number it is enough to avoid the issue.
Not a fix but "avoidance" for now:
--- a/src/lib/json-parser.c +++ b/src/lib/json-parser.c @@ -386,6 +386,8 @@ static int json_parse_int(struct json_pa return 1; }
+#pragma GCC push_options +#pragma GCC optimize ("-O0") static int json_parse_number(struct json_parser *parser, const char **value_r) { int ret; @@ -415,6 +417,7 @@ static int json_parse_number(struct json *value_r = str_c(parser->value); return 1; } +#pragma GCC pop_options
static int json_parse_atom(struct json_parser *parser, const char *atom) {
I was trying to see what happened and tracked each str_append_c that happened inside json_parse_number. A breakpoint on str_append_c and backtraces showed that we clearly do the "right" parsing. From the backtraces I've seen it does (just from positions):
For example (gdb) bt #0 str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 #1 json_parse_digits (parser=<optimized out>) at json-parser.c:366 => that is a digit
It does "- digit digit dot digit digit digit"
This perfectly matches the input value, but still the result stays empty: (gdb) p (char*)(str->data) $738 = 0x555555693110 ""
In comparison - when parsing the latter 12.456e9 which works I see two odd things: 1. the value isn't cleared despite str_truncate(parser->value, 0); at the beginning of json_parse_number 2. the value (this time) is read exactly as I assumed
Breakpoint 17, json_parse_number (value_r=0x7fffffffe1a0, parser=0x555555692ee0) at json-parser.c:394 394 str_truncate(parser->value, 0); Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $741 = 0x555555693110 "sub3"
^^ not cleared, still has "sub3" even later on
Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $742 = 0x555555693110 "1ub3"
^^ reading left to right one char at a time "over the old value", which didn't update the data.
So it seems in json_parse_number the ordering of a) str_truncate(parser->value, 0); b) the parsing itself is messed up by optimization.
We already know from the above that reducing just this function to -O1 or less avoids the issue. But what is it really - I don't know?
Maybe you have a better idea what is going on, the issue should be reproducible when building dovecot with the toolchain present on the latest Ubuntu being 21.04 (Hirsute) right now.
-- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd
-- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd
On Wed, Mar 31, 2021 at 8:46 AM Christian Ehrhardt
On Tue, Mar 30, 2021 at 9:21 PM Josef 'Jeff' Sipek
wrote: On Tue, Mar 30, 2021 at 13:34:54 -0400, Josef 'Jeff' Sipek wrote:
On Tue, Mar 30, 2021 at 17:53:27 +0200, Christian Ehrhardt wrote:
Hi, the recent Ubuntu (re)builds uncovered an issue with dovecot 1:2.3.13+dfsg1-1 build log: https://launchpadlibrarian.net/529849650/buildlog_ubuntu-hirsute-amd64.dovec... A coworker tried 2.3.14 but got the same result.
What fails is the json_parser build time test like: test-json-parser.c:161: Assert(#25) failed: null_strcmp(json_output[pos].value, value) == 0
I was looking into that a bit more and what I found is that it is dependent on the new toolchain of gcc 10.2.0-1.
FWIW, I managed to reproduce it on FreeBSD with gcc 11, so the good news for you is that it isn't Ubuntu specific :)
I'll debug further.
The culprit seems to be LTO. If you disable LTO, everything should work just fine.
I've had LTO disabled and it has still shown the same effect (with my gcc 10.2.0-1). I'll give it a non-LTO retry and double check if it really changed the compile options accordingly. I'll let you know about that later on.
Indeed, I wonder what I tried yesterday in regard to LTO then .. :-/ I can confirm that disabling LTO fixes the issue for me as well and for now that should be a good mitigation until the root cause is found and fixed. Since it might help debugging the underlying problem with LTO here is another data point. With LTO enabled (and skipping the json-parser issues with my optimization trick) there is another testcase later that fails (but works with LTO disabled): test-istream-attachment.c:354: Assert failed: memcmp(data + sizeof(BINARY_TEXT_LONG)-1, BINARY_TEXT_SHORT, strlen(BINARY_TEXT_SHORT)) == 0 istream attachment ................................................... : FAILED Panic: file test-istream-attachment.c: line 395 (test_istream_attachment_extractor_one): assertion failed: (size >= prefix_len && memcmp(data, mail_broken_input_body_prefix, prefix_len) == 0) Error: Raw backtrace: ./test-istream-attachment(+0x4cd95) [0x55c0db91bd95] -> ./test-istream-attachment(backtrace_get+0x75) [0x55c0db91bf65] -> ./test-istream-attachment(+0x2a7fb) [0x55c0db8f97fb] -> ./test-istream-attachment(+0x2a837) [0x55c0db8f9837] -> ./test-istream-attachment(+0x13c5c) [0x55c0db8e2c5c] -> ./test-istream-attachment(+0x12d39) [0x55c0db8e1d39] -> ./test-istream-attachment(+0x1cca3) [0x55c0db8ebca3] -> ./test-istream-attachment(+0x2424d) [0x55c0db8f324d] -> ./test-istream-attachment(test_run+0x63) [0x55c0db8f32f3] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xd5) [0x7f60d232d565] -> ./test-istream-attachment(_start+0x2e) [0x55c0db8e7c2e] /bin/bash: line 1: 1650909 Aborted (core dumped) ./$bin
So, I think that'll be the "official" workaround - and a much better one than disabling optimization completely.
Well, "completely" is a bit hard, as I only disabled it on a single function and not the full build :-) But yeah if it really turns out to be LTO then disabling that will be fine as an avoidance until we've found the underlying root cause.
Now, the big question is, is something in the test breaking or is the parser itself somehow triggering this.
Jeff.
Thanks again for the report,
Jeff.
Not all calls to json_parse_* fail, e.g. the first one looks all good and passes I was iterating the tests using a report function defined like
(gdb) define repcon
c p pos p json_output[pos].type p type p json_output[pos].value p value call null_strcmp(json_output[pos].value, value) end
The first one to be bad was: Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $84 = 25 $85 = JSON_TYPE_NUMBER $86 = JSON_TYPE_NUMBER $87 = 0x555555633b25 "-12.456" $88 = 0x555555693110 "" $89 = 45
Earlier and later parsing was happy, for example
Breakpoint 2, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:161 161 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $90 = 27 $91 = JSON_TYPE_NUMBER $92 = JSON_TYPE_NUMBER $93 = 0x555555633b32 "12.456e9" $94 = 0x555555693110 "12.456e9" $95 = 0 (gdb)
We have two things we compare here. 1. json_output[] which is a static define and for this value is 67 »···{ JSON_TYPE_NUMBER, "-12.456" }, 2. the return value that json_parse_next returns. 25 »···" \"sub2\":-12.456,\n" 148 »···»···»···»···ret = json_parse_next(parser, &type, &value);
I tried a non negative number and got success which is suspicious
Breakpoint 5, test_json_parser_success (full_size=<optimized out>) at test-json-parser.c:164 164 test_assert_idx(null_strcmp(json_output[pos].value, value) == 0, pos); $122 = 25 $123 = JSON_TYPE_NUMBER $124 = JSON_TYPE_NUMBER $125 = 0x555555633b2c "12.456" $126 = 0x555555693110 "12.456" $127 = 0 (gdb)
Also the return value otherwise LGTM, it is recognized as a number: 540 } else if ((ret = json_parse_number(parser, value_r)) >= 0) { (gdb) n 541 *type_r = JSON_TYPE_NUMBER;
So parsing worked one might think, but the value was not set.
(gdb) p *value_r $833 = 0x555555693110 ""
It turns out json_parse_number is too optimized. Too optimized for debugging further into json_parse_number. But also optimization is causing the issue. With -O0 or -O1 it works fine, even putting the lower optimization level just around json_parse_number it is enough to avoid the issue.
Not a fix but "avoidance" for now:
--- a/src/lib/json-parser.c +++ b/src/lib/json-parser.c @@ -386,6 +386,8 @@ static int json_parse_int(struct json_pa return 1; }
+#pragma GCC push_options +#pragma GCC optimize ("-O0") static int json_parse_number(struct json_parser *parser, const char **value_r) { int ret; @@ -415,6 +417,7 @@ static int json_parse_number(struct json *value_r = str_c(parser->value); return 1; } +#pragma GCC pop_options
static int json_parse_atom(struct json_parser *parser, const char *atom) {
I was trying to see what happened and tracked each str_append_c that happened inside json_parse_number. A breakpoint on str_append_c and backtraces showed that we clearly do the "right" parsing. From the backtraces I've seen it does (just from positions):
For example (gdb) bt #0 str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 #1 json_parse_digits (parser=<optimized out>) at json-parser.c:366 => that is a digit
It does "- digit digit dot digit digit digit"
This perfectly matches the input value, but still the result stays empty: (gdb) p (char*)(str->data) $738 = 0x555555693110 ""
In comparison - when parsing the latter 12.456e9 which works I see two odd things: 1. the value isn't cleared despite str_truncate(parser->value, 0); at the beginning of json_parse_number 2. the value (this time) is read exactly as I assumed
Breakpoint 17, json_parse_number (value_r=0x7fffffffe1a0, parser=0x555555692ee0) at json-parser.c:394 394 str_truncate(parser->value, 0); Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $741 = 0x555555693110 "sub3"
^^ not cleared, still has "sub3" even later on
Breakpoint 18, str_append_c (chr=<optimized out>, str=0x5555556930d0) at str.h:44 44 buffer_append_c(str, chr); (gdb) p (char*)(str->data) $742 = 0x555555693110 "1ub3"
^^ reading left to right one char at a time "over the old value", which didn't update the data.
So it seems in json_parse_number the ordering of a) str_truncate(parser->value, 0); b) the parsing itself is messed up by optimization.
We already know from the above that reducing just this function to -O1 or less avoids the issue. But what is it really - I don't know?
Maybe you have a better idea what is going on, the issue should be reproducible when building dovecot with the toolchain present on the latest Ubuntu being 21.04 (Hirsute) right now.
-- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd
-- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd
-- Christian Ehrhardt Staff Engineer, Ubuntu Server Canonical Ltd
On Wed, Mar 31, 2021 at 09:07:28 +0200, Christian Ehrhardt wrote:
On Wed, Mar 31, 2021 at 8:46 AM Christian Ehrhardt christian.ehrhardt@canonical.com wrote:
On Tue, Mar 30, 2021 at 9:21 PM Josef 'Jeff' Sipek jeff.sipek@open-xchange.com wrote: ...
The culprit seems to be LTO. If you disable LTO, everything should work just fine.
I've had LTO disabled and it has still shown the same effect (with my gcc 10.2.0-1). I'll give it a non-LTO retry and double check if it really changed the compile options accordingly. I'll let you know about that later on.
Indeed, I wonder what I tried yesterday in regard to LTO then .. :-/ I can confirm that disabling LTO fixes the issue for me as well and for now that should be a good mitigation until the root cause is found and fixed.
Sounds good. Thanks for the confirmation.
Since it might help debugging the underlying problem with LTO here is another data point. With LTO enabled (and skipping the json-parser issues with my optimization trick) there is another testcase later that fails (but works with LTO disabled):
test-istream-attachment.c:354: Assert failed: memcmp(data + sizeof(BINARY_TEXT_LONG)-1, BINARY_TEXT_SHORT, strlen(BINARY_TEXT_SHORT)) == 0 istream attachment ................................................... : FAILED
Yep, fails here as well.
...
So, I think that'll be the "official" workaround - and a much better one than disabling optimization completely.
Well, "completely" is a bit hard, as I only disabled it on a single function and not the full build :-)
Agreed ;)
Jeff.
On Wed, Mar 31, 2021 at 11:26:14 -0400, Josef 'Jeff' Sipek wrote:
On Wed, Mar 31, 2021 at 09:07:28 +0200, Christian Ehrhardt wrote:
On Wed, Mar 31, 2021 at 8:46 AM Christian Ehrhardt
wrote: On Tue, Mar 30, 2021 at 9:21 PM Josef 'Jeff' Sipek
wrote: ... The culprit seems to be LTO. If you disable LTO, everything should work just fine.
I've had LTO disabled and it has still shown the same effect (with my gcc 10.2.0-1). I'll give it a non-LTO retry and double check if it really changed the compile options accordingly. I'll let you know about that later on.
Indeed, I wonder what I tried yesterday in regard to LTO then .. :-/ I can confirm that disabling LTO fixes the issue for me as well and for now that should be a good mitigation until the root cause is found and fixed.
Sounds good. Thanks for the confirmation.
Does the following patch make the tests pass for you? We still need to
clean it up a bit, but I figured I'd share the work-in-progress version to
make sure we're seeing the same issues as you. (And the obvious disclaimer:
this patch hasn't gone through the usual before-commit scrutiny, so don't
ship it.)
Thanks,
Jeff.
diff --git a/src/anvil/anvil-settings.c b/src/anvil/anvil-settings.c
--- a/src/anvil/anvil-settings.c
+++ b/src/anvil/anvil-settings.c
@@ -16,8 +16,8 @@ static struct file_listener_settings *an
&anvil_unix_listeners_array[0],
&anvil_unix_listeners_array[1]
};
-static buffer_t anvil_unix_listeners_buf = {
- anvil_unix_listeners, sizeof(anvil_unix_listeners), { NULL, }
+static union static_buffer anvil_unix_listeners_buf = {
+ .buf = { anvil_unix_listeners, sizeof(anvil_unix_listeners) }
};
/* */
@@ -41,7 +41,7 @@ struct service_settings anvil_service_se
.idle_kill = UINT_MAX,
.vsz_limit = UOFF_T_MAX,
- .unix_listeners = { { &anvil_unix_listeners_buf,
+ .unix_listeners = { { &anvil_unix_listeners_buf.buf,
sizeof(anvil_unix_listeners[0]) } },
.fifo_listeners = ARRAY_INIT,
.inet_listeners = ARRAY_INIT,
diff --git a/src/auth/auth-master-connection.c b/src/auth/auth-master-connection.c
--- a/src/auth/auth-master-connection.c
+++ b/src/auth/auth-master-connection.c
@@ -108,7 +108,7 @@ master_input_request(struct auth_master_
const char *const *list, *const *params;
unsigned int id, client_pid, client_id;
uint8_t cookie[MASTER_AUTH_COOKIE_SIZE];
- buffer_t buf;
+ union static_buffer buf;
/* <id> <client-pid> <client-id> <cookie> [<parameters>] */
list = t_strsplit_tabescaped(args);
@@ -121,7 +121,7 @@ master_input_request(struct auth_master_
}
buffer_create_from_data(&buf, cookie, sizeof(cookie));
- if (hex_to_binary(list[3], &buf) < 0) {
+ if (hex_to_binary(list[3], &buf.buf) < 0) {
e_error(conn->event, "BUG: Master sent broken REQUEST cookie");
return FALSE;
}
diff --git a/src/auth/auth-settings.c b/src/auth/auth-settings.c
--- a/src/auth/auth-settings.c
+++ b/src/auth/auth-settings.c
@@ -32,8 +32,8 @@ static struct file_listener_settings *au
&auth_unix_listeners_array[4],
&auth_unix_listeners_array[5]
};
-static buffer_t auth_unix_listeners_buf = {
- auth_unix_listeners, sizeof(auth_unix_listeners), { NULL, }
+static union static_buffer auth_unix_listeners_buf = {
+ .buf = { auth_unix_listeners, sizeof(auth_unix_listeners) }
};
/* */
@@ -57,7 +57,7 @@ struct service_settings auth_service_set
.idle_kill = 0,
.vsz_limit = UOFF_T_MAX,
- .unix_listeners = { { &auth_unix_listeners_buf,
+ .unix_listeners = { { &auth_unix_listeners_buf.buf,
sizeof(auth_unix_listeners[0]) } },
.fifo_listeners = ARRAY_INIT,
.inet_listeners = ARRAY_INIT,
@@ -72,8 +72,8 @@ static struct file_listener_settings aut
static struct file_listener_settings *auth_worker_unix_listeners[] = {
&auth_worker_unix_listeners_array[0]
};
-static buffer_t auth_worker_unix_listeners_buf = {
- auth_worker_unix_listeners, sizeof(auth_worker_unix_listeners), { NULL, }
+static union static_buffer auth_worker_unix_listeners_buf = {
+ .buf = { auth_worker_unix_listeners, sizeof(auth_worker_unix_listeners) }
};
/* */
@@ -97,7 +97,7 @@ struct service_settings auth_worker_serv
.idle_kill = 0,
.vsz_limit = UOFF_T_MAX,
- .unix_listeners = { { &auth_worker_unix_listeners_buf,
+ .unix_listeners = { { &auth_worker_unix_listeners_buf.buf,
sizeof(auth_worker_unix_listeners[0]) } },
.fifo_listeners = ARRAY_INIT,
.inet_listeners = ARRAY_INIT
diff --git a/src/auth/mech-digest-md5.c b/src/auth/mech-digest-md5.c
--- a/src/auth/mech-digest-md5.c
+++ b/src/auth/mech-digest-md5.c
@@ -25,7 +25,7 @@ static const char *qop_names[] = { "auth
static string_t *get_digest_challenge(struct digest_auth_request *request)
{
const struct auth_settings *set = request->auth_request.set;
- buffer_t buf;
+ union static_buffer buf;
string_t *str;
const char *const *tmp;
unsigned char nonce[16];
@@ -47,9 +47,9 @@ static string_t *get_digest_challenge(st
random_fill(nonce, sizeof(nonce));
buffer_create_from_data(&buf, nonce_base64, sizeof(nonce_base64));
- base64_encode(nonce, sizeof(nonce), &buf);
- buffer_append_c(&buf, '\0');
- request->nonce = p_strdup(request->pool, buf.data);
+ base64_encode(nonce, sizeof(nonce), &buf.buf);
+ buffer_append_c(&buf.buf, '\0');
+ request->nonce = p_strdup(request->pool, buf.buf.data);
str = t_str_new(256);
if (*set->realms_arr == NULL) {
diff --git a/src/auth/password-scheme-pbkdf2.c b/src/auth/password-scheme-pbkdf2.c
--- a/src/auth/password-scheme-pbkdf2.c
+++ b/src/auth/password-scheme-pbkdf2.c
@@ -18,13 +18,13 @@ pbkdf_run(const char *plaintext, const c
unsigned int rounds, unsigned char key_r[PBKDF2_KEY_SIZE_SHA1])
{
memset(key_r, 0, PBKDF2_KEY_SIZE_SHA1);
- buffer_t buf;
+ union static_buffer buf;
buffer_create_from_data(&buf, key_r, PBKDF2_KEY_SIZE_SHA1);
pkcs5_pbkdf(PKCS5_PBKDF2, hash_method_lookup("sha1"),
(const unsigned char *)plaintext, strlen(plaintext),
(const unsigned char *)salt, strlen(salt),
- rounds, PBKDF2_KEY_SIZE_SHA1, &buf);
+ rounds, PBKDF2_KEY_SIZE_SHA1, &buf.buf);
}
void pbkdf2_generate(const char *plaintext, const struct password_generate_params *params ATTR_UNUSED,
@@ -55,7 +55,7 @@ int pbkdf2_verify(const char *plaintext,
const char *salt;
unsigned int rounds;
unsigned char key1[PBKDF2_KEY_SIZE_SHA1], key2[PBKDF2_KEY_SIZE_SHA1];
- buffer_t buf;
+ union static_buffer buf;
/* $1$salt$rounds$hash */
if (size < 3 || memcmp(raw_password, "$1$", 3) != 0) {
@@ -72,7 +72,7 @@ int pbkdf2_verify(const char *plaintext,
}
buffer_create_from_data(&buf, key1, sizeof(key1));
if (strlen(fields[2]) != sizeof(key1)*2 ||
- hex_to_binary(fields[2], &buf) < 0) {
+ hex_to_binary(fields[2], &buf.buf) < 0) {
*error_r = "PBKDF2 hash not 160bit hex-encoded";
return -1;
}
diff --git a/src/auth/test-db-dict.c b/src/auth/test-db-dict.c
--- a/src/auth/test-db-dict.c
+++ b/src/auth/test-db-dict.c
@@ -23,7 +23,7 @@ void test_db_dict_parse_cache_key(void)
const struct db_dict_key *objects[] = {
&keys[3], &keys[4]
};
- buffer_t keybuf, fieldbuf, objectbuf;
+ union static_buffer keybuf, fieldbuf, objectbuf;
ARRAY_TYPE(db_dict_key) keyarr;
ARRAY_TYPE(db_dict_field) fieldarr;
ARRAY_TYPE(db_dict_key_p) objectarr;
@@ -33,9 +33,9 @@ void test_db_dict_parse_cache_key(void)
buffer_create_from_const_data(&keybuf, keys, sizeof(keys));
buffer_create_from_const_data(&fieldbuf, fields, sizeof(fields));
buffer_create_from_const_data(&objectbuf, objects, sizeof(objects));
- array_create_from_buffer(&keyarr, &keybuf, sizeof(keys[0]));
- array_create_from_buffer(&fieldarr, &fieldbuf, sizeof(fields[0]));
- array_create_from_buffer(&objectarr, &objectbuf, sizeof(objects[0]));
+ array_create_from_buffer(&keyarr, &keybuf.buf, sizeof(keys[0]));
+ array_create_from_buffer(&fieldarr, &fieldbuf.buf, sizeof(fields[0]));
+ array_create_from_buffer(&objectarr, &objectbuf.buf, sizeof(objects[0]));
test_assert(strcmp(db_dict_parse_cache_key(&keyarr, &fieldarr, &objectarr),
"\t%d and %n\t%l\t%{foo}%r%{bar}\t%{test1}/path\t%{extra}\tpath2/%{test2}\t%{plop}") == 0);
diff --git a/src/config/config-settings.c b/src/config/config-settings.c
--- a/src/config/config-settings.c
+++ b/src/config/config-settings.c
@@ -14,8 +14,8 @@ static struct file_listener_settings con
static struct file_listener_settings *config_unix_listeners[] = {
&config_unix_listeners_array[0]
};
-static buffer_t config_unix_listeners_buf = {
- config_unix_listeners, sizeof(config_unix_listeners), { NULL, }
+static union static_buffer config_unix_listeners_buf = {
+ .buf = { config_unix_listeners, sizeof(config_unix_listeners) }
};
/* */
@@ -39,7 +39,7 @@ struct service_settings config_service_s
.idle_kill = UINT_MAX,
.vsz_limit = UOFF_T_MAX,
- .unix_listeners = { { &config_unix_listeners_buf,
+ .unix_listeners = { { &config_unix_listeners_buf.buf,
sizeof(config_unix_listeners[0]) } },
.fifo_listeners = ARRAY_INIT,
.inet_listeners = ARRAY_INIT
diff --git a/src/config/settings-get.pl b/src/config/settings-get.pl
--- a/src/config/settings-get.pl
+++ b/src/config/settings-get.pl
@@ -23,7 +23,7 @@ print '#include "all-settings.h"'."\n";
print '#include
On 1. Apr 2021, at 19.14, Josef 'Jeff' Sipek jeff.sipek@open-xchange.com wrote:
On Wed, Mar 31, 2021 at 11:26:14 -0400, Josef 'Jeff' Sipek wrote:
On Wed, Mar 31, 2021 at 09:07:28 +0200, Christian Ehrhardt wrote:
On Wed, Mar 31, 2021 at 8:46 AM Christian Ehrhardt christian.ehrhardt@canonical.com wrote:
On Tue, Mar 30, 2021 at 9:21 PM Josef 'Jeff' Sipek jeff.sipek@open-xchange.com wrote: ...
The culprit seems to be LTO. If you disable LTO, everything should work just fine.
I've had LTO disabled and it has still shown the same effect (with my gcc 10.2.0-1). I'll give it a non-LTO retry and double check if it really changed the compile options accordingly. I'll let you know about that later on.
Indeed, I wonder what I tried yesterday in regard to LTO then .. :-/ I can confirm that disabling LTO fixes the issue for me as well and for now that should be a good mitigation until the root cause is found and fixed.
Sounds good. Thanks for the confirmation.
Does the following patch make the tests pass for you? We still need to clean it up a bit, but I figured I'd share the work-in-progress version to make sure we're seeing the same issues as you. (And the obvious disclaimer: this patch hasn't gone through the usual before-commit scrutiny, so don't ship it.)
The fix is now merged to master. It ended up being a bit different (simpler) from what Jeff sent to the list. The main fixes are:
https://github.com/dovecot/core/commit/4e5bb6738d8907ede8ec157d07463a8e77293... https://github.com/dovecot/core/commit/4e5bb6738d8907ede8ec157d07463a8e77293... https://github.com/dovecot/core/commit/1d9b4e14008b15b7a34b5c633b09b9670e866... https://github.com/dovecot/core/commit/1d9b4e14008b15b7a34b5c633b09b9670e866...
participants (4)
-
Aki Tuomi
-
Christian Ehrhardt
-
Josef 'Jeff' Sipek
-
Timo Sirainen