[dovecot/core] d0dc63: lib-storage: Fix truncating header values at NUL w...
GitHub
noreply at github.com
Mon Apr 23 14:30:18 EEST 2018
Branch: refs/heads/master
Home: https://github.com/dovecot/core
Commit: d0dc63004b67a111570d6db4b0fe596c9a56d55c
https://github.com/dovecot/core/commit/d0dc63004b67a111570d6db4b0fe596c9a56d55c
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-21 (Sat, 21 Apr 2018)
Changed paths:
M src/lib-storage/index/index-mail-headers.c
Log Message:
-----------
lib-storage: Fix truncating header values at NUL when adding to cache
For example FETCH BODY.PEEK[HEADER.FIELDS (Subject)] could have returned
the NUL on the first fetch when the mail wasn't in dovecot.index.cache,
but the second FETCH would have returned it truncated at the NUL.
Commit: 9abd4cb55a90fe4ff5fd574da932e11c08189839
https://github.com/dovecot/core/commit/9abd4cb55a90fe4ff5fd574da932e11c08189839
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-21 (Sat, 21 Apr 2018)
Changed paths:
M src/lib-mail/message-snippet.c
M src/lib-mail/test-message-snippet.c
Log Message:
-----------
lib-mail: message_snippet_generate() - Ignore NULs without shrinking snippet size
Previously the NULs also weren't in the snippet content, but they were
included in the snippet size.
Commit: 980dc2b45ea7981c8eef628a14580a238a36cc44
https://github.com/dovecot/core/commit/980dc2b45ea7981c8eef628a14580a238a36cc44
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-21 (Sat, 21 Apr 2018)
Changed paths:
M src/lib-mail/rfc822-parser.c
Log Message:
-----------
lib-mail: rfc822-parser: Don't truncate comment/quoted-string/domain-literal at NUL
The other tokens couldn't have contained NULs in the first place.
Commit: 6f456ea32afed46e62b06ef69c1965743fd73057
https://github.com/dovecot/core/commit/6f456ea32afed46e62b06ef69c1965743fd73057
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/doveadm/doveadm-dump-index.c
Log Message:
-----------
doveadm dump: Don't truncate dovecot.index.cache values at NULs
Commit: 1bb4e53ea674fe84dc0e1633f1cf1cb5f2fd2e49
https://github.com/dovecot/core/commit/1bb4e53ea674fe84dc0e1633f1cf1cb5f2fd2e49
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/doveadm/doveadm-print-table.c
Log Message:
-----------
doveadm: Fix table printer to not truncate output at NULs
Commit: a60deaa8cc47f8ecb0155b52ca4b849ee1fcfe5b
https://github.com/dovecot/core/commit/a60deaa8cc47f8ecb0155b52ca4b849ee1fcfe5b
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/auth/db-checkpassword.c
Log Message:
-----------
auth: checkpassword - Fail if input from script contains NULs
Previously the input was just silently truncated at NULs.
Commit: 302708ee9a2bcc5876cd1c54e7832da4f3943786
https://github.com/dovecot/core/commit/302708ee9a2bcc5876cd1c54e7832da4f3943786
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-dict-extra/dict-fs.c
M src/lib-dict/dict-memcached-ascii.c
M src/lib-dict/dict-redis.c
Log Message:
-----------
dict-fs/memcached-ascii/redis: Consistently truncate value at NULs
The dict API currently doesn't support NULs in values, so they have to be
truncated. However, previously they were truncated at NULs, but more data
may have still been appended to the value. This could have produced
somewhat random results.
Commit: 3821df49f69db6fab8402a14bb8d7646dea1e835
https://github.com/dovecot/core/commit/3821df49f69db6fab8402a14bb8d7646dea1e835
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/config/config-parser.c
Log Message:
-----------
config: Consistently truncate value at the first NUL in value input file
When value contained "<file", the file was added as a value. The setting
values don't currently support NULs, so the value gets truncated at the
first NUL. However, the previous code could have appended more data to
the value. This could have produced somewhat confusing results.
Commit: 0f9bbf22016c50de7f3cc30f4aaf5d877069e3dd
https://github.com/dovecot/core/commit/0f9bbf22016c50de7f3cc30f4aaf5d877069e3dd
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-master/master-login.c
M src/util/script-login.c
Log Message:
-----------
lib-master, util: Consistently truncate tabescaped input at the first NUL
The NULs are supposed to come tab-escaped, so there's no need to support
actual NUL characters. The previous code truncated at NULs, but could
have kept appending more data to the value. This could have produced
somewhat random results.
Commit: 46aa1c4ce13f637161dd2a9c135f108555a6dbe9
https://github.com/dovecot/core/commit/46aa1c4ce13f637161dd2a9c135f108555a6dbe9
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/login-common/client-common-auth.c
Log Message:
-----------
login-common: Consistently truncate client SASL input at first NUL
The input is supposed to be base64-encoded, so there's no need to support
actual NUL characters. The previous code truncated at NULs, but could have
kept appending more data to the value. This could have produced somewhat
random results.
Commit: fc92bc3d1126fdb15a7c6ef58a0fa61ad87dadb7
https://github.com/dovecot/core/commit/fc92bc3d1126fdb15a7c6ef58a0fa61ad87dadb7
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/plugins/fts-squat/fts-backend-squat.c
Log Message:
-----------
fts-squat: Fix truncating headers at NULs while indexing
Commit: b60044d481498ca4bfd2a47cfd11ecabb0471a5d
https://github.com/dovecot/core/commit/b60044d481498ca4bfd2a47cfd11ecabb0471a5d
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-mail/rfc2231-parser.c
M src/lib-mail/rfc822-parser.c
M src/lib-mail/rfc822-parser.h
M src/lib-mail/test-rfc822-parser.c
Log Message:
-----------
lib-mail: Change rfc822_parse_content_param() API to allow NULs in value
This was the only function in rfc822-parser.h that wasn't NUL-safe.
This won't fix anything, but it makes the rfc822-parser.h API fully
consistent with the NUL handling.
Normally rfc2231_parse() should be called instead of calling
rfc822_parse_content_param() directly, so this shouldn't break any plugins.
Commit: ee120303949217b7f9ac33e036cf38aea39822c3
https://github.com/dovecot/core/commit/ee120303949217b7f9ac33e036cf38aea39822c3
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-mail/rfc822-parser.c
M src/lib-mail/test-rfc822-parser.c
Log Message:
-----------
lib-mail: rfc822-parser: Don't allow preserving escaped [CR]LF
It's not valid to have "\<CR>" or "\<LF>", so the old behavior isn't really
wrong either. However, rfc822_parse_quoted_string() callers are more likely
to expect that the output won't contain any [CR]LF so this new behavior is
a bit better.
Commit: 39f9e4d3db603f312a43fddce21f566a960ff0b6
https://github.com/dovecot/core/commit/39f9e4d3db603f312a43fddce21f566a960ff0b6
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-mail/rfc822-parser.c
M src/lib-mail/test-rfc822-parser.c
Log Message:
-----------
lib-mail: rfc822-parser: Improve domain-literal parsing
Strip away folding whitespace. Treat any extra '[' as an error.
Commit: 8bdf274e70a562a611950de3765472337e8825e8
https://github.com/dovecot/core/commit/8bdf274e70a562a611950de3765472337e8825e8
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-mail/rfc822-parser.c
M src/lib-mail/test-rfc822-parser.c
Log Message:
-----------
lib-mail: rfc822-parser: Strip away folding whitespace from comments
Commit: 8d2ad317fce454e0aca3bd34ebf5fcf967393e3c
https://github.com/dovecot/core/commit/8d2ad317fce454e0aca3bd34ebf5fcf967393e3c
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-mail/rfc822-parser.c
M src/lib-mail/rfc822-parser.h
M src/lib-mail/test-rfc822-parser.c
Log Message:
-----------
lib-mail: rfc822-parser: Add nul_replacement_char
Commit: 39323a4666ad1be728c7f13c05f2fe4e15d43aa0
https://github.com/dovecot/core/commit/39323a4666ad1be728c7f13c05f2fe4e15d43aa0
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-mail/message-address.c
M src/lib-mail/test-message-address.c
Log Message:
-----------
lib-mail: message_address_parse*() - Replace NULs with 0x80
Instead of truncating the strings at NULs.
Commit: 621cd6da69cb9592d6324d70245863179a39c338
https://github.com/dovecot/core/commit/621cd6da69cb9592d6324d70245863179a39c338
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-mail/rfc2231-parser.c
M src/lib-mail/rfc2231-parser.h
M src/lib-mail/test-rfc2231-parser.c
Log Message:
-----------
lib-mail: rfc2231_parse() - Replace NULs with 0x80
Instead of truncating the strings at NULs.
Commit: 95fba2c295bb84639dc64ca5c9f2ccb9edc00ba7
https://github.com/dovecot/core/commit/95fba2c295bb84639dc64ca5c9f2ccb9edc00ba7
Author: Sergey Kitov <sergey.kitov at open-xchange.com>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-mail/message-header-parser.c
M src/lib-mail/message-header-parser.h
Log Message:
-----------
lib-mail: Add MESSAGE_HEADER_REPLACE_NULS_WITH_0x80 flag
The flag signals that input stream for message_parse_header() should replace
0x0 symbols with 0x80.
Commit: ab32237ffd78259013b4b2186292b0c80c34e955
https://github.com/dovecot/core/commit/ab32237ffd78259013b4b2186292b0c80c34e955
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib/str.h
Log Message:
-----------
lib: Improve str_append_n() comment
Commit: 495c324e4d361d52ac869306cc8c060b60fb317b
https://github.com/dovecot/core/commit/495c324e4d361d52ac869306cc8c060b60fb317b
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib/str.c
M src/lib/str.h
M src/lib/test-str.c
Log Message:
-----------
lib: Add str_append_max()
It's otherwise the same as str_append_n(), except it takes const char* as
input instead of const void*. This, as well as its name, should make it
less likely to be used wrong when the input may legitimately have NULs.
The unit test is changed from str_append_n() to str_append_max(), because
str_append_n() will be deprecated.
Commit: 07cbb7b116e0be888b43b4d27510f69e25b6e41d
https://github.com/dovecot/core/commit/07cbb7b116e0be888b43b4d27510f69e25b6e41d
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/auth/auth-cache.c
M src/auth/db-dict-cache-key.c
M src/auth/db-ldap.c
M src/auth/password-scheme-md5crypt.c
M src/config/config-parser.c
M src/config/doveconf.c
M src/lib-dict-backend/dict-ldap.c
M src/lib-dict/dict.c
M src/lib-fs/fs-posix.c
M src/lib-fs/fs-sis.c
M src/lib-fts/fts-tokenizer-address.c
M src/lib-http/http-auth.c
M src/lib-http/http-header-parser.c
M src/lib-http/http-parser.c
M src/lib-http/http-url.c
M src/lib-imap/imap-quote.c
M src/lib-imap/imap-url.c
M src/lib-imap/imap-utf7.c
M src/lib-mail/message-header-parser.c
M src/lib-mail/message-snippet.c
M src/lib-mail/rfc822-parser.c
M src/lib-mail/test-istream-dot.c
M src/lib-mail/test-istream-header-filter.c
M src/lib-mail/test-istream-qp-decoder.c
M src/lib-mail/test-istream-qp-encoder.c
M src/lib-mail/test-message-header-parser.c
M src/lib-mail/test-message-parser.c
M src/lib-settings/settings-parser.c
M src/lib-settings/settings.c
M src/lib-smtp/smtp-address.c
M src/lib-smtp/smtp-parser.c
M src/lib-smtp/smtp-reply-parser.c
M src/lib-smtp/smtp-server-reply.c
M src/lib-smtp/smtp-syntax.c
M src/lib-storage/index/index-mail-headers.c
M src/lib-storage/index/maildir/maildir-uidlist.c
M src/lib-storage/index/mbox/mbox-save.c
M src/lib-storage/index/mbox/mbox-sync-parse.c
M src/lib-storage/mailbox-attribute.c
M src/lib-storage/mailbox-list.c
M src/lib-storage/mailbox-tree.c
M src/lib/failures.c
M src/lib/file-dotlock.c
M src/lib/istream.c
M src/lib/str-sanitize.c
M src/lib/strescape.c
M src/lib/test-path-util.c
M src/lib/uri-util.c
M src/plugins/fts-solr/fts-backend-solr-old.c
M src/plugins/fts-solr/fts-backend-solr.c
Log Message:
-----------
global: Replace str_append_n() with str_append_data() wherever possible
It shouldn't be possible for input to have NULs in any of these places.
This makes the extra NUL-check in str_append_n() unnecessary.
Commit: b8b0e5660b871c40024063308d40c9607e3e3aad
https://github.com/dovecot/core/commit/b8b0e5660b871c40024063308d40c9607e3e3aad
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib/strescape.c
Log Message:
-----------
lib: str_append_[tab]unescaped(): Don't truncate input at NULs
For str_append_tabunescaped() the input is supposed to have NULs already
escaped though, so it was truncating only with invalid input.
Commit: c68782f4b1e3086cbadf533e6aae647c5e60fd00
https://github.com/dovecot/core/commit/c68782f4b1e3086cbadf533e6aae647c5e60fd00
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-fts/fts-tokenizer-address.c
Log Message:
-----------
lib-fts: fts-tokenizer-address - Cleanup NUL handling in parent_data
The end result is anyway that the token will be truncated at the first NUL.
This change just makes the code a bit more understandable on where the
truncation will happen.
Commit: b50b90941fc1f024b0a4e4fb4fed981fa6565726
https://github.com/dovecot/core/commit/b50b90941fc1f024b0a4e4fb4fed981fa6565726
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/config/doveconf.c
M src/doveadm/dsync/dsync-mailbox-tree-sync.c
M src/lib/failures.c
M src/lib/var-expand.c
Log Message:
-----------
global: Replace str_append_n() with str_append_max()
Commit: dd477f872229fb42715b0f9591b04df08a725775
https://github.com/dovecot/core/commit/dd477f872229fb42715b0f9591b04df08a725775
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib/compat.h
M src/lib/macros.h
Log Message:
-----------
lib: Add ATTR_DEPRECATED macro
Commit: d8449ed1b2327db1d02bf14b0f122479a26bf01c
https://github.com/dovecot/core/commit/d8449ed1b2327db1d02bf14b0f122479a26bf01c
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib/str.h
Log Message:
-----------
lib: Mark str_append_n() deprecated
It was almost always used wrongly. Most of the time str_append_data()
should have been used instead. str_append_max() has the old functionality.
Commit: 955384f916aafb6912b03154bb4ca7743c88daf0
https://github.com/dovecot/core/commit/955384f916aafb6912b03154bb4ca7743c88daf0
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-23 (Mon, 23 Apr 2018)
Changed paths:
M src/lib-storage/index/dbox-common/dbox-mail.c
M src/lib-storage/index/imapc/imapc-mail.c
M src/lib-storage/index/imapc/imapc-mailbox.c
M src/lib-storage/index/index-mail.c
M src/lib-storage/index/pop3c/pop3c-sync.c
Log Message:
-----------
lib-storage: Don't add trailing NUL to strings in dovecot.index.cache
They are unnecessary and just unnecessarily consume disk space. The
existing reading code works even if they don't exist, so this change won't
break backwards compatibility.
Compare: https://github.com/dovecot/core/compare/5c62b6eebed4...955384f916aa
More information about the dovecot-cvs
mailing list