[dovecot/core] 013680: fts: Fix indexing input that contains NULs
GitHub
noreply at github.com
Thu Apr 26 13:00:19 EEST 2018
Branch: refs/heads/master
Home: https://github.com/dovecot/core
Commit: 013680e738a89037ac07dd87aa727f440c18bc26
https://github.com/dovecot/core/commit/013680e738a89037ac07dd87aa727f440c18bc26
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-26 (Thu, 26 Apr 2018)
Changed paths:
M src/plugins/fts/fts-parser.c
Log Message:
-----------
fts: Fix indexing input that contains NULs
Any message_block that contained NULs, but otherwise was valid UTF-8, was
simply dropped.
Commit: dd1efd73534f0859eef667b56f51b33f9b6330b9
https://github.com/dovecot/core/commit/dd1efd73534f0859eef667b56f51b33f9b6330b9
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-26 (Thu, 26 Apr 2018)
Changed paths:
M src/lib-fts/test-fts-tokenizer.c
Log Message:
-----------
lib-fts: Minor fix to randomness test in test-fts-tokenizer
If the random input was entirely valid UTF-8, the input was truncated to
empty.
Commit: 39163816936b1ff4c369952b2132dd53634aa8d7
https://github.com/dovecot/core/commit/39163816936b1ff4c369952b2132dd53634aa8d7
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-26 (Thu, 26 Apr 2018)
Changed paths:
M src/lib-storage/list/mailbox-list-fs-iter.c
M src/lib-storage/list/mailbox-list-maildir-iter.c
Log Message:
-----------
lib-storage: Don't ignore uni_utf8_get_valid_data() return value
It's not supposed to happen, so call i_unreached()
Commit: 135935671914a099ec4279b7480805b72a96cac7
https://github.com/dovecot/core/commit/135935671914a099ec4279b7480805b72a96cac7
Author: Timo Sirainen <timo.sirainen at dovecot.fi>
Date: 2018-04-26 (Thu, 26 Apr 2018)
Changed paths:
M src/lib/unichar.h
Log Message:
-----------
lib: uni_utf8_get_valid_data() - Add ATTR_WARN_UNUSED_RESULT
Ignoring the return value most likely leads to buggy code, so make sure it's
checked.
Commit: dff3bf002d29ddd284e56b1293c516c211456802
https://github.com/dovecot/core/commit/dff3bf002d29ddd284e56b1293c516c211456802
Author: Phil Carmody <phil at dovecot.fi>
Date: 2018-04-26 (Thu, 26 Apr 2018)
Changed paths:
M src/lib/strfuncs.c
M src/lib/strfuncs.h
M src/lib/test-strfuncs.c
Log Message:
-----------
lib: strfuncs - string match length and prefix checking helpers
strncmp(input, "literal", 7) is an idiom used everywhere, but leaves
room for human error in calculating the length.
strncmp(input, "literal", strlen("literal")) is an idiom also used
everywhere, but is both verbose and might be inefficient on some
legacy or ultralightweight compilers.
The old techniques are presumed to be optimal code-wise, but are
verbose (and, containing redundancy, they leave room for human error),
so make the macro fall back onto this operation, simply avoiding the
redundancy/verbosity.
The macro expansion does not multiply evaluate any of its parameters,
so should be safe even in the strangest of situations.
Signed-off-by: Phil Carmody <phil at dovecot.fi>
Commit: e4c57ac2d23db47943b1159ab0f7691498ba9c45
https://github.com/dovecot/core/commit/e4c57ac2d23db47943b1159ab0f7691498ba9c45
Author: Phil Carmody <phil at dovecot.fi>
Date: 2018-04-26 (Thu, 26 Apr 2018)
Changed paths:
M src/auth/auth-client-connection.c
M src/auth/auth-master-connection.c
M src/auth/auth-request.c
M src/auth/auth-worker-server.c
M src/auth/db-ldap.c
M src/auth/db-oauth2.c
M src/auth/main.c
M src/auth/mech-oauth2.c
M src/auth/mech-otp.c
M src/auth/passdb-blocking.c
M src/auth/passdb-bsdauth.c
M src/auth/passdb-checkpassword.c
M src/auth/passdb-lua.c
M src/auth/passdb-pam.c
M src/auth/passdb-vpopmail.c
M src/auth/password-scheme-crypt.c
M src/auth/password-scheme.c
M src/auth/userdb-blocking.c
M src/auth/userdb-checkpassword.c
M src/auth/userdb-lua.c
M src/auth/userdb-passwd-file.c
M src/auth/userdb-vpopmail.c
M src/config/config-connection.c
M src/config/doveconf.c
M src/config/old-set-parser.c
M src/config/sysinfo-get.c
M src/director/login-connection.c
M src/dns/dns-client.c
M src/doveadm/client-connection-tcp.c
M src/doveadm/doveadm-auth-server.c
M src/doveadm/doveadm-auth.c
M src/doveadm/doveadm-dsync.c
M src/doveadm/doveadm-dump-dbox.c
M src/doveadm/doveadm-mail-fetch.c
M src/doveadm/doveadm-mail-mailbox-metadata.c
M src/doveadm/doveadm-mail-server.c
M src/doveadm/doveadm-oldstats.c
M src/doveadm/doveadm-proxy.c
M src/doveadm/doveadm-util.c
M src/doveadm/doveadm-zlib.c
M src/doveadm/server-connection.c
M src/imap-hibernate/imap-client.c
M src/imap-login/imap-proxy.c
M src/imap-urlauth/imap-urlauth.c
M src/imap/cmd-notify.c
M src/imap/imap-fetch-body.c
M src/imap/imap-sync.c
M src/lib-auth/auth-client-request.c
M src/lib-auth/auth-master.c
M src/lib-auth/auth-server-connection.c
M src/lib-dcrypt/dcrypt-openssl.c
M src/lib-dict-backend/dict-sql-settings.c
M src/lib-dict/dict-client.c
M src/lib-dict/dict-memcached-ascii.c
M src/lib-dict/dict-memcached.c
M src/lib-dict/dict-redis.c
M src/lib-dns/dns-lookup.c
M src/lib-fs/fs-posix.c
M src/lib-imap-storage/imap-msgpart.c
M src/lib-imap/imap-base-subject.c
M src/lib-index/mail-index.c
M src/lib-mail/test-message-header-encode.c
M src/lib-master/master-login-auth.c
M src/lib-master/master-login.c
M src/lib-master/master-service-settings-cache.c
M src/lib-master/master-service-settings.c
M src/lib-master/master-service.c
M src/lib-program-client/program-client.c
M src/lib-settings/settings-parser.c
M src/lib-smtp/test-smtp-client-errors.c
M src/lib-sql/driver-pgsql.c
M src/lib-storage/index/imapc/imapc-mail-fetch.c
M src/lib-storage/index/index-status.c
M src/lib-storage/index/index-storage.c
M src/lib-storage/index/maildir/maildir-sync-index.c
M src/lib-storage/index/mbox/mbox-save.c
M src/lib-storage/index/pop3c/pop3c-mail.c
M src/lib-storage/index/pop3c/pop3c-storage.c
M src/lib-storage/list/mailbox-list-fs-flags.c
M src/lib-storage/list/mailbox-list-maildir-iter.c
M src/lib-storage/mail-namespace.c
M src/lib-storage/mail-search-register-imap.c
M src/lib-storage/mail-storage-hooks.c
M src/lib-storage/mail-storage-service.c
M src/lib-storage/mail-storage-settings.c
M src/lib-storage/mail-storage.c
M src/lib-storage/mailbox-list.c
M src/lib-storage/test-mail-storage.c
M src/lib/iostream-rawlog.c
M src/lib/module-dir.c
M src/lib/test-str-sanitize.c
M src/lib/unlink-directory.c
M src/log/log-connection.c
M src/login-common/client-common-auth.c
M src/login-common/sasl-server.c
M src/master/main.c
M src/old-stats/client-export.c
M src/old-stats/mail-session.c
M src/plugins/acl/acl-backend-vfile.c
M src/plugins/acl/acl-backend.c
M src/plugins/fs-compress/fs-compress.c
M src/plugins/fts-lucene/fts-lucene-plugin.c
M src/plugins/fts-solr/fts-solr-plugin.c
M src/plugins/fts-squat/fts-backend-squat.c
M src/plugins/fts-squat/squat-test.c
M src/plugins/fts/fts-build-mail.c
M src/plugins/fts/fts-indexer.c
M src/plugins/old-stats/mail-stats-fill.c
M src/plugins/quota/quota-status.c
M src/plugins/quota/quota-util.c
M src/plugins/quota/quota.c
M src/pop3-login/pop3-proxy.c
M src/util/script.c
Log Message:
-----------
global - migrate strncmp literals to str_begins
Simplify a bunch of strncmp(,,number) calls.
git ls-files \*.[ch] | xargs perl -p -i -e 's/strncmp\((.*?), ?(\".*?\"), ?(\d+)\) == 0/str_begins($1, $2)/g'
git ls-files \*.[ch] | xargs perl -p -i -e 's/strncmp\((.*?), ?(\".*?\"), ?(\d+)\) != 0/!str_begins($1, $2)/g'
I ran a longer script to verify that all of the string literals and
the length matched. They didn't:
$ git grep strncmp | perl -ne 'print if(m/strncmp\([^,]*,\s*"(.*?)",\s*(\d+)/ and ($s=$1,$t=$2,$s=~s/\\[tn]/#/g,length($s)) != $t)'
src/auth/db-oauth2.c: if (strncmp(field, "oauth2:", 8) == 0 &&
With the new functions, that kind of typo is impossible.
Signed-off-by: Phil Carmody <phil at dovecot.fi>
Commit: c05976729405180c565aad464cb494c731a640b3
https://github.com/dovecot/core/commit/c05976729405180c565aad464cb494c731a640b3
Author: Phil Carmody <phil at dovecot.fi>
Date: 2018-04-26 (Thu, 26 Apr 2018)
Changed paths:
M src/doveadm/doveadm-dict.c
M src/imap-hibernate/imap-client.c
M src/imap-login/imap-proxy.c
M src/lib-dict-backend/dict-cdb.c
M src/lib-dict-extra/dict-fs.c
M src/lib-dict/dict-memcached-ascii.c
M src/lib-dict/dict-memcached.c
M src/lib-dict/dict-redis.c
M src/lib-dict/dict.c
M src/lib-fs/fs-metawrap.c
M src/lib-fs/fs-posix.c
M src/lib-imap-storage/imap-metadata.c
M src/lib-storage/index/dbox-multi/mdbox-purge.c
M src/lib-storage/index/dbox-single/sdbox-sync-rebuild.c
M src/lib-storage/index/index-search-mime.c
M src/lib-storage/index/index-storage.c
M src/lib-storage/index/mbox/mbox-storage.c
M src/lib-storage/list/mailbox-list-delete.c
M src/lib-storage/list/mailbox-list-fs.c
M src/lib-storage/mail-storage-service.c
M src/lib-storage/mailbox-attribute.c
M src/lib-storage/mailbox-attribute.h
M src/lib-storage/mailbox-list.c
M src/lib/mempool-alloconly.c
M src/lib/test-strfuncs.c
M src/master/main.c
M src/plugins/acl/acl-api.c
M src/plugins/acl/acl-attributes.c
M src/plugins/imap-acl/imap-acl-plugin.c
Log Message:
-----------
global - migrate from strncmp to str_begins
Simplify a bunch of verbose strncmp(,,strlen()) calls.
--- 8< --- strncmp.cocci ---
@@
expression e1, e2;
@@
- strncmp(e1, e2, strlen(e2)) == 0
+ str_begins(e1, e2)
@@
expression e1, e2;
@@
- strncmp(e1, e2, strlen(e2)) != 0
+ !str_begins(e1, e2)
@@
expression e1, e2;
@@
- strncmp(e1, e2, strlen(e1)) == 0
+ str_begins(e2, e1)
@@
expression e1, e2;
@@
- strncmp(e1, e2, strlen(e1)) != 0
+ !str_begins(e2, e1)
--- 8< ---------------------
Signed-off-by: Phil Carmody <phil at dovecot.fi>
Commit: 5cef037a24b7c2d9b28a322d202833a65b31bbbf
https://github.com/dovecot/core/commit/5cef037a24b7c2d9b28a322d202833a65b31bbbf
Author: Phil Carmody <phil at dovecot.fi>
Date: 2018-04-26 (Thu, 26 Apr 2018)
Changed paths:
M src/auth/auth-cache.c
M src/config/old-set-parser.c
M src/doveadm/doveadm-cmd.c
M src/doveadm/doveadm-mail.c
M src/doveadm/dsync/dsync-brain-mailbox-tree.c
M src/imap/cmd-notify.c
M src/imap/cmd-rename.c
M src/imap/imap-notify.c
M src/lib-storage/index/imapc/imapc-list.c
M src/lib-storage/index/imapc/imapc-storage.c
M src/lib-storage/index/index-search-mime.c
M src/lib-storage/index/shared/shared-list.c
M src/lib-storage/list/mailbox-list-fs-iter.c
M src/lib-storage/list/mailbox-list-index-backend.c
M src/lib-storage/list/mailbox-list-iter.c
M src/lib-storage/mailbox-list.c
Log Message:
-----------
global - migrate more complicated strncmp expressions to str_begins
Simplify a bunch of verbose var=strlen(); strncmp(,,var) calls.
Fortunately, all of these examples use the length variable, so no
"unused value" warnings occur.
--- 8< --- strncmp2.cocci ---
@@
expression e1, e2;
identifier i2;
@@
i2 = strlen(e2)
...
- strncmp(e1, e2, i2) == 0
+ str_begins(e1, e2)
@@
expression e1, e2;
identifier i2;
@@
i2 = strlen(e2)
...
- strncmp(e1, e2, i2) != 0
+ !str_begins(e1, e2)
@@
expression e1, e2;
identifier i1;
@@
i1 = strlen(e1)
...
- strncmp(e1, e2, i1) == 0
+ str_begins(e2, e1)
@@
expression e1, e2;
identifier i1;
@@
i1 = strlen(e1)
...
- strncmp(e1, e2, i1) != 0
+ !str_begins(e2, e1)
--- 8< ---------------------
Signed-off-by: Phil Carmody <phil at dovecot.fi>
Compare: https://github.com/dovecot/core/compare/4bda91b76480...5cef037a24b7
More information about the dovecot-cvs
mailing list