[dovecot/core] a26745: lib-http: client: Merge max_idle_time_msecs settin...

Mon Mar 12 10:00:39 EET 2018

  Branch: refs/heads/master-2.3
  Home:   https://github.com/dovecot/core
  Commit: a267451237ba3b14d5fc54efa36ca325236edb3b
      https://github.com/dovecot/core/commit/a267451237ba3b14d5fc54efa36ca325236edb3b
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client.c

  Log Message:
  -----------
  lib-http: client: Merge max_idle_time_msecs setting with setting from shared context.

While using the default global context, per-client max_idle_time_msecs setting was ignored.

  Commit: f43229fe8828577f46ca61b35a8594ff578c8251
      https://github.com/dovecot/core/commit/f43229fe8828577f46ca61b35a8594ff578c8251
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client.c

  Log Message:
  -----------
  lib-http: client: Merge socket buffer size settings with settings from shared context.

While using the default global context, per-client socket buffer size settings were ignored.

  Commit: 8570767d0e67bc257dd7587852385e0ce113fecd
      https://github.com/dovecot/core/commit/8570767d0e67bc257dd7587852385e0ce113fecd
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client-request.c

  Log Message:
  -----------
  lib-http: client: Set req->client to NULL once the request is destroyed.

The http_client_request_destroy() function does not free the request
immediately, as long as it is still referenced. It can still be referenced by a
connection that has sent it and is waiting for a reply (payload). In the mean
time the actual client can be gone, so we want to make sure nothing is pointing
to that anymore.

This change adds a few extra assertions to make sure nothing tries to use a
NULL client later on. Some direct references to req->client are replaced with a
local client variable if there is one.

  Commit: 96215d67d9988c71e03e5aa795946241a1bbe8a5
      https://github.com/dovecot/core/commit/96215d67d9988c71e03e5aa795946241a1bbe8a5
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client-peer.c
    M src/lib-http/http-client.c

  Log Message:
  -----------
  lib-http: client: Call switching ioloop for shared peer from shared client context.

Before, it was called from the client peer.

  Commit: 4307246c8b26a081aaeb4c68b585d77517da4ff7
      https://github.com/dovecot/core/commit/4307246c8b26a081aaeb4c68b585d77517da4ff7
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client-private.h
    M src/lib-http/http-client.c

  Log Message:
  -----------
  lib-http: client: Register client objects in the associated client shared context.

  Commit: 56c77bb544ea72a7322029329db0621e6990df55
      https://github.com/dovecot/core/commit/56c77bb544ea72a7322029329db0621e6990df55
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client.c

  Log Message:
  -----------
  lib-http: client: Base forcing shared context debug logging on the settings of the context itself and each associated client.

  Commit: 39b0a7bb13bc3467df5643920f90b9737616e2e2
      https://github.com/dovecot/core/commit/39b0a7bb13bc3467df5643920f90b9737616e2e2
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client-host.c
    M src/lib-http/http-client-private.h
    M src/lib-http/http-client.c

  Log Message:
  -----------
  lib-http: client: Use merged DNS settings from all clients connected to a shared context for DNS lookups.

  Commit: 08e718826a63de4336928cdfbba2512fb7e39692
      https://github.com/dovecot/core/commit/08e718826a63de4336928cdfbba2512fb7e39692
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client.c

  Log Message:
  -----------
  lib-http: client: Implicitly switch ioloop for global shared client context.

If the new ioloop is NULL, close the shared client context. In that case all
shared hosts, peers and idle connetions are closed and destroyed. The context
just remains as a container for settings and shared context. This is only
allowed when there are no clients left.

  Commit: 46f463558f736abc14d2efd95c7cac1dd233e35e
      https://github.com/dovecot/core/commit/46f463558f736abc14d2efd95c7cac1dd233e35e
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/test-http-payload.c

  Log Message:
  -----------
  lib-http: test-http-payload: Added test with clients using shared global context.

  Commit: 995c1977859080cde1208073eae46c18a7a46d07
      https://github.com/dovecot/core/commit/995c1977859080cde1208073eae46c18a7a46d07
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/test-http-client.c

  Log Message:
  -----------
  lib-http: test-http-client: Free the ssl_iostream_context cache.

  Commit: f6ab416bbddf399c0e15faaa57c338dc47c57d5d
      https://github.com/dovecot/core/commit/f6ab416bbddf399c0e15faaa57c338dc47c57d5d
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib/ioloop.c
    M src/lib/ioloop.h

  Log Message:
  -----------
  lib: ioloop: Add functions for adding/moving timeouts and ios to a specific ioloop.

  Commit: 0cb6dc4e66ff336c16164d5504e50f9677b78df0
      https://github.com/dovecot/core/commit/0cb6dc4e66ff336c16164d5504e50f9677b78df0
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib/iostream-private.h
    M src/lib/iostream.c
    M src/lib/istream.c
    M src/lib/ostream.c

  Log Message:
  -----------
  lib: iostream: Record the ioloop that the iostream was last switched to.

  Commit: e364ae8bdbd885fd3262a79d78af5d80689c3ce2
      https://github.com/dovecot/core/commit/e364ae8bdbd885fd3262a79d78af5d80689c3ce2
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-server-request.c
    M src/lib/istream-multiplex.c
    M src/lib/istream-private.h
    M src/lib/istream-timeout.c
    M src/lib/istream.c
    M src/lib/istream.h

  Log Message:
  -----------
  lib: istream: Allow switching to a specific ioloop.

  Commit: 860ba28cf9f844f061e5116d1ff047268aa86d8e
      https://github.com/dovecot/core/commit/860ba28cf9f844f061e5116d1ff047268aa86d8e
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-ssl-iostream/ostream-openssl.c
    M src/lib/ostream-file.c
    M src/lib/ostream-private.h
    M src/lib/ostream.c
    M src/lib/ostream.h

  Log Message:
  -----------
  lib: ostream: Allow switching to a specific ioloop.

  Commit: 43b57457cc6b10b2d0b699b0730b4bed3b3bf954
      https://github.com/dovecot/core/commit/43b57457cc6b10b2d0b699b0730b4bed3b3bf954
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib/connection.c
    M src/lib/connection.h

  Log Message:
  -----------
  lib: connection: Allow switching to a specific ioloop.

  Commit: 083d7e26be5e196ae6ae67e53d485b0e35789728
      https://github.com/dovecot/core/commit/083d7e26be5e196ae6ae67e53d485b0e35789728
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-dns/dns-lookup.c
    M src/lib-dns/dns-lookup.h

  Log Message:
  -----------
  lib-dns: Allow setting the ioloop that the dns_lookup/dns_client is started on.

  Commit: 080415ce425d9ca3019bbb5c9ee88fb0323b948f
      https://github.com/dovecot/core/commit/080415ce425d9ca3019bbb5c9ee88fb0323b948f
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client-connection.c
    M src/lib-http/http-client-host.c
    M src/lib-http/http-client-peer.c
    M src/lib-http/http-client-private.h
    M src/lib-http/http-client-queue.c
    M src/lib-http/http-client-request.c
    M src/lib-http/http-client.c

  Log Message:
  -----------
  lib-http: client: Make sure all ioloop objects are created on the ioloop that the client/context is switched to.

  Commit: 76f7e777d2a141c2337b6401b5b607c8df546599
      https://github.com/dovecot/core/commit/76f7e777d2a141c2337b6401b5b607c8df546599
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-ssl-iostream/iostream-openssl.c

  Log Message:
  -----------
  lib-ssl-iostream: openssl: Make verbose logging robust against i_debug() writing to stream itself.

In dsync, i_debug() is overridden to write to the SSL stream itself through a
multiplexed data stream. So, during the i_debug() call all kinds of things can
happen to the persisted error string in the stream, which caused problems.

  Commit: 648a82367132cc9b05208ced9c828ff2bf6339e0
      https://github.com/dovecot/core/commit/648a82367132cc9b05208ced9c828ff2bf6339e0
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/doveadm/doveadm-dsync.c

  Log Message:
  -----------
  doveadm: dsync: Switch ioloop for input/output streams while making TCP connection.

This task is performed in a sub-ioloop, and when returning from that ioloop, the
output stream would sometimes still have an object on the sub-ioloop that was
just destroyed.

  Commit: 7372d2b2ff4d6b95aaa3fac16cf92926595bea1a
      https://github.com/dovecot/core/commit/7372d2b2ff4d6b95aaa3fac16cf92926595bea1a
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client-connection.c

  Log Message:
  -----------
  lib-http: client: Move connection to shared context's ioloop, rather than the current_ioloop.

This is an omission.

  Commit: 241d9352afcb87644da86381e56844db2b5b7dc0
      https://github.com/dovecot/core/commit/241d9352afcb87644da86381e56844db2b5b7dc0
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib/connection.c
    M src/lib/connection.h

  Log Message:
  -----------
  lib: connection: Record the ioloop the connection was last switched to.

  Commit: de7ac2ee821957eeea8e45fd15dd6701f8e4f6c5
      https://github.com/dovecot/core/commit/de7ac2ee821957eeea8e45fd15dd6701f8e4f6c5
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client-connection.c

  Log Message:
  -----------
  lib-http: client: Use conn->conn.ioloop rather than cctx->ioloop in connection.

This leads to somewhat simpler code, and makes having connections on different
ioloops easier in the future.

  Commit: 99ef56d405df488048d3b6c7d26984ed4bf0ebe3
      https://github.com/dovecot/core/commit/99ef56d405df488048d3b6c7d26984ed4bf0ebe3
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-smtp/smtp-client-private.h
    M src/lib-smtp/smtp-client.h

  Log Message:
  -----------
  lib-smtp: client: Moved smtp_client_init_ssl_ctx() from smtp-client.h to smtp-client-private.h.

  Commit: f1d116e1f5f5a7ca5be7874db8ace5110602aec8
      https://github.com/dovecot/core/commit/f1d116e1f5f5a7ca5be7874db8ace5110602aec8
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib/connection.c
    M src/lib/connection.h

  Log Message:
  -----------
  lib: connection: Add connection_input_halt() and connection_input_resume().

These are convenience functions that remove and add conn->io respectively.

  Commit: 7b652dc1e60bbc591f479aa44b1da2b2fb85ac6b
      https://github.com/dovecot/core/commit/7b652dc1e60bbc591f479aa44b1da2b2fb85ac6b
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-ssl-iostream/ostream-openssl.c

  Log Message:
  -----------
  lib-ssl-iostream: ostream-openssl: Create local variable for sstream->ssl_io->plain_output in o_stream_ssl_flush().

  Commit: 6d4e1c7ab69247faf3bf270707ad8482abc5978a
      https://github.com/dovecot/core/commit/6d4e1c7ab69247faf3bf270707ad8482abc5978a
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-ssl-iostream/ostream-openssl.c

  Log Message:
  -----------
  lib-ssl-iostream: ostream-openssl: Fix behavior of o_stream_flush() so that 1 is only returned when buffer is empty.

  Commit: 758b6496f4dcddffc3c6a0e3f13e7ed698200ed4
      https://github.com/dovecot/core/commit/758b6496f4dcddffc3c6a0e3f13e7ed698200ed4
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-smtp/smtp-client-connection.c

  Log Message:
  -----------
  lib-smtp: client: Use the new connection_input_halt/resume() functions.

  Commit: c786a720ca40375b5fbe0c1842425cd803a19ff0
      https://github.com/dovecot/core/commit/c786a720ca40375b5fbe0c1842425cd803a19ff0
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-smtp/smtp-server-connection.c

  Log Message:
  -----------
  lib-smtp: server: Use the new connection_input_halt/resume() functions.

  Commit: a0b53dc859a8b1f4d2d0688c46f4d1d6e606688c
      https://github.com/dovecot/core/commit/a0b53dc859a8b1f4d2d0688c46f4d1d6e606688c
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client-connection.c

  Log Message:
  -----------
  lib-http: client: Use the new connection_input_halt/resume() functions.

  Commit: 8df7b1fbc7c9e1f2e8157c98db72087477ae7e4b
      https://github.com/dovecot/core/commit/8df7b1fbc7c9e1f2e8157c98db72087477ae7e4b
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-server-connection.c

  Log Message:
  -----------
  lib-http: server: Use the new connection_input_halt/resume() functions.

  Commit: 4a17af3f7df664795c3aa86dd39de0554eba53e9
      https://github.com/dovecot/core/commit/4a17af3f7df664795c3aa86dd39de0554eba53e9
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-smtp/smtp-server-connection.c

  Log Message:
  -----------
  lib-smtp: server: Recreate connection IO after streams change.

  Commit: eb35204a8437870a70cb0110e01ef809bb54ccee
      https://github.com/dovecot/core/commit/eb35204a8437870a70cb0110e01ef809bb54ccee
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-client-connection.c

  Log Message:
  -----------
  lib-http: client: Recreate connection IO after streams change.

  Commit: 9d91b7ed5117a1206c45b1b4a48cb4d1cf9eb046
      https://github.com/dovecot/core/commit/9d91b7ed5117a1206c45b1b4a48cb4d1cf9eb046
  Author: Stephan Bosch <stephan.bosch at dovecot.fi>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib-http/http-server-connection.c

  Log Message:
  -----------
  lib-http: server: Recreate connection IO after streams change.

  Commit: b3cd198c557fe12295c5104f2d49d5fc09305b59
      https://github.com/dovecot/core/commit/b3cd198c557fe12295c5104f2d49d5fc09305b59
  Author: Sergey Kitov <sergey.kitov at open-xchange.com>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib/array.h

  Log Message:
  -----------
  lib: array_idx_get_space is defined as alias to array_idx_modifiable

  Commit: 053041e713e7a8f9cf65271bdd540c07a0c3bc9f
      https://github.com/dovecot/core/commit/053041e713e7a8f9cf65271bdd540c07a0c3bc9f
  Author: Sergey Kitov <sergey.kitov at open-xchange.com>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/auth/main.c
    M src/lib-index/mail-cache-transaction.c
    M src/lib-index/mail-index-modseq.c
    M src/lib-index/mail-index-sync-ext.c
    M src/lib-index/mail-index-transaction-update.c
    M src/lib-storage/index/index-thread-finish.c
    M src/lib-storage/index/index-thread-links.c
    M src/lib-storage/index/maildir/maildir-keywords.c
    M src/lib/hash2.c
    M src/lib/ioloop-epoll.c
    M src/lib/module-context.h

  Log Message:
  -----------
  global: Change calls to array_idx_modifiable to array_idx_get_space.

Only calls that require space allocation are to be changed.

  Commit: 0fe4a1ddfd8bc590505a44d838654ed5f976b314
      https://github.com/dovecot/core/commit/0fe4a1ddfd8bc590505a44d838654ed5f976b314
  Author: Sergey Kitov <sergey.kitov at open-xchange.com>
  Date:   2018-03-12 (Mon, 12 Mar 2018)

  Changed paths:
    M src/lib/array.c
    M src/lib/array.h

  Log Message:
  -----------
  lib: array_idx_modifiable changed not to allocate space.

Compare: https://github.com/dovecot/core/compare/ff5d02182573...0fe4a1ddfd8b