[dovecot/core] 7e41ef: lib: test-ostream-* - Call o_stream_nfinish() afte...

Mon Oct 30 13:30:10 EET 2017

  Branch: refs/heads/master
  Home:   https://github.com/dovecot/core
  Commit: 7e41ef20f6e39649f620fc4e4d21924822ee3682
      https://github.com/dovecot/core/commit/7e41ef20f6e39649f620fc4e4d21924822ee3682
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib/test-ostream-buffer.c
    M src/lib/test-ostream-file.c

  Log Message:
  -----------
  lib: test-ostream-* - Call o_stream_nfinish() after uncorking

  Commit: 23c8c511f98924acfc7048e9dc4ccc982c34e320
      https://github.com/dovecot/core/commit/23c8c511f98924acfc7048e9dc4ccc982c34e320
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib/ostream-file.c
    M src/lib/ostream.c
    M src/lib/ostream.h

  Log Message:
  -----------
  lib: o_stream_uncork() - call o_stream_nflush() instead of o_stream_flush()

Otherwise flush errors may be unintentionally ignored. This change requires
that all the o_stream_uncork() callers must call o_stream_nfinish()
afterwards, unless the ostream is set to ignore errors.

  Commit: f81141f5ade2c5c946852efbb4ec6412bf074949
      https://github.com/dovecot/core/commit/f81141f5ade2c5c946852efbb4ec6412bf074949
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/imap/cmd-append.c
    M src/imap/cmd-setmetadata.c
    M src/imap/imap-client.c
    M src/lib-http/http-server-connection.c
    M src/lib-index/mail-index-write.c

  Log Message:
  -----------
  global: Remove unnecessary o_stream_nflush() calls

The following o_stream_nfinish() or o_stream_uncork() calls it anyway.

  Commit: 414bbc6dfb81276908f130354890b143cbf1203c
      https://github.com/dovecot/core/commit/414bbc6dfb81276908f130354890b143cbf1203c
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib-imap/imap-parser.c

  Log Message:
  -----------
  lib-imap: When sending "+ OK" to client, uncork ostream immediately

This makes sure that it's sent to the client as soon as possible.

Also removed o_stream_nflush(), since uncorking does that already.
If there is no corking, the "+ OK" was already attempted to be sent
by the o_stream_nsend() call and a flush is unlikely to help.

  Commit: 401b336c846089939720908974de79c937d9e49f
      https://github.com/dovecot/core/commit/401b336c846089939720908974de79c937d9e49f
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/imap/cmd-copy.c

  Log Message:
  -----------
  imap: Make sure long COPY commands send "* OK Hang in there" packets

o_stream_nflush() already flushed the writes to kernel, and Linux should
have sent them after 200ms delay, so this probably doesn't fix anything.
Using uncork+cork is a bit clearer way to indicate that the data really
is wanted to be sent now.

  Commit: b688884f23be2333bea1a8bd4ee849336aebedaa
      https://github.com/dovecot/core/commit/b688884f23be2333bea1a8bd4ee849336aebedaa
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib/ostream.c
    M src/lib/ostream.h

  Log Message:
  -----------
  lib: Remove o_stream_nflush()

None of its previous callers were calling it correctly, so it's better to
remove it entirely to avoid more of such wrong calls.

  Commit: 34d5077c37dc6224a2d430a72ae51a3f38e9e4f6
      https://github.com/dovecot/core/commit/34d5077c37dc6224a2d430a72ae51a3f38e9e4f6
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib/ostream.c
    M src/lib/ostream.h

  Log Message:
  -----------
  lib: o_stream_flush() - check for send overflows and reset error handling

This begins the removal of o_stream_nfinish().

  Commit: 84717f00ddbf5cd0ebe3f285090d5e97e458e19c
      https://github.com/dovecot/core/commit/84717f00ddbf5cd0ebe3f285090d5e97e458e19c
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib/ostream-private.h
    M src/lib/ostream.c
    M src/lib/ostream.h

  Log Message:
  -----------
  lib: Add o_stream_finish()

This marks the ostream as fully finished, with no further writes coming to
it. Otherwise it behaves like o_stream_flush().

The ostream.flush() methods are supposed to write their trailer (if the
ostream has one) when they see that ostream.finished is set. These will be
implemented by the following commits.

  Commit: ad9afb64630511d5e25bc5bc11c5304986156928
      https://github.com/dovecot/core/commit/ad9afb64630511d5e25bc5bc11c5304986156928
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/doveadm/client-connection-http.c
    M src/doveadm/dsync/dsync-ibc-stream.c
    M src/lib-compression/test-compression.c
    M src/lib-dcrypt/test-stream.c
    M src/lib-dict/dict-file.c
    M src/lib-fs/fs-api.c
    M src/lib-index/mail-cache-compress.c
    M src/lib-index/mail-index-strmap.c
    M src/lib-index/mail-index-write.c
    M src/lib-mail/istream-attachment-extractor.c
    M src/lib-mail/test-istream-attachment.c
    M src/lib-master/master-instance.c
    M src/lib-program-client/test-program-client-net.c
    M src/lib-program-client/test-program-client-unix.c
    M src/lib-storage/index/cydir/cydir-save.c
    M src/lib-storage/index/dbox-common/dbox-save.c
    M src/lib-storage/index/dbox-single/sdbox-file.c
    M src/lib-storage/index/imapc/imapc-save.c
    M src/lib-storage/index/maildir/maildir-save.c
    M src/lib-storage/index/maildir/maildir-uidlist.c
    M src/lib-storage/list/subscription-file.c
    M src/lib-storage/mail-duplicate.c
    M src/lib/test-istream-multiplex.c
    M src/lib/test-multiplex.c
    M src/lib/test-ostream-buffer.c
    M src/lib/test-ostream-file.c
    M src/lib/test-ostream-multiplex.c
    M src/plugins/acl/acl-backend-vfile-acllist.c
    M src/plugins/acl/acl-backend-vfile-update.c
    M src/plugins/fts-squat/squat-trie.c
    M src/plugins/fts-squat/squat-uidlist.c
    M src/replication/replicator/replicator-queue.c
    M src/util/rawlog.c

  Log Message:
  -----------
  global: Replace o_stream_nfinish() with o_stream_finish() whenever possible

  Commit: d1ba8ecbb936ace90179d2292952546708d68f71
      https://github.com/dovecot/core/commit/d1ba8ecbb936ace90179d2292952546708d68f71
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/auth/auth-worker-client.c
    M src/lib-auth/auth-master.c
    M src/lib-fs/fs-api.h
    M src/lib-storage/index/dbox-common/dbox-file-fix.c
    M src/lib-storage/index/dbox-common/dbox-file.c
    M src/lib-storage/index/dbox-common/dbox-save.c
    M src/lib-storage/index/dbox-multi/mdbox-purge.c
    M src/lib-storage/index/mbox/mbox-save.c
    M src/lib/iostream-rawlog.c
    M src/lmtp/commands.c

  Log Message:
  -----------
  global: Replace o_stream_nfinish() with o_stream_flush()

  Commit: 4805c8a7d4bfcafa0282c31b2982757f852f8568
      https://github.com/dovecot/core/commit/4805c8a7d4bfcafa0282c31b2982757f852f8568
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib/ostream.h

  Log Message:
  -----------
  lib: Remove o_stream_nfinish()

  Commit: 6fb8641aa5619d0c0dcd7be7570767f9688a953d
      https://github.com/dovecot/core/commit/6fb8641aa5619d0c0dcd7be7570767f9688a953d
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib-compression/ostream-zlib.c
    M src/lib-compression/test-compression.c

  Log Message:
  -----------
  lib-compression: ostream-zlib - Require o_stream_finish() to write the trailer

It's now possible to call o_stream_flush() without the trailer getting
written.

It's also now required to call o_stream_finish() to write the trailer before
closing the ostream. This guarantees that the caller checks that the
trailer was successfully written. This isn't needed if the ostream already
failed before writing the trailer.

  Commit: 6630a5734436953a61ce6ec471d0d91a5f206083
      https://github.com/dovecot/core/commit/6630a5734436953a61ce6ec471d0d91a5f206083
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib-dcrypt/ostream-encrypt.c

  Log Message:
  -----------
  lib-dcrypt: ostream-dcrypt - Require o_stream_finish() to write the trailer

Similar to ostream-zlib.

  Commit: de535bbd17564819b51241b71dae796093cbbabb
      https://github.com/dovecot/core/commit/de535bbd17564819b51241b71dae796093cbbabb
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib/ostream-multiplex.c

  Log Message:
  -----------
  lib: ostream-multiplex - Don't close parent streams at destroy

The close() method is already called as needed.

  Commit: a2583397219ebbb877fcf84107f0ca7f56362760
      https://github.com/dovecot/core/commit/a2583397219ebbb877fcf84107f0ca7f56362760
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/doveadm/server-connection.c
    M src/lib-mail/ostream-dot.c
    M src/lib-mail/test-ostream-dot.c
    M src/lib-program-client/program-client.c
    M src/lib-program-client/test-program-client-net.c

  Log Message:
  -----------
  lib-mail: ostream-dot - Require o_stream_finish() to write the trailer

Similar to ostream-zlib.

  Commit: ec2c310074340f3020c9ef2b9ab3bfd207e3c31d
      https://github.com/dovecot/core/commit/ec2c310074340f3020c9ef2b9ab3bfd207e3c31d
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/plugins/mail-filter/ostream-ext-filter.c

  Log Message:
  -----------
  mail-filter: ostream-ext-filter - Require o_stream_finish() to be called

Similar to ostream-zlib.

  Commit: 5ec4fc44e8d4e2160f07b1a7f4fce1ccfec3f6c1
      https://github.com/dovecot/core/commit/5ec4fc44e8d4e2160f07b1a7f4fce1ccfec3f6c1
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib/ostream.c
    M src/lib/ostream.h

  Log Message:
  -----------
  lib: Automatically flush ostream when it's closed

Also verify that it unexpectedly didn't get fully flushed.

  Commit: 5bc75087b6feea54307be14ebbf009f8e9a8dd13
      https://github.com/dovecot/core/commit/5bc75087b6feea54307be14ebbf009f8e9a8dd13
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2017-10-30 (Mon, 30 Oct 2017)

  Changed paths:
    M src/lib-compression/ostream-bzlib.c
    M src/lib-compression/ostream-lz4.c
    M src/lib-compression/ostream-lzma.c
    M src/lib-fs/ostream-cmp.c
    M src/lib/ostream-file.c
    M src/lib/ostream-multiplex.c
    M src/lib/ostream-rawlog.c

  Log Message:
  -----------
  global: Remove explicit o_stream_flush() calls from ostream.close() methods

It's now being done automatically.

Compare: https://github.com/dovecot/core/compare/2fc183541ffd...5bc75087b6fe