[dovecot/core] 6dc129: lib-storage: Don't enforce creation of vsize heade...

Tue Jun 26 12:00:17 EEST 2018

  Branch: refs/heads/master-2.3
  Home:   https://github.com/dovecot/core
  Commit: 6dc1296d0b5bf462e9c3f3b901c84cdb4841178d
      https://github.com/dovecot/core/commit/6dc1296d0b5bf462e9c3f3b901c84cdb4841178d
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/index-mailbox-size.c

  Log Message:
  -----------
  lib-storage: Don't enforce creation of vsize header.

It's useful for caching quota=count and for STATUS (X-SIZE). If neither is
used the header shouldn't be created and kept up-to-date. If the header is
enabled, it also triggers creation of per-email vsize records.

  Commit: 764e86b8bc043bd5de5ba6a109b66d68987e3f64
      https://github.com/dovecot/core/commit/764e86b8bc043bd5de5ba6a109b66d68987e3f64
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/index-mail.c

  Log Message:
  -----------
  lib-storage: Don't set virtual_size from index record if it's already set

The existing virtual_size may be a correct one, while the one in the index
may be wrong. This will be an especially important fix with the next
commit, because it would make the existing inexact_total_sizes=FALSE wrong.

  Commit: 69b714836fa3c18a8645ef957a5d5e187b3ad072
      https://github.com/dovecot/core/commit/69b714836fa3c18a8645ef957a5d5e187b3ad072
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/index-mail.c

  Log Message:
  -----------
  lib-storage: Move code to index_mail_try_set_body_size()

This also adds a check to make sure virtual_size is set. With the current
caller it was already guaranteed to be set.

  Commit: 440645ada57adb8420490e025a55aff9610f574f
      https://github.com/dovecot/core/commit/440645ada57adb8420490e025a55aff9610f574f
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/imapc/imapc-mail-fetch.c
    M src/lib-storage/index/imapc/imapc-mail.c
    M src/lib-storage/index/index-mail.c
    M src/lib-storage/index/index-mail.h

  Log Message:
  -----------
  imapc: Don't use RFC822.SIZE values for message body size calculation

If the RFC822.SIZE doesn't match the exact stream size, the body size
calculation will become wrong. The only downside to this patch is that
now the body size will need to be calculated by parsing the stream, but
there shouldn't be any need to do that unless the body was already
FETCHed, so it shouldn't cause any extra IMAP traffic.

  Commit: 42625103ef3d2cddf3d840449fa6fcd5674e84fb
      https://github.com/dovecot/core/commit/42625103ef3d2cddf3d840449fa6fcd5674e84fb
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/index-mail.c

  Log Message:
  -----------
  lib-storage: Set index_mail_data.body_size if it can be calculated easily

Probably doesn't help with anything currently, but makes the code more
consistent in how it works.

  Commit: 7c58144cb5ad148102998cb63b8f7e8bf80b01bf
      https://github.com/dovecot/core/commit/7c58144cb5ad148102998cb63b8f7e8bf80b01bf
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/imapc/imapc-mailbox.c

  Log Message:
  -----------
  imapc: Move expunging from index to imapc_mailbox_index_expunge()

  Commit: 7a52620e098bfa61d74a45ab92cffbdb4031a8aa
      https://github.com/dovecot/core/commit/7a52620e098bfa61d74a45ab92cffbdb4031a8aa
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/imapc/imapc-mailbox.c

  Log Message:
  -----------
  imapc: Fix assert-crash when reconnecting expunges newly added messages

The initial FETCH 1:* during reconnection may notice some messages that are
missing and tries to expunge them. But those messages could still be in
delayed_sync_trans, and currently lib-index doesn't support expunging anything
but the latest mail from a transaction.

Fixes:
Panic: file mail-index-transaction-update.c: line 337 (mail_index_expunge_last_append): assertion failed: (seq == t->last_new_seq)

  Commit: 56b939c191f8598fd10105117759ce541c8723e0
      https://github.com/dovecot/core/commit/56b939c191f8598fd10105117759ce541c8723e0
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/imapc/imapc-storage.c

  Log Message:
  -----------
  imapc: Fix assert-crash when reconnecting during initial FETCH finds expunges

The initial FETCH is automatically retried on failure, but the FETCH state
wasn't reset during the reconnection.

Fixes:
Panic: file imapc-sync.c: line 328 (imapc_initial_sync_check): assertion failed: (mail_index_is_expunged(view, lseq) || seq_range_exists(&ctx->mbox->delayed_expunged_uids, luid))

  Commit: b352767f8a2dae9fe12c68f890a2d85ff7a5b28a
      https://github.com/dovecot/core/commit/b352767f8a2dae9fe12c68f890a2d85ff7a5b28a
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/imapc/imapc-storage.c
    M src/lib-storage/index/imapc/imapc-sync.c

  Log Message:
  -----------
  imapc: Fix potential assert-crash when reconnecting during syncing

Fixes:
Panic: file imapc-sync.c: line 328 (imapc_initial_sync_check): assertion failed: (mail_index_is_expunged(view, lseq) || seq_range_exists(&ctx->mbox->delayed_expunged_uids, luid))

  Commit: 6a20890a448caeabb08bd204d7ac1b143a28ed54
      https://github.com/dovecot/core/commit/6a20890a448caeabb08bd204d7ac1b143a28ed54
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/imapc/imapc-sync.c

  Log Message:
  -----------
  imapc: Fix assert-crash if syncing adds delayed expunges, but fails afterwards

Fixes:
Panic: file imapc-sync.c: line 328 (imapc_initial_sync_check): assertion failed: (mail_index_is_expunged(view, lseq) || seq_range_exists(&ctx->mbox->delayed_expunged_uids, luid))

  Commit: a4a1c842411bc26d3bd4d406bb8f64574a9ffe68
      https://github.com/dovecot/core/commit/a4a1c842411bc26d3bd4d406bb8f64574a9ffe68
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-imap-client/imapc-client.c

  Log Message:
  -----------
  lib-imap-client: Remove assert from imapc_client_mailbox_reconnect

It's possible for the assert to trigger if there are multiple reconnections.
imapc_connection_can_reconnect() checks a bit later whether the reconnection
can still be done.

  Commit: bb336da1693bb7ba8b6b1f818567a1e4901ca3c2
      https://github.com/dovecot/core/commit/bb336da1693bb7ba8b6b1f818567a1e4901ca3c2
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-storage/index/imapc/imapc-mail-fetch.c

  Log Message:
  -----------
  imapc: Don't trust mail stream to have correct CRLFs

It's not that expensive to just recalculate them. If they're not correct in
the stream, it can result in e.g. truncated emails due to wrong virtual_size
being used in body_size calculation. The wrong CRLFs can especially happen
if plugins modify the stream in istream_opened() hook.

  Commit: 68ba703aca7f32a1c76c21d71bc506b453c9b5c8
      https://github.com/dovecot/core/commit/68ba703aca7f32a1c76c21d71bc506b453c9b5c8
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2018-06-26 (Tue, 26 Jun 2018)

  Changed paths:
    M src/lib-imap-client/imapc-connection.c

  Log Message:
  -----------
  lib-imap-client: Fix crash when ostream output blocks before connection finishes

It's possible that ostream tries to add IO_WRITE callback before
imapc_connection_connected() callback is called.

Fixes:
Panic: io_add(0x2) called twice fd=15, callback=0x... -> 0x...

Compare: https://github.com/dovecot/core/compare/6eb02203a91d...68ba703aca7f
      **NOTE:** This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.