[dovecot/core] 3c3001: director: Code cleanup - allow commands handlers t...

Tue Oct 25 18:00:13 UTC 2016

  Branch: refs/heads/master
  Home:   https://github.com/dovecot/core
  Commit: 3c3001681da75afc68578a180ec8f8b2d6dfacfa
      https://github.com/dovecot/core/commit/3c3001681da75afc68578a180ec8f8b2d6dfacfa
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/doveadm-connection.c

  Log Message:
  -----------
  director: Code cleanup - allow commands handlers to stop processing input

Now returning 0 exits the input reading without disconnecting client.

  Commit: ffedd89747e11373c0dd3a172d3e94e0cf724fd7
      https://github.com/dovecot/core/commit/ffedd89747e11373c0dd3a172d3e94e0cf724fd7
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director.c
    M src/director/director.h
    M src/director/main.c

  Log Message:
  -----------
  director: Call state_change_callback on user move timeout.

  Commit: 3751234328db786e53680f4df21a4d10b446e252
      https://github.com/dovecot/core/commit/3751234328db786e53680f4df21a4d10b446e252
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director.c

  Log Message:
  -----------
  director: Code cleanup - Finish user move always in the same function.

  Commit: d3bae1f9d2448e5c398145ea250849ec12583845
      https://github.com/dovecot/core/commit/d3bae1f9d2448e5c398145ea250849ec12583845
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director.c
    M src/director/director.h
    M src/director/main.c

  Log Message:
  -----------
  director: Show in process title how many users are being moved.

  Commit: 9ba1a38e2fa5ffe9d0db83c4a14a6552bcff3181
      https://github.com/dovecot/core/commit/9ba1a38e2fa5ffe9d0db83c4a14a6552bcff3181
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/doveadm-connection.c

  Log Message:
  -----------
  director: Code cleanup - moved command handling to its own function.

  Commit: 49ec317299dedfbc27be79ab97d5b7a8ba7a4eac
      https://github.com/dovecot/core/commit/49ec317299dedfbc27be79ab97d5b7a8ba7a4eac
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/doveadm-connection.c

  Log Message:
  -----------
  director: Make sure IP address parsing works in DIRECTOR-ADD/REMOVE

We were passing the entire string through to net_addr2ip(). It seems that
inet_aton() stops at whitespace though, so this wasn't actually causing
errors at least on Linux.

  Commit: 60835f193436504acd13acc09928c4b4ff6934d8
      https://github.com/dovecot/core/commit/60835f193436504acd13acc09928c4b4ff6934d8
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/doveadm-connection.c

  Log Message:
  -----------
  director: USER-MOVE didn't handle non-hash username correctly.

The entire command line was being hashed instead of just the username.

  Commit: 892e25e1c0caad62ced087d9eba2741a59e3d9ce
      https://github.com/dovecot/core/commit/892e25e1c0caad62ced087d9eba2741a59e3d9ce
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/doveadm-connection.c

  Log Message:
  -----------
  director: Code cleanup - Give tab-unescaped args to all doveadm command handlers.

  Commit: 67cb14c7fb54a031818228522dc7255d5cd00f0a
      https://github.com/dovecot/core/commit/67cb14c7fb54a031818228522dc7255d5cd00f0a
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director-connection.c
    M src/director/director-host.h

  Log Message:
  -----------
  director: If SYNCs are received during handshake, send them later.

This fixes delays during handshake:
Error: Ring SYNC appears to have got lost, resending

  Commit: a9ade104616bbb81c34cc6f8bfde5dab0571afac
      https://github.com/dovecot/core/commit/a9ade104616bbb81c34cc6f8bfde5dab0571afac
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director-connection.c

  Log Message:
  -----------
  director: Fix sending up/down state in handshakes.

They were never sent, because HOSTs were sent before director had waited for
the remote to send its version number. So sender thought that the remote's
minor_version was too old and it didn't send the up/down state at all.

This caused errors like:

Warning: director(10.0.0.30:9090/left): Host 10.0.0.30 is being updated before previous update had finished (up -> down) - setting to state=down vhosts=100
Error: director(10.0.0.30:9090/left): Director 10.0.0.30 SYNC request hosts don't match us - resending hosts (seq=6, remote hosts_hash=262126213, my hosts_hash=2458934259)

  Commit: 3d76809be2f9f3b5b4110e58eae8613e7c1d6fb7
      https://github.com/dovecot/core/commit/3d76809be2f9f3b5b4110e58eae8613e7c1d6fb7
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/doveadm/doveadm-director.c

  Log Message:
  -----------
  doveadm director: Invalid parameters cause doveadm to fail silently.

doveadm_cmd_context.cmd doesn't point directly to doveadm_cmd_director.

  Commit: ade4a0e1a8f4bc3467f4fcff9c219558fe348abf
      https://github.com/dovecot/core/commit/ade4a0e1a8f4bc3467f4fcff9c219558fe348abf
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director-connection.c
    M src/director/user-directory.c
    M src/director/user-directory.h

  Log Message:
  -----------
  director: Log user's kill_state with a human-readable string.

  Commit: 3c89b4794ac52cc229b2b1dcec2324b490539c30
      https://github.com/dovecot/core/commit/3c89b4794ac52cc229b2b1dcec2324b490539c30
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director.c

  Log Message:
  -----------
  director: If user move times out, log the state it was in at the time.

  Commit: 11ee3b40320a31669bd717fecbe1e332dad4fd84
      https://github.com/dovecot/core/commit/11ee3b40320a31669bd717fecbe1e332dad4fd84
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/doveadm-connection.c
    M src/director/doveadm-connection.h
    M src/director/main.c
    M src/director/user-directory.h

  Log Message:
  -----------
  director: HOST-RESET-USERS moves users more slowly now.

By default only 100 users can be moved in parallel. This can be overridden
with HOST-RESET-USERS parameter.

This delaying is especially useful when director_flush_socket is used to
avoid huge floods to the script service. Even without the socket it's still
good for avoiding unnecessary load spikes when all users are kicked at once
and they reconnect back at the same time.

  Commit: b79120aa36d1b7935c9aef8b0a7cf3f1045a36ca
      https://github.com/dovecot/core/commit/b79120aa36d1b7935c9aef8b0a7cf3f1045a36ca
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/doveadm/doveadm-director.c

  Log Message:
  -----------
  doveadm director flush: Added --max-parallel parameter.

  Commit: fb9dfa9ea15abdbf248021cfb7bf2846410116e6
      https://github.com/dovecot/core/commit/fb9dfa9ea15abdbf248021cfb7bf2846410116e6
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director-connection.c

  Log Message:
  -----------
  director: If user host conflict is detected, make sure new host is sent back.

USER-KICK-HASH was sent, but the sender didn't get back a USER reply with
the new host. This could have increased how long user's host differred in
directors.

Avoids repeating this error:
Error: User hash 2957018085 is being redirected to two hosts: 10.0.0.30 and 10.0.0.201 (old_ts=1477338836)

  Commit: f9c76559302e6485f59ab0e6050b76fd6d268fee
      https://github.com/dovecot/core/commit/f9c76559302e6485f59ab0e6050b76fd6d268fee
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director.c
    M src/director/user-directory.c
    M src/director/user-directory.h

  Log Message:
  -----------
  director: Prevent race conditions by adding USER_KILL_STATE_FLUSHING

In theory it's possible that a user is freed during a flush and added back
before flush is finished, possibly even being moved again. This check makes
sure that we don't finish such move unless we're actually at the correct
flushing state. (If there's another flush also running for the user it'll
be ignored.)

This is also useful for logging purposes.

  Commit: fd882f791279d0bb0cbf15ccec75cff9f6b399cc
      https://github.com/dovecot/core/commit/fd882f791279d0bb0cbf15ccec75cff9f6b399cc
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director-connection.c
    M src/director/director.c

  Log Message:
  -----------
  director: Fix user move hangs when another move is triggered early.

Running "doveadm director flush" before the previous flush hadn't fully
finished could have caused users to hang until their move timeout:

director: Error: Finishing user 3249070169 move timed out, its state may now be inconsistent (state=waiting-for-everyone)

  Commit: 0874d7a4fa15322318c71291f0134ff7cc49bbb8
      https://github.com/dovecot/core/commit/0874d7a4fa15322318c71291f0134ff7cc49bbb8
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director-connection.c
    M src/director/director.c

  Log Message:
  -----------
  director: Improve debug logging output.

  Commit: 6af8cc0573832465c621bcd91634e15461e8bf58
      https://github.com/dovecot/core/commit/6af8cc0573832465c621bcd91634e15461e8bf58
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/director/director.c
    M src/director/user-directory.h

  Log Message:
  -----------
  director: Execute director_flush_socket only from one director.

Having each director do it would be redundant since they're all supposed to
be performing the same flush task to the same backend.

  Commit: 3cbc0cb7c454ad5132dd335a7ae47bd6f5da2aa6
      https://github.com/dovecot/core/commit/3cbc0cb7c454ad5132dd335a7ae47bd6f5da2aa6
  Author: Timo Sirainen <timo.sirainen at dovecot.fi>
  Date:   2016-10-25 (Tue, 25 Oct 2016)

  Changed paths:
    M src/doveadm/doveadm-director.c

  Log Message:
  -----------
  doveadm director dump: Fixed output to include LFs

Otherwise all the output will be mixed up in the same line.

Compare: https://github.com/dovecot/core/compare/1fce089d565e...3cbc0cb7c454