Hello,
Debian Stretch impact free security upgrade.
This is for the default version/unit which has a Type=forking service, not the backport and buster one which switched to simple and has dovecot running in the foreground. However it should work just the same, obviously try this on a test machine.
Create /etc/systemd/system/dovecot.service.d/override.conf:
[Service] ExecStart= ExecStart=/bin/true ExecStop= ExecStop=/bin/true KillMode=none
Make it active with: systemctl daemon-reload
Do a "systemctl status dovecot.service" and note the PID, the "Main PID: pidnum (dovecot)" one from the status output. A stop action will have systemd delete the pid file (why, oh why?) and thus during the course of an upgrade we will have to re-establish it in another session with:
echo pidnum >/var/run/dovecot/master.pid
I had to do this 3 times, every time there was a setting up or processing triggers for dovecot-core. You can see it by things stalling out or watching "systemctl status dovecot.service" which will have it "starting". Since no daemons actually get stopped or started this is harmless, as in there won't be any impact to existing or new connections.
And after the upgrade is finished a:
systemctl status dovecot.service
should have the status back to active.
At this point any new IMAP sessions will be using the new code, but imap-login will NOT. A "systemctl reload dovecot.service" will have the master spawn off new daemons while NOT killing off the old ones, so existing proxy connections won't be killed either.
Afterwards you will likely want to comment out the "Exec" lines and do another "systemctl daemon-reload", to re-establish normal behavior.
The Killmode bit depends on if you want the dovecot option "shutdown_clients = no" to work as expected. That is, on stretch backports dovecot or buster, with the default stretch version a "systemctl restart" will still kill off existing IMAP sessions for reasons that are currently beyond me.
On a stretch-backport install we get initially:
ps faxv|grep dove
3342 ? Ss 0:00 0 78 18449 3164 0.0 /usr/sbin/dovecot -F 3344 ? S 0:00 0 20 23471 3460 0.0 \_ dovecot/pop3-login 3345 ? S 0:00 0 30 23469 3740 0.0 \_ dovecot/imap-login 3346 ? S 0:00 0 16 9867 1096 0.0 \_ dovecot/anvil [17 connections] 3347 ? S 0:00 0 17 9998 2668 0.0 \_ dovecot/log [etc] 3362 ? S 0:00 0 117 26438 4452 0.1 \_ dovecot/config 3363 ? S 0:00 0 19 10120 2988 0.0 \_ dovecot/stats [19 connections] 3365 ? S 0:00 0 356 87703 8032 0.1 \_ dovecot/auth [0 wait, 0 passdb, 0 userdb] 3408 ? S 0:00 0 25 9866 1108 0.0 \_ dovecot/imap-hibernate [0 connections] 4690 ? S 0:00 0 235 26372 5312 0.1 \_ dovecot/imap [test@goltest.com 203.216.99.99 IDLE]
and after a restart as expected:
3347 ? S 0:00 0 17 9998 2668 0.0 dovecot/log 3363 ? S 0:00 0 19 10120 2988 0.0 dovecot/stats [1 connections] 4690 ? S 0:00 0 235 26372 5312 0.1 \_ dovecot/imap [test@goltest.com 203.216.99.99 IDLE] 4701 ? Ss 0:00 0 78 18449 3096 0.0 /usr/sbin/dovecot -F 4704 ? S 0:00 0 20 23471 3460 0.0 \_ dovecot/pop3-login 4705 ? S 0:00 0 30 23469 3468 0.0 \_ dovecot/imap-login 4706 ? S 0:00 0 16 9867 1096 0.0 \_ dovecot/anvil [17 connections] 4707 ? S 0:00 0 17 9998 2424 0.0 \_ dovecot/log [etc] 4722 ? S 0:00 0 117 26306 4448 0.1 \_ dovecot/config 4723 ? S 0:00 0 19 9996 2540 0.0 \_ dovecot/stats [17 connections] 4725 ? S 0:00 0 356 87703 8048 0.1 \_ dovecot/auth [0 wait, 0 passdb, 0 userdb]
Regards,
Christian
On Sat, 31 Aug 2019 11:30:12 +0900 Christian Balzer via dovecot wrote:
Daniel,
thanks so much for the detailed pointers.
So it turns out to be both the evil that is systemd and an overzealous upgrade script.
Apollon, should I raise a Debian bug for this?
As for reasons, how do 50k proxy session on the proxy servers and 25k imap processes on the mailbox servers sound?
Even on a server with just 6k users and 7k imap processes that causes a massive load spike and a far longer service interruption (about 50 seconds) than I'm happy with.
Penultimately if people do set "shutdown_clients = no" they hopefully know what they are doing and do expect that to work.
Regards,
Christian
On Fri, 30 Aug 2019 17:44:23 +0200 Daniel Lange via dovecot wrote:
Am 30.08.19 um 17:38 schrieb Daniel Lange via dovecot:
Am 30.08.19 um 10:00 schrieb Christian Balzer via dovecot:
When upgrading on Debian Stretch with the security fix packages all dovecot processes get killed and then restarted despite having "shutdown_clients = no" set.
This is systemd doing its "magic" (kill all control group processes), see https://dovecot.org/pipermail/dovecot/2016-June/104546.html for a potential fix.
Actually that will not be enough in the upgrade case as the maintainer script calls deb-systemd-invoke stop dovecot.socket dovecot.service
I personally think re-connecting clients are normal operations so I wouldn't bother. But you could override the stop action in the systemd unit if you have local reasons that warrant such a hack.
-- Christian Balzer Network/Systems Engineer
chibi@gol.com Rakuten Mobile Inc.
--
Christian Balzer Network/Systems Engineer
chibi@gol.com Rakuten Mobile Inc.