Dovecot and Letsencrypt certs
So this morning at 4am I was awoken to my mail clients getting certificate errors for an expired certificate.
I hopped on to the server and checked and… no, the LE certs renewed last month and are valid until November.
After some moments of confusion I noticed that dovecot had been running since before the renewal, so I did a quick service dovecot restart which fixed everything.
Should dovecot check for certs being refreshed? Or is this an artifact of my using symbolic links everywhere to point to the newest LE certs (which are themselves links the dehydrate script creates to point to the newest cert-1502534746.csr etc files?
Should I just create a monthly cron to restart dovecot or is there something else?
-- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
Dovecot seems to load certificates into memory and don't refresh them until restart, or may be reload. And this is a correct logic. You better add restart/reload task to the LE cron job after the successful renewal of LE certificate. Check that it really works as it should. Dovecot shouldn't be restarted/reloaded if certificate wasn't changed.
2017-09-08 17:47 GMT+05:00 @lbutlr kremels@kreme.com:
So this morning at 4am I was awoken to my mail clients getting certificate errors for an expired certificate.
I hopped on to the server and checked and… no, the LE certs renewed last month and are valid until November.
After some moments of confusion I noticed that dovecot had been running since before the renewal, so I did a quick service dovecot restart which fixed everything.
Should dovecot check for certs being refreshed? Or is this an artifact of my using symbolic links everywhere to point to the newest LE certs (which are themselves links the dehydrate script creates to point to the newest cert-1502534746.csr etc files?
Should I just create a monthly cron to restart dovecot or is there something else?
-- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
On 08-09-2017 09:47, @lbutlr wrote:
Should dovecot check for certs being refreshed? Or is this an artifact of my using symbolic links everywhere to point to the newest LE certs (which are themselves links the dehydrate script creates to point to the newest cert-1502534746.csr etc files?
Should I just create a monthly cron to restart dovecot or is there something else? Dovecot needs a restart after the certificate is changed. certbot allows you to define hooks to be run after a certificate is renewed, so you could use that feature to restart dovecot after the renewal. Other clients might have similar features.
-- While you recently had your problems on the run, they've regrouped and are making another attack.
Eduardo M KALINOWSKI eduardo@kalinowski.com.br
On Fri, Sep 08, 2017 at 06:47:25AM -0600, @lbutlr wrote:
So this morning at 4am I was awoken to my mail clients getting certificate errors for an expired certificate.
I hopped on to the server and checked and… no, the LE certs renewed last month and are valid until November.
After some moments of confusion I noticed that dovecot had been running since before the renewal, so I did a quick service dovecot restart which fixed everything.
Should dovecot check for certs being refreshed? Or is this an artifact of my using symbolic links everywhere to point to the newest LE certs (which are themselves links the dehydrate script creates to point to the newest cert-1502534746.csr etc files?
As you're using dehydrated, I can share what I do. My hook script basically calls "run-parts /etc/dehydrated/hooks.d/" so I can just drop hook scripts into that directory. Then in the hooks.d directory, I have the following:
#!/bin/bash
set -e
set -u
set -o pipefail
if [[ ${1} == "deploy_cert" && ${2} == "mail.darac.org.uk" ]]; then
echo " + Hook: Restarting Dovecot..."
/usr/sbin/service dovecot restart
fiThat means that dovecot will be restarted only if the certificate for the mail server is being deployed. If dehydrated runs, but fails to renew the certificate, then dovecot won't be restarted. Similarly, if it renews a different certificate, dovecot won't be restarted.
Hope that helps.
Should I just create a monthly cron to restart dovecot or is there something else?
-- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
-- For more information, please reread.
On Sep 8, 2017, at 07:56, Darac Marjal mailinglist@darac.org.uk wrote:
#!/bin/bash
set -e set -u set -o pipefail
if [[ ${1} == "deploy_cert" && ${2} == "mail.darac.org.uk" ]]; then echo " + Hook: Restarting Dovecot..." /usr/sbin/service dovecot restart fi
That means that dovecot will be restarted only if the certificate for the mail server is being deployed. If dehydrated runs, but fails to renew the certificate, then dovecot won't be restarted. Similarly, if it renews a different certificate, dovecot won't be restarted.
That is a great solution, but I think it’s probably easier to just kick dovecot once a month.
4 4 4 * * service dovecot restart
However, it seems like checking the certs is something that dovecot should be doing on its own.
-- This is my signature. There are many like it, but this one is mine.
"I think it’s probably easier to just kick dovecot once a month." - that's not good from system administration's point of view. You can get into trouble when certificate is renewed but dovecot isn't reloaded yet. And, doing something via cron just by-guess, once a month - is a no no logic. "it seems like checking the certs is something that dovecot should be doing on its own" if dovecot loads it in memory, it shouldn't reread certificates. Why to take servers resources just 'because of something may be changed' restarting dovecot with no need ? And, never do restart if reload suits your needs. But check it first, if reload action rereads certificate from file system.
2017-09-08 19:20 GMT+05:00 LuKreme kremels@kreme.com:
On Sep 8, 2017, at 07:56, Darac Marjal mailinglist@darac.org.uk wrote:
#!/bin/bash
set -e set -u set -o pipefail
if [[ ${1} == "deploy_cert" && ${2} == "mail.darac.org.uk" ]]; then echo " + Hook: Restarting Dovecot..." /usr/sbin/service dovecot restart fi
That means that dovecot will be restarted only if the certificate for the mail server is being deployed. If dehydrated runs, but fails to renew the certificate, then dovecot won't be restarted. Similarly, if it renews a different certificate, dovecot won't be restarted.
That is a great solution, but I think it’s probably easier to just kick dovecot once a month.
4 4 4 * * service dovecot restart
However, it seems like checking the certs is something that dovecot should be doing on its own.
-- This is my signature. There are many like it, but this one is mine.
On 08 Sep 2017, at 09:28, Вадим Бажов master@remort.net wrote:
"I think it’s probably easier to just kick dovecot once a month." - that's not good from system administration's point of view. You can get into trouble when certificate is renewed but dovecot isn't reloaded yet.
That's simply not possible. The cert renews well before it expires.
"it seems like checking the certs is something that dovecot should be doing on its own" if dovecot loads it in memory, it shouldn't reread certificates.
Of course it should because certs are DESIGNED to expire and MUST expire, and dovecot certainly has the ability to see when the cert expires.
Why to take servers resources just 'because of something may be changed'
Something WILL be changed, absolutely certain of that. All certs expire.
restarting dovecot with no need ?
restarting/reloading dovecot is trivial and takes far less time than writing a script to check the certs and then creating a crontab for that which also gives a tertiary point of failure.
-- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
On 08.09.2017 16:20, LuKreme wrote:
That is a great solution, but I think it’s probably easier to just kick dovecot once a month.
Certbot hooks are very easy to write, and are only executed when the certificate is updated. In that light, I can see no advantage in "kick dovecot once a month". ;-)
However, it seems like checking the certs is something that dovecot should be doing on its own.
What is Dovecot supposed to do? Keep track of the certificate expiry date? And if that is passed, then what? Automatically shutdown/restart? What if the certificate has not been updated in between? I think that handling certificates is better left to the administrator.
-Ralph
On 08 Sep 2017, at 10:08, Ralph Seichter m16+dovecot@monksofcool.net wrote:
What is Dovecot supposed to do? Keep track of the certificate expiry date? And if that is passed, then what? Automatically shutdown/restart? What if the certificate has not been updated in between? I think that handling certificates is better left to the administrator.
How I would do it is IF the certificate is expired, the dovecot should check if there is a new cert and if so, load it. This prevents a failure event, but doesn't interfere with reloading the cert when it is renewed.
-- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
On 08.09.2017 19:51, @lbutlr wrote:
How I would do it is IF the certificate is expired, the dovecot should check if there is a new cert and if so, load it.
New cert as in file modification date or checksum changed? Might work. Still, from what I seem to remember, Dovecot loads certificate data before dropping privileges, which is why reloading the data might be problematic without some changes. Not worth spending development effort on, IMO, given that Dovecot can easily be restarted by the external processes that update the cert (like Certbot hook, Ansible, etc.).
-Ralph
On 08 Sep 2017, at 12:21, Ralph Seichter m16+dovecot@monksofcool.net wrote:
On 08.09.2017 19:51, @lbutlr wrote:
How I would do it is IF the certificate is expired, the dovecot should check if there is a new cert and if so, load it.
New cert as in file modification date or checksum changed?
Either one, but checksum is going to be more reliable.
Might work. Still, from what I seem to remember, Dovecot loads certificate data before dropping privileges, which is why reloading the data might be problematic without some changes.
Can't dovecot reload itself? That could be a problem if not.
Not worth spending development effort on, IMO, given that Dovecot can easily be restarted by the external processes that update the cert (like Certbot hook, Ansible, etc.).
All I'm saying is that it's a failure event that doesn't need to occur.
-- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
"writing a script to check the certs" - there is no need to write any scripts. As one mentioned, it's done by a hook to certbot. Please read the manuals for LE or certbot. The issue you have is quite common and of course certbot designed to do it for you. The manual: https://certbot.eff.org/docs/using.html#renewing-certificates. Thats it. Problem solved.
2017-09-09 0:18 GMT+05:00 @lbutlr kremels@kreme.com:
On 08 Sep 2017, at 12:21, Ralph Seichter m16+dovecot@monksofcool.net wrote:
On 08.09.2017 19:51, @lbutlr wrote:
How I would do it is IF the certificate is expired, the dovecot should check if there is a new cert and if so, load it.
New cert as in file modification date or checksum changed?
Either one, but checksum is going to be more reliable.
Might work. Still, from what I seem to remember, Dovecot loads certificate data before dropping privileges, which is why reloading the data might be problematic without some changes.
Can't dovecot reload itself? That could be a problem if not.
Not worth spending development effort on, IMO, given that Dovecot can easily be restarted by the external processes that update the cert (like Certbot hook, Ansible, etc.).
All I'm saying is that it's a failure event that doesn't need to occur.
-- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
On Friday 08 of September 2017, Ralph Seichter wrote:
On 08.09.2017 16:20, LuKreme wrote:
However, it seems like checking the certs is something that dovecot should be doing on its own.
What is Dovecot supposed to do? Keep track of the certificate expiry date?
That was already discussed but due to other reason. dovecot shouldn't load SSL certificates into memory and instead open & load cert on demand (when client connects and requests particular domain via SNI (or default if no SNI)).
Why? Because dovecot *cannot* handle thousands of virtual domains and SSL certificates for these. It wastes so much RAM and timeouts on reloads in such case. Tested here. [1]
That's why the only sensible solution is to work like exim - load cert from disk on demand.
That fixes both problems - ram wasting/timeouts and refreshing certificates.
-Ralph
-- Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )
If you're using acme.sh:
acme.sh --installcert -d imap.example.com
--keypath /etc/pki/dovecot/private/imap.example.com.pem
--certpath /etc/pki/dovecot/certs/imap.example.com.crt
--fullchainpath /etc/pki/dovecot/certs/imap.example.com.full.chain.crt
--reloadcmd "systemctl reload dovecot.service"
HTH, Bill
On 9/8/2017 9:56 AM, Darac Marjal wrote:
On Fri, Sep 08, 2017 at 06:47:25AM -0600, @lbutlr wrote:
So this morning at 4am I was awoken to my mail clients getting certificate errors for an expired certificate.
I hopped on to the server and checked and… no, the LE certs renewed last month and are valid until November.
After some moments of confusion I noticed that dovecot had been running since before the renewal, so I did a quick service dovecot restart which fixed everything.
Should dovecot check for certs being refreshed? Or is this an artifact of my using symbolic links everywhere to point to the newest LE certs (which are themselves links the dehydrate script creates to point to the newest cert-1502534746.csr etc files?
As you're using dehydrated, I can share what I do. My hook script basically calls "run-parts /etc/dehydrated/hooks.d/" so I can just drop hook scripts into that directory. Then in the hooks.d directory, I have the following:
#!/bin/bash
set -e set -u set -o pipefail
if [[ ${1} == "deploy_cert" && ${2} == "mail.darac.org.uk" ]]; then echo " + Hook: Restarting Dovecot..." /usr/sbin/service dovecot restart fi
That means that dovecot will be restarted only if the certificate for the mail server is being deployed. If dehydrated runs, but fails to renew the certificate, then dovecot won't be restarted. Similarly, if it renews a different certificate, dovecot won't be restarted.
Hope that helps.
Should I just create a monthly cron to restart dovecot or is there something else?
-- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
Oh, also I removed the '2>1> /dev/null' from the acme.sh crontab entry so that it will always send an email; and entered this in sieve: # ------- let's encrypt ------- if header :contains "subject" "acme.sh" { if body :regex "Error[[:space:]]+renew" { # redirect :copy "bill@example.com"; addflag "$label1"; # Thunderbird red stop; } if body :regex "-----BEGIN CERTIFICATE-----" { # redirect :copy "bill@example.com"; addflag "$label4"; # Thunderbird blue stop; } fileinto "AASystemAdministration.Cron.certificate"; stop; } The redirect :copy I enable for my other servers to forward a copy to me.
HTH, BIll
On 9/9/2017 3:16 PM, Bill Shirley wrote:
If you're using acme.sh: acme.sh --installcert -d imap.example.com
--keypath /etc/pki/dovecot/private/imap.example.com.pem
--certpath /etc/pki/dovecot/certs/imap.example.com.crt
--fullchainpath /etc/pki/dovecot/certs/imap.example.com.full.chain.crt
--reloadcmd "systemctl reload dovecot.service"HTH, Bill
On 9/8/2017 9:56 AM, Darac Marjal wrote:
On Fri, Sep 08, 2017 at 06:47:25AM -0600, @lbutlr wrote:
So this morning at 4am I was awoken to my mail clients getting certificate errors for an expired certificate.
I hopped on to the server and checked and… no, the LE certs renewed last month and are valid until November.
After some moments of confusion I noticed that dovecot had been running since before the renewal, so I did a quick service dovecot restart which fixed everything.
Should dovecot check for certs being refreshed? Or is this an artifact of my using symbolic links everywhere to point to the newest LE certs (which are themselves links the dehydrate script creates to point to the newest cert-1502534746.csr etc files?
As you're using dehydrated, I can share what I do. My hook script basically calls "run-parts /etc/dehydrated/hooks.d/" so I can just drop hook scripts into that directory. Then in the hooks.d directory, I have the following:
#!/bin/bash
set -e set -u set -o pipefail
if [[ ${1} == "deploy_cert" && ${2} == "mail.darac.org.uk" ]]; then echo " + Hook: Restarting Dovecot..." /usr/sbin/service dovecot restart fi
That means that dovecot will be restarted only if the certificate for the mail server is being deployed. If dehydrated runs, but fails to renew the certificate, then dovecot won't be restarted. Similarly, if it renews a different certificate, dovecot won't be restarted.
Hope that helps.
Should I just create a monthly cron to restart dovecot or is there something else?
-- Apple broke AppleScripting signatures in Mail.app, so no random signatures.
participants (8)
-
＠lbutlr
-
Arkadiusz Miśkiewicz
-
Bill Shirley
-
Darac Marjal
-
Eduardo M KALINOWSKI
-
LuKreme
-
Ralph Seichter
-
Вадим Бажов