Aki hello, thank you. Hopefully excerpts and top posting are acceptable in the mailing list?  On that assumption: Thanks for the input. I've checked out your suggestions (details below) but unfortunately no joy. I also restored my backup 10-ssl.conf. It indeed has the "<" sign with a space before the explicit paths to the files: ssl_cert = </etc/certbot/live/privustech.com/fullchain.pem ssl_key = </etc/certbot/live/privustech.com/privkey.pem It returns several complaints after restarting dovecot which I addressed: https://wiki2.dovecot.org/Upgrading/2.3 https://github.com/dovecot/core/blob/master/doc/example-config/conf .d/10-ssl.conf • Changed ssl_protocols to ssl_min_protocol = TLSv1 • Added ssl_dh = </etc/dovecot/dh.pem and check it with cat. It reads as a properly hashed DH PARAMETERS file. At this point we are back to the complaint about ssl_cert: Permission denied.  The certificates are root:root 0777 and of course dovecot is running as root. The conf files are andy:user 0644. The documentation says        

# PEM encoded X.509 SSL/TLS certificate and private key. They're opened before # dropping root privileges, so keep the key file unreadable by anyone but # root However if I remove the < then dovecot starts up correctly.   I delete them one at a time, test, and it shows that file read, but then fails on the next. So carry on. After the ssl_cert and ssl_key  < are removed dovecot runs (ssl_dh still has <): Dec 14 10:49:31 lavarre systemd[1]: Started Dovecot IMAP/POP3 email server.    Dec 14 10:49:31 lavarre dovecot[14059]: master: Dovecot v2.3.1 (8e2f634)     starting up for imap, pop3, lmtp But then logging in imap fails: open(old-stats-user) failed: Permission denied The documentation for 2.3 says to remove stats from mail-plugin settings, but I do not find that in either dovecot.conf or 10-mail.conf. The mail system is working correctly. Mail is received and stored in /home/alavarre/Maildir/new I'm sure it's something simple, since it worked before the version upgrade. So maybe the answer is just go back to the older version... :-(

Thanks again. Andy

Here are the results of addressing your suggestions, thank you again:
>You should set ssl_prefer_server_ciphers = yes
	Done. No change in status however...
	
>>4. We do NOT include the less than (<) symbol before the paths
because then dovecot fails to load complaining it cannot find the
files.
> Yes, this is probably indication that you are missing the files 
    The files are not missing or corrupted. cat shows apparently
properly hashed certificates and keys.
	
>or are chrooting dovecot in unsupported way. Not including the <
symbol will not help with this.
	Mmmmm:
      https://wiki.archlinux.org/index.php/Chroot
    I did not intentionally or explicitly chroot dovecot. However, it
is possible that yast2 may have done this to perform the upgrade from
Leap 42.3 to 15.0 and didn't undo it?
    However, this does not seem to have happened:
          https://stackoverflow.com/questions/75182/detecting-a-chroot-
jail-from-within
			   stat indicates that root is indeed the normal root:
				     stat -c %i /
              returns 2. (But thanks for the education! :-) I now know
about chroot...)

>You should use
>	ssl_cert =</etc/certbot/live/privustech.com/fullchain.pem  >	ssl_key =</etc/certbot/live/privustech.com/privkey.pem>	ssl_dh =</etc/dovecot/dh.pem
	
	When I do that (= <, with) or (=< without) a space between = and < and try restarting dovecot I receive:
		    Fatal: Error in configuration file /etc/dovecot/conf.d/10-ssl.conf line 16:     ssl_cert: Can't open file /etc/certbot/live/privustech.com/fullchain.pem:    Permission denied	
		
	However if I remove the < then dovecot starts up correctly:
		    Dec 14 10:49:31 lavarre systemd[1]: Started Dovecot IMAP/POP3 email server.    Dec 14 10:49:31 lavarre dovecot[14059]: master: Dovecot v2.3.1 (8e2f634)     starting up for imap, pop3, lmtp	But then logging in imap fails:
		Dec 14 11:24:22 lavarre dovecot[14062]: imap-login: Disconnected: TLS initialization failed. (no auth attempts in 0 secs): user=<>, rip=107.107.60.219, lip=70.186.159.22, session=<D6gm3f18gcZrazzb>
		Dec 14 11:24:22 lavarre dovecot[14062]: imap-login: Error: Failed to initialize SSL server context: Can't load SSL certificate: There is no valid PEM certificate.: user=<>, rip=107.107.60.219, lip=70.186.159.22, session=<XWQo3f18IcVrazzb>I'm inclined to think that the "less than" symbol is the problem. The
documentation says 
	the <paths/to/files "are relative to the currently parsed
config file's directory (/etc/dovecot/conf.d), similar to how !include
works. The file is read immediately whenever parsing the configuration
file." It also shows a space between  = and <.
	By that logic I should use
           ssl_cert = <../../certbot/live/privustech.com/fullchain.pem
           ssl_key = <../../certbot/live/privustech.com/privkey.pem	
           ssl_dh = <../../dovecot/dh.pem
  
  
      but this doesn't work either. Restoring the explicit path without <
gets us back to dovecot starting up but not able to log in with imap...

On Fri, 2018-12-14 at 07:19 +0200, Aki Tuomi wrote:
> > 
> > On 14 December 2018 at 02:12 "C. Andrews Lavarre" 
> > om> wrote:
> > 
> > 
> > Problem:
> > We had Dovecot v2.2 working just fine under openSUSE Leap 42.3. But
> > we
> > upgraded openSUSE to Leap 15.0.
> > In the process, Dovecot got upgraded from 2.2 to 2.3.1. It no
> > longer
> > works and I haven't figured out how to downgrade to the older
> > working
> > version.
> > 
> > The key issue seems to be the change to requiring dh.pem and
> > changing s
> > sl_protocols to ssl_min_protocols. I think I've navigated both
> > correctly, but it still doesn't work.
> > The error is
> > 	     auth: Error: stats: open(old-stats-user) failed:
> > Permission denied
> > 
> > 	as a consequence of which we get
> > 		    imap-login: Error: Failed to initialize SSL server
> > context: Can't
> >     load SSL certificate: There is no valid PEM certificate.
> > 
> > We have followed the instructions at 	https://wiki.dovecot.o
> > rg/S
> > SL/DovecotConfiguration
> > 	1. We have created /etc/dovecot/dh.pem (yes it took five
> > hours) 
> > 
> > 	2. We have edited 10-ssl.conf as directed by the Wiki:
> > 			    	    ssl = yes
> > 			    	    ssl_cert =
> >     /etc/certbot/live/privustech.com/fullchain.pem
> > 			    	    ssl_key =
> > /etc/certbot/live/privustech.com/privkey.pem
> > 			    	    ssl_dh = /etc/dovecot
> /dh.pem    	    #(yes, it took five hours to create...)
> 
> 
> Hi! You should use
> 
> ssl_cert =</etc/certbot/live/privustech.com/fullchain.pem  
> ssl_key =</etc/certbot/live/privustech.com/privkey.pem
> ssl_dh =</etc/dovecot/dh.pem
> 
> > 
> > 			    	    ssl_min_protocol = TLSv1
> > 			    	    ssl_cipher_list =
> > ALL:!kRSA:!SRP:!kDHd:!DSS:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PS
> > K:!RC4:!ADH:!LOW@STRENGTH
> > 			    	    ssl_prefer_server_ciphers = no
> > 
> You should set ssl_prefer_server_ciphers = yes. 
> 
> > 
> > 	3. We have checked 10-ssl.conf against the 2.3 default at
> > 		https://github.com/dovecot/core/blob/master/doc/example
> > -config/conf.d/10-ssl.conf
> > 
> > 	4. We do NOT include the less than (<) symbol before the paths
> > because then dovecot fails to load complaining it cannot find the
> > files.
> > 
> Yes, this is probably indication that you are missing the files or
> are chrooting dovecot in unsupported way. Not including the < symbol
> will not help with this.
> 
> > 
> > 	5. we have checked all the pem keys, certificates, and  dh
> > files with cat, they all exist and are in the expected hash format.
> > 
> > 	6. We have followed the instructions to set their permissions
> > root:root 0444 and 0400 accordingly.
> > 	7. We have rebooted the host.
> > 
> This is correct.
>  
> > 
> > Any help or clues would be most appreciated.
> > 
> > Kind regards, Andy
> > 

Alexander good afternoon. Thank you. I have spent the day learning about AppArmor: • I've reviewed your link, found /etc/apparmor.d/ and its local/ directory. • I ran aa-logprof and it found the change in stat to old-stat  that is discussed in the upgrade documentation. So I Allow (A) that. There are no other reports. • I followed the discussion on using yast to manage the profiles. I'm on ssh to the server so do not have the GUI yast, only the ncurses version and it does not contain editing, only adding, profiles. I tried creating a profile for imap-login with that method and scanned for any issues, there were none reported, but still cannot log in. • I followed the local/README to explicitly add /etc/certbot/live/privustech.com/* r,

	to /etc/apparmor.d/local/usr.lib.dovecot.imap-login but still

cannot login with either the mail client or with explicit openssl: it complains error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:794:

	I check yast2 sw_single for the dovecot installation. Indeed

the module dovecot23-xxx where xxx is anything that looks like "clnt" ( client?) does not exist. Is there a missing module in my installation? It lists only dovecot dovecot23 dovecot23-backend-mysql dovecot23-backend-pgsql dovecot23-backend-sqlite dovecot23-fts dovecot23-fts-squat

I'll pursue this further. Thank  you again. Kind regards, Andy

On Fri, 2018-12-14 at 23:44 +0100, Alexander Dalloz wrote:

Am 14.12.2018 um 19:58 schrieb C. Andrews Lavarre:

...

Thanks for the input. I've checked out your suggestions (details below) but unfortunately no joy. I also restored my backup 10-ssl.conf. It indeed has the "<" sign with a space before the explicit paths to the files: ssl_cert = </etc/certbot/live/privustech.com/fullchain.pem ssl_key = </etc/certbot/live/privustech.com/privkey.pem

Hi,

the syntax you see in the documentation is mandatory. Your issue is  really a permissions problem.

Check your AppArmor setup. The path you use for storing the chained  certificate and the private key is certainly not known to AppArmor. See  your /var/log/audit/audit.log for indications.

https://doc.opensuse.org/documentation/leap/security/html/book.securi ty/cha.apparmor.managing.html

may help.

Btw. permissions setting to 0777, especially for the cert and key, is  awful, even for debugging issues.

Alexander

Excellent, thank you again. The openssl command I have tried (that used to work with Dovecot 2.2) is: openssl s_client -connect mail.privustech.com:143 I have also tried openssl s_client -connect mail.privustech.com:143 -servername mail.privustech.com I've posted the full output from this to https://pastebin.com/eUSarQdx

I've posted te full output from dovecot -n to https://pastebin.com/F8Ra C4bt

Thank you again, Andy On Sat, 2018-12-15 at 17:27 +0100, Alexander Dalloz wrote:

Am 15.12.2018 um 17:16 schrieb C. Andrews Lavarre:

...

  to /etc/apparmor.d/local/usr.lib.dovecot.imap-login but 

still cannot login with either the mail client or with explicit openssl: it complains error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:794:

Hi,

that error above typically means to connect with SSL to STARTTLS or vice  versa.

Please provide the complete command you issue using "openssl s_client"  together with the corresponding dovecot logging. As well the output of  "doveconf -n" would be useful to help you further.

Alexander

Alexander, Thanks, as described before, if I include the "<" then Dovecot fails to start at all. Thank you again for your time. I have forwarded my latest to Aki to the group. Enjoy your weekend. Best regards, Andy On Sat, 2018-12-15 at 23:08 +0100, Alexander Dalloz wrote:

Am 15.12.2018 um 19:43 schrieb Aki Tuomi:

...

...

I've posted te full output from dovecot -n to https://pastebin.co m/F8Ra C4bt

You again broke your setup. From your pastebin:

ssl_cert = /etc/certbot/live/privustech.com/fullchain.pem

That's missing the "<" in front of the path to the certificate file.  Proably the same mistake for the ssl_key parameter.

Alexander

Phil hi. Thank you for explaining what the symbol does... so it is like the BASH from symbol. OK.That is new information. So without it dovecot reads the path/to/file as if it were a hashed cert, which of course doesn't work. So with the symbol dovecot tries to follow the path to read the cert but for some reason cannot read it. Now, that is curious, since I can cat the path/to/file and read the cert or key... Now, while the /path/to/file permission is presently  root:root 0777 (y es, I know 0777 is not good, but I was trying to eliminate any prevention to reading it) it is actually a soft link to yet another file. Let'sEncrypt has to be renewed every so often so the cert engine (certbot) recreates the softlink to the new cert so that we don't need to edit 10-ssl.conf.  So I have entered the actual full path/to/file for the cert and key (not the softlinks) to eliminate that possibility, buty it didn't help. So it's something else.  As you say, focus on the problem: Simply put, why can 2.3.1 not read a file while we can list and print out (ls, cat) the file? What changed in that regard from 2.2.x to 2.3.1?

I'm very grateful for the time folks have spent on this, including my own time. I'm not being rude, just factual. This is what is happening. But "something is wrong with your configuration",  while equally factual, is also equally ineffective.  OTOH, in my experience factually describing an anomaly can lead to someone wondering why it might be, and if they are more knowledgeable of the inner workings of the system be better able to understand why that might be.  For example, I didn't know anything about AppArmor before, now I do, have gone down that rabbit hole, and seem to be able to say, nope, that's not the problem. So now I can move on to checking out something else.  Similarly, under BASH the path/to/files are all correct and I can read them from the command line. And 2.2.x didn't have any problem with them. So why might 2.3.1 not be able to read them? So we all need to leave this alone, for now. I'll work along, and when/if I figure it out shall return to report. I'm sure it's something simple: Easy when you know how. :-) Thanks again. Andy On Sun, 2018-12-16 at 07:41 -0500, Phil Turmel wrote:

Andy,

This is just rude.  You have been told multiple times that the less- than symbol is required to read the certificate from the file.  Otherwise, the filename is parsed as if it is the certificate itself.  Which yields garbage.

If dovecot can't read that file, it is *not* dovecot's fault.  You are simply not going to succeed until *you* figure out what security differences you have in your new installation.  So dovecot can read the files.  Every single attempt to connect via openssh depends on dovecot reading your certificate and key files.  They are pointless exercises until dovecot actually loads your files.  Focus on the real problem if you wish to fix your service.

On 12/15/18 5:12 PM, C. Andrews Lavarre wrote:

...

Alexander, Thanks, as described before, if I include the "<" then Dovecot fails to start at all.

Thank you again for your time. I have forwarded my latest to Aki to the group.

Regards,

Phil

permissions should be 644 or 444 owned by root. if the permissions are too open, ssl/dovecot will refuse to load them. you may even see a message about it if you have verbose messages/ check your sys logs. I had this problem once with certs that checked out fine, correct < in dovcot config but didn't load. chmod 644 /etc/ssl/certs/dovecot.cert /etc/ssl/private/dovecot.key fixed the problem regards, Tim

On 16/12/2018 14:33, C. Andrews Lavarre wrote:

For what it's worth, this gives the server an A: https://www.ssllabs.com/ssltest/analyze.html?d=mail.privustech.com

So there is no problem with the certificates and key...

Thanks again.

On Sun, 2018-12-16 at 09:19 -0500, C. Andrews Lavarre wrote:

...

So it's something else.

Alexander Dalloz skrev den 2018-12-16 21:30:

Am 16.12.2018 um 19:41 schrieb Tim Dickson:

...

permissions should be 644 or 444 owned by root.

The key file should even only be readable by root and not the world. 0400 would be a good choice.

all ssl pem files must only be readeble from root, nothing else, so permisson 0400 is very god safety, dovecot read pem files before drop priviledges so that why it need to be so

Tim, Daniel, Aki, all. Problem solved. Well, sort of: It is AppArmor. I disabled AppArmor based on another sufferer's experience, and I quote: https://forums.opensuse.org/showthread.php/531740-Unexpected-pe rmissions-issue-with-Dovecot I have made some progress on solving this and tracked down the problem to apparmor which is some sort of application based security system.  (How I wish Linux followed KISS principals, this appears to be yet another security layer on top of the chmod/chown layer, and not an intuitive/obvious thing either...) So once again, a victim of political correctness. This was all more Scr ewtape distraction: There is nothing wrong with dovecot 3.2.1, there is nothing wrong with my "configuration", I am not being rude, but AppArmor got hosed by the OS upgrade. https://www.suse.com/documentation/sles11/book_security/data/se c_aaintro_enable.html Tomorrow is another day, I'll fight the AppArmor alligator then. In the meantime, on to that G&T! Woohoo! :-) Thanks again to all. Kind regards, Andy On Sun, 2018-12-16 at 18:41 +0000, Tim Dickson wrote:

permissions should be 644 or 444 owned by root. if the permissions are too open, ssl/dovecot will refuse to load them. you may even see a message about it if you have verbose messages/ check your sys logs. I had this problem once with certs that checked out fine, correct < in dovcot config but didn't load. chmod 644 /etc/ssl/certs/dovecot.cert /etc/ssl/private/dovecot.key fixed the problem regards, Tim

On 16/12/2018 14:33, C. Andrews Lavarre wrote:

...

For what it's worth, this gives the server an A: https://www.ssllabs.com/ssltest/analyze.html?d=mail.privustech. com

So there is no problem with the certificates and key...

Thanks again.

On Sun, 2018-12-16 at 09:19 -0500, C. Andrews Lavarre wrote:

...

So it's something else. 

Am 15.12.2018 um 19:02 schrieb C. Andrews Lavarre:

The openssl command I have tried (that used to work with Dovecot 2.2) is: openssl s_client -connect mail.privustech.com:143 I have also tried openssl s_client -connect mail.privustech.com:143 -servername mail.privustech.com

Please, there is zero need to mail me personlly. Keep your replies to the list. I am following here as you can see.

And to your command: it is wrong. As I guessed you are talking with SSL to the IMAP STARTTLS port. That of course cannot work.

SSL here means directly doing a secure handshaking, just like HTTPS is working. The default port for IMAPS is 993, not 143.

If you test against IMAP/STARTTLS on port 143, then do with

openssl s_client -connect mail.privustech.com:143 -starttls imap

As your "doveconf -n" does not show any special setup regarding IMAPS or IMAP/STARTTLS the case is as analyzed.

Alexander