Cannot connect to Dovecot IMAP or POP
Hello all. Thank you for your service.
Easy when you know how, but presently I do not. After literally months of research and experimentation we simply cannot log into our PAM / apache2 / postfix / dovecot pop3/imap STARTTLS email server with an ordinary email client, e.g., Evolution or Thunderbird.
We can connect to the host server in a host of different ways (no pun intended)—http, https, ssh, vnc, telnet, openssl -sclient
Similarly we can connect to postfix and dovecot in yet another number of ways—telnet, openssl -sclient—but cannot log in to the email server with a normal email client (either Evolution or Thunderbird) by either pop3 or imap.
SSL certificates are in place, verified, and tested.
Part of the problem is the many changes in all the involved operating systems and protocols (e.g., imaps and pop3s are deprecated, openSUSE has migrated to LEAP, etc.) so many of the docs from Google are no longer valid. Additionally, there simply are bugs: Leap 42.1 YAST does not work when it comes to setting up websites. Documented. But I digress.
I'm sure it's something really simple, but it evades me. Research details below. Any help would be more than appreciated.
Thanks in advance, Andy
======================= Configuration testing details =======================
System is: Linux openSUSE Leap 42.1 Dovecot --version 2.2.18, Postfix Version: 2.11.6-3.1 Apache2 Version: 2.4.16-9.1
Connections 1. Evolution or Thunderbird to pop3 or imap reports: The reported error was "Could not connect to mail.privustech.com: Connection refused". Both connect successfully to googlemail.com with the same protocol: Port 993 SSL on a dedicated port
I have also tried
Port 143 STARTTLS after connecting
without success
2. openssl s_client -connect mail.privustech.com:xxx
a. xxx=25, 110, 143 all return
error:140770FC
b. xxx=993, 995 return
socket: Connection refused
connect:errno=111
3.telnet to
a. smtp works.
b. pop3
andy@tm2t:~> telnet 70.186.159.22 110
...
+OK POP3 2007e.104 server ready <48fa.572a0769@privustech.com>
...
user andy
-ERR Unknown AUTHORIZATION state command
c. imap connects but does not allow login, and should not.
http://marc.info/?l=imap&m=118775891829506&w=2
The most simple answer is "you cannot TELNET to a modern, correctly-configured,
IMAP server and log in to it."
andy@tm2t:~> telnet 70.186.159.22 143
...
* OK [...] privustech.com IMAP4rev1 2007e.404 at Wed, 4 May 2016 10:26:28
-0400 (EDT)
... A NO Invalid login credentials
Modules
• Apache2 works just fine. The server is up and answering. ping works just fine. We have http and https to all vhost sites (privustech, mailprivustech, nptbeyond, gvhl, truthcourage, and their www. subsites).
• Postfix reports no errors. We can log in on localhost, send a message to ourselves and see the message.
• Dovecot:
a. Logging is enabled in 10-logging.conf to /var/log/dovecot.conf but no logging has occurred there.
b. doveconf -n throws no errors.
Checks and tests completed
1. /etc/hosts is just fine.
2. Firewall is open for telnet, postfix, dovecot.
3. Added andy to dovecot, postfix groups, in addition to mail, reset password to ANDYbbs14@.
4. We tried enabling imaps, pop3s, but this command returns errors about these protocols being obsolete.
https://tools.ietf.org/html/rfc2595
Use of these ports is discouraged in favor of the STARTTLS or STLS
commands.
5. Reviewed doveconf -n:
a. Note, there are no Dovecot users established other than
user postfix
group postfix
service auth {
unix_listener auth-userdb {
group = postfix
user = postfix
}
}
i. postfix has its own set of users, including andy, which works just fine within postfix.
We can send mail and read mail in the mailbox.
b. Authentication is performed by PAM:
passdb {
driver = pam
}
i. Examined PAM:
A. The files /etc/pam.d/xxx, where xxx = dovecot, pop, imap, are all the same
lavarre:~ # cat /etc/pam.d/xxx
#%PAM-1.0
auth include common-auth
account include common-account
password include common-password
session include common-session
B. They do not resemble at all the form presented in
http://wiki2.dovecot.org/PasswordDatabase/PAM
passdb {
driver = pam
args = %s
}
C. Add (B.) to see if that works: No change.
Comment out the original (A.): No change.
Restore it.
c. SSL is required and apparently configured correctly
(the less-than symbol '<'causes the succeeding file to be read into the variable):
ssl = required
ssl_cert = </etc/ssl/certs/dovecot.pem
ssl_dh_parameters_length = 2048
ssl_key = </etc/ssl/private/dovecot.pem
ssl_options = no_compression
ssl_prefer_server_ciphers = yes
userdb {
driver = passwd
}
i. dovecot.pem, both cert and key, are installed in /etc/ssl as above and verified as a pair with
openssl x509.
And we point to them in /etc/dovecot/conf.d/10-ssl.conf as seen in the above.
6. Checked listening as it does not appear in doveconf -n:
lavarre:~ # doveconf protocols listen
protocols = imap pop3 lmtp
listen = *, ::
a. conf.d/10-master.conf
ports for service xxx-login {inet_listener} are commented out.
In fact, the entire file is commented out.
Uncomment the listeners, restart. But no change. So undo.
Re-read the following:
1st http://wiki2.dovecot.org/PasswordDatabase
2nd http://wiki2.dovecot.org/Authentication/Mechanisms
then edit /etc/dovecot/conf.d/10-auth.conf auth_mechanisms = plain login
On 05/04/16 19:00, C. Andrews Lavarre wrote:
Hello all. Thank you for your service.
Easy when you know how, but presently I do not. After literally months of research and experimentation we simply cannot log into our PAM / apache2 / postfix / dovecot pop3/imap STARTTLS email server with an ordinary email client, e.g., Evolution or Thunderbird.
We can connect to the host server in a host of different ways (no pun intended)—http, https, ssh, vnc, telnet, openssl -sclient
Similarly we can connect to postfix and dovecot in yet another number of ways—telnet, openssl -sclient—but cannot log in to the email server with a normal email client (either Evolution or Thunderbird) by either pop3 or imap.
SSL certificates are in place, verified, and tested.
Part of the problem is the many changes in all the involved operating systems and protocols (e.g., imaps and pop3s are deprecated, openSUSE has migrated to LEAP, etc.) so many of the docs from Google are no longer valid. Additionally, there simply are bugs: Leap 42.1 YAST does not work when it comes to setting up websites. Documented. But I digress.
I'm sure it's something really simple, but it evades me. Research details below. Any help would be more than appreciated.
Thanks in advance, Andy
======================= Configuration testing details =======================
System is: Linux openSUSE Leap 42.1 Dovecot --version 2.2.18, Postfix Version: 2.11.6-3.1 Apache2 Version: 2.4.16-9.1
Connections
Evolution or Thunderbird to pop3 or imap reports: The reported error was "Could not connect to mail.privustech.com: Connection refused".
Both connect successfully to googlemail.com with the same protocol: Port 993 SSL on a dedicated port
I have also tried Port 143 STARTTLS after connecting without success 2. openssl s_client -connect mail.privustech.com:xxx a. xxx=25, 110, 143 all return error:140770FC
b. xxx=993, 995 return socket: Connection refused connect:errno=111
3.telnet to a. smtp works. b. pop3 andy@tm2t:~> telnet 70.186.159.22 110 ... +OK POP3 2007e.104 server ready 48fa.572a0769@privustech.com ... user andy -ERR Unknown AUTHORIZATION state command
c. imap connects but does not allow login, and should not. http://marc.info/?l=imap&m=118775891829506&w=2 The most simple answer is "you cannot TELNET to a modern, correctly-configured, IMAP server and log in to it." andy@tm2t:~> telnet 70.186.159.22 143 ... * OK [...] privustech.com IMAP4rev1 2007e.404 at Wed, 4 May 2016 10:26:28 -0400 (EDT) ... A NO Invalid login credentials
Modules
• Apache2 works just fine. The server is up and answering. ping works just fine. We have http and https to all vhost sites (privustech, mailprivustech, nptbeyond, gvhl, truthcourage, and their www. subsites).
• Postfix reports no errors. We can log in on localhost, send a message to ourselves and see the message.
• Dovecot: a. Logging is enabled in 10-logging.conf to /var/log/dovecot.conf but no logging has occurred there. b. doveconf -n throws no errors.
Checks and tests completed
- /etc/hosts is just fine.
2. Firewall is open for telnet, postfix, dovecot.
3. Added andy to dovecot, postfix groups, in addition to mail, reset password to ANDYbbs14@.
- We tried enabling imaps, pop3s, but this command returns errors about these protocols being obsolete. https://tools.ietf.org/html/rfc2595 Use of these ports is discouraged in favor of the STARTTLS or STLS commands.
5. Reviewed doveconf -n: a. Note, there are no Dovecot users established other than user postfix group postfix service auth { unix_listener auth-userdb { group = postfix user = postfix } } i. postfix has its own set of users, including andy, which works just fine within postfix. We can send mail and read mail in the mailbox. b. Authentication is performed by PAM: passdb { driver = pam } i. Examined PAM: A. The files /etc/pam.d/xxx, where xxx = dovecot, pop, imap, are all the same lavarre:~ # cat /etc/pam.d/xxx #%PAM-1.0 auth include common-auth account include common-account password include common-password session include common-session B. They do not resemble at all the form presented in http://wiki2.dovecot.org/PasswordDatabase/PAM passdb { driver = pam args = %s } C. Add (B.) to see if that works: No change. Comment out the original (A.): No change. Restore it. c. SSL is required and apparently configured correctly (the less-than symbol '<'causes the succeeding file to be read into the variable): ssl = required ssl_cert = </etc/ssl/certs/dovecot.pem ssl_dh_parameters_length = 2048 ssl_key = </etc/ssl/private/dovecot.pem ssl_options = no_compression ssl_prefer_server_ciphers = yes userdb { driver = passwd } i. dovecot.pem, both cert and key, are installed in /etc/ssl as above and verified as a pair with openssl x509. And we point to them in /etc/dovecot/conf.d/10-ssl.conf as seen in the above.
6. Checked listening as it does not appear in doveconf -n: lavarre:~ # doveconf protocols listen protocols = imap pop3 lmtp listen = *, ::
a. conf.d/10-master.conf ports for service xxx-login {inet_listener} are commented out. In fact, the entire file is commented out. Uncomment the listeners, restart. But no change. So undo.
Hello all, thank you again for your help. Thanks to Edgar Pettijohn's inspiration, we changed /etc/dovecot/conf.d /10-auth.conf to include login (which did not work) and cram-md5 (whichdid work): auth_mechanisms = plain login cram-md5 and we no longer get Connection refused.
Although it doesn't say so explicitly, my reading of
http://wiki2.dovecot.org/Authentication/Mechanisms
is that SSL/TLS puts a wrapper around plaintext passwords,
so you don't need an encrypted password.
However, obviously, you need a scheme to first decrypt the TLS envelope!
So does cram-md5 do that?
Seems to work. Thank you.
So now, as Joseph Tam points out, (thank you for the exposure to nc—cool) we are back to "Server certificate not installed".
But "the certificate" is installed AFAICT on mail.privustech.com and dovecot:
So which server? The choices are
• The root server: 70.186.159.22
• The virtual host mail server: mail.privustech.com
• The dovecot server: /etc/dovecot/dovecot.conf
• Something else.
Presumably, as Joseph shows with his nc call, imap calls are to ServerName mail.privustech.com. So we need it to exist and we need cert files for that ServerName: · We can connect, so the server exists and is responding. · It is configured as a virtual host and has its own Apache2 configuration files mail.privustech.com.conf mail.privustech.com-ssl.conf These in turn specify SSL cert, key, and CA files with the CN mail.privustech.com This host is specified as a port 443 vhost, but changing to 143 had no effect. I can also connect with https, so the cert is valid.
So I cannot imagine how better to "install" it to a valid host with a valid cert... ??? :-(
I examined the other possible "servers" and they all seem correctly established as well. Details of today's angst appended below.
Thanks again for the help and inspiration. Tomorrow is another day.
Best regards, Andy
==========================================
The root server is 70.186.159.22 It is configured in /etc/apache2/default-server.conf This file specifies ServerName as 70.186.159.22 The root server under Apache2 does not have an SSL.conf file, however the root server also is installed as a virtual host in /etc/apache2/vhosts.d through /etc/apache2/vhosts.d/70.186.159.22.conf /etc/apache2/vhosts.d/70.186.159.22-ssl.conf The latter file specifies three SSL files: SSLCertificateFile /etc/apache2/ssl.crt/mail.privustech.com_start.crt SSLCertificateKeyFile /etc/apache2/ssl.key/mailprivustech.key SSLCertificateChainFile /etc/apache2/ssl.crt/mailprivustech_root_bundle.crt Of course, the Common Name (CN) in these files does not match the root ServerName. If dovecot connects from the root server rather than mail.privustech.com that would explain the matter. We'll check that out tomorrow.
We are not, however, trying to connect to the root server, rather to mail.privustech.com This virtual host is manifested in Apache2 through /etc/apache2/vhosts.d/mail.privustech.com.conf /etc/apache2/vhosts.d/mail.privustech.com-ssl.conf The ServerName does match the CN in this case. The port number in the vhost is 443 vice 143, but we changed that with no effect. So it does not make sense that an imap connection responds with "Server certificate not installed" How more to "install" the cert than to specify it in the vhost -ssl.conf file?
The mail server vhost StartSSL certificate is /etc/apache2/ssl.crt/mail.privustech.com_start.crt and has been validated against its key. Its CN is mail.privustech.com.
The dovecot server SSL certificate is specified in the configuration file: /etc/dovecot/dovecot.conf It does not specify a key, however it includes all files in /etc/dovecot/conf.d This contains a number of files, including 10-auth.conf 10-ssl.conf
The first includes auth-mechanisms plain login cram-md5 Adding cram-md5 today resolved the "Connection Refused" issue. Although it doesn't say so explicitly, my reading of http://wiki2.dovecot.org/Authentication/Mechanisms is that SSL/TLS puts a wrapper around plaintext passwords, so you don't need an encrypted database. However, obviously, you need a scheme to first decrypt the TLS envelope! So does cram-md5 do that? Seems to work. Thank you. Default settings are included but commented out. In particular, plaintext is by default disabled. So we uncomment and explicitly declare disable_plaintext_auth = no Restart: No change. Restore. /etc/dovecot/conf.d/10-ssl.conf contains explicit referral to the mail.privustech.com SSL files discussed above: ssl = required ssl_cert = </etc/apache2/ssl.crt/mail.privustech.com_start.crt ssl_key = </etc/apache2/ssl.key/mailprivustech.key ssl_ca = </etc/apache2/ssl.crt/mailprivustech_root_bundle.crt So again, it would appear that the certs are indeed installed.
Other possibilities. We look around for other possible certificate assignment locations that might be overriding the explicit settings above.
a. Check /etc/apache2/httpd.conf. It only contains include statements pointing to the other .conf files.
b. Check the included /etc/apache2/ssl-global.conf. SSLCertificateFile SSLCertificateKeyFile SSLCertificateChainFile SSLCACertificateFile are all commented out. We haven't needed them in the past because we had vhosts for all the sites with their own .conf and -ssl.conf files. But perhaps they are now needed for the root domain if it, not mail.privustech.com, is being answered by dovecot. So set them up to mail.privustech.com: SSLCertificateFile /etc/apache2/ssl.crt/mail.privustech.com_start.crt SSLCertificateKeyFile /etc/apache2/ssl.key/mailprivustech.key SSLCertificateChainFile /etc/apache2/ssl.crt/mailprivustech_root_bundle.crt SSLCACertificateFile /etc/apache2/ssl.crt/mailprivustech_root_bundle.crt Save and restart Apache2.but no change.
c. Default Server /etc/apache2/default-server.conf specifies the default server as ServerName 70.186.159.22 We have covered that with 1. and 2. above.
d. Invalid permissions? .conf and SSL file permissions: all rw-r--r-- root:root On Wed, 2016-05-04 at 20:01 -0500, Edgar Pettijohn wrote:
Re-read the following:
1st http://wiki2.dovecot.org/PasswordDatabase
2nd http://wiki2.dovecot.org/Authentication/Mechanisms
then edit /etc/dovecot/conf.d/10-auth.conf auth_mechanisms = plain login
On 05/04/16 19:00, C. Andrews Lavarre wrote:
Hello all. Thank you for your service.
Easy when you know how, but presently I do not. After literally months of research and experimentation we simply cannot log into our PAM / apache2 / postfix / dovecot pop3/imap STARTTLS email server with an ordinary email client, e.g., Evolution or Thunderbird.
We can connect to the host server in a host of different ways (no pun intended)—http, https, ssh, vnc, telnet, openssl -sclient
Similarly we can connect to postfix and dovecot in yet another number of ways—telnet, openssl -sclient—but cannot log in to the email server with a normal email client (either Evolution or Thunderbird) by either pop3 or imap.
SSL certificates are in place, verified, and tested.
Part of the problem is the many changes in all the involved operating systems and protocols (e.g., imaps and pop3s are deprecated, openSUSE has migrated to LEAP, etc.) so many of the docs from Google are no longer valid. Additionally, there simply are bugs: Leap 42.1 YAST does not work when it comes to setting up websites. Documented. But I digress.
I'm sure it's something really simple, but it evades me. Research details below. Any help would be more than appreciated.
Thanks in advance, Andy
======================= Configuration testing details
System is:
Linux openSUSE Leap 42.1
Dovecot --version 2.2.18, Postfix Version: 2.11.6-3.1 Apache2 Version: 2.4.16-9.1
Connections
- Evolution or Thunderbird to pop3 or imap reports:
The reported error was "Could not connect to mail.privustech.com: Connection refused".
Both connect successfully to googlemail.com with the same protocol: > > Port 993 SSL on a dedicated port
> > I have also tried > > > > Port 143 STARTTLS after connecting > > > > without success
> > 2. openssl s_client -connect mail.privustech.com:xxx > > > > > > a. xxx=25, 110, 143 all return > > > > > > > > error:140770FC
> > > > > > b. xxx=993, 995 return > > > > > > > > socket: Connection refused
> > > > > > connect:errno=111 > > > > > >
3.telnet to
a. smtp works.
b. pop3 andy@tm2t:~> telnet 70.186.159.22 110 > > ... > > +OK POP3 2007e.104 server ready < 48fa.572a0769@privustech.com> > > ... > > user andy > > -ERR Unknown AUTHORIZATION state command
c. > > > > imap connects but does not allow login, and should not. > > > > http://marc.info/?l=imap&m=118775891829506&w=2 > > > > > > > > The most simple answer is "you cannot TELNET to a modern, correctly-configured, > > > > > > > > IMAP server and log in to it." andy@tm2t:~> telnet 70.186.159.22 143 > > ... > > * OK [...] privustech.com IMAP4rev1 2007e.404 at Wed, 4 May 2016 10:26:28 > > -0400 (EDT) > > ... A NO Invalid login credentials > > Modules
• Apache2 works just fine. The server is up and answering. ping works just fine. We have http and https to all vhost sites (privustech, mailprivustech, nptbeyond, gvhl, truthcourage, and their www. subsites).
• Postfix reports no errors. We can log in on localhost, send a message to ourselves and see the message.
• Dovecot:
> > > > a. Logging is enabled in 10-logging.conf to /var/log/dovecot.conf but no logging has occurred there. > > > > > > b. doveconf -n throws no errors. > > > >
Checks and tests completed
- /etc/hosts is just fine.
> > > > 2. Firewall is open for telnet, postfix, dovecot.
> > 3. Added andy to dovecot, postfix groups, in addition to mail, reset password to ANDYbbs14@.
- We tried enabling imaps, pop3s, but this command returns errors about these protocols being obsolete.
> > > > > https://tools.ietf.org/html/rfc2595
> > Use of these ports is discouraged in favor of the STARTTLS or STLS commands.
> > > > 5. Reviewed doveconf -n: > > > > > > a. Note, there are no Dovecot users established
other than
> > user postfix > > group postfix > > > > service auth { > > > > unix_listener auth-userdb { > > > > > > group = postfix > > > > > > user = postfix > > > > } > > > > }
> > > > i. postfix has its own set of users, including andy, which works just fine within postfix. > > > > We can send mail and read mail in the mailbox.
b. Authentication is performed by PAM: > > passdb { > > driver = pam > > }
> > i. Examined PAM: > > > > A. The files /etc/pam.d/xxx, where xxx = dovecot, pop, imap, are all the same > > > > > > lavarre:~ # cat /etc/pam.d/xxx > > > > > > #%PAM-1.0 > > > > > > auth include common -auth > > > > > > account include common -account > > > > > > password include common -password > > > > > > session include common -session > > > > B. They do not resemble at all the form presented in > > > > http://wiki2.dovecot.org/PasswordDatabase/PAM > > > > > > > > passdb { > > > > > > > > driver = pam > > > > > > > > args = %s > > > > > > > > } > > > > C. Add (B.) to see if that works: No change. > > > > Comment out the original (A.): No change. > > > > Restore it. > > c. SSL is required and apparently configured correctly (the less-than symbol '<'causes the succeeding file to be read into the variable): > > ssl = required > > ssl_cert = > > ssl_dh_parameters_length = 2048 > > ssl_key = > > ssl_options = no_compression > > ssl_prefer_server_ciphers = yes > > userdb { > > driver = passwd > > }
> > > > i. dovecot.pem, both cert and key, are installed in /etc/ssl as above and verified as a pair with > > > > openssl x509. > > > > > > And we point to them in /etc/dovecot/conf.d/10-ssl.conf as seen in the above.
- Checked listening as it does not appear in doveconf -n:
lavarre:~ # doveconf protocols listen protocols = imap pop3 lmtp listen = *, ::
a. conf.d/10-master.conf > > ports for service xxx-login {inet_listener} are commented out. > > In fact, the entire file is commented out.
> > Uncomment the listeners, restart. But no change. So undo.
Edgar, thank you for your help:
i also noticed your certificate chain is broken.
http://wiki2.dovecot.org/testinstallation
i have worked through this link, thank you
changing permissions for /var/mail/* to
root:mail > > > > > > ($UID:$GID = 1000:12)
and then changing /etc/dovecot/users correspondingly to
user@privustech.com:{plain}actualpassword:1000:12::/var/ma il/vhosts/privustech/user
now allows a normal login. and we can create a new folder (e.g., work) and delete it.
If we set permissions to
vmail:vmail> > > > 100:5000
and update /etc/dovecot/users correspondingly it fails.
Regardless, despite following FindMailLocation neither we nor the system can find the mailbox:
b select inbox
- 0 EXISTS
- 0 RECENT
and postfix fails with 2016-07-23T21:22:37.312039-04:00 lavarre postfix/error[17088]: A8DA2C1BB2: to=andy@privustech.com, orig_to=, relay =none, delay=278572, delays=278271/300/0/0.04, dsn=4.4.2, status=deferred (delivery temporarily suspended: conversation with mail.privustech.com[private/lmtp] timed out while receiving the initial server greeting)
lmtp may be the culprit...
participants (2)
-
C. Andrews Lavarre
-
Edgar Pettijohn