Hello, I'm trying to work through an issue that cropped up on a server I've been working on and haven't found a very good workaround.
Dovecot is operating in a jailed environment.
The configuration in dovecot-sql.conf.ext has been set appropriately with the host=127.0.0.1 (which works from a jailed environment) and when dovecot attempts to auth it appears to perform a reverse dns lookup and overwrites the host with the result localhost before using the msql credentials 'user'@'localhost' which then fails after timing out.
I'm currently running version 2.2.33.2 of dovecot.
The workaround seems to be to have two credentials for the same user. 'user@'localhost' and 'user'@'127.0.0.1'; postfix operates in a jail as well and works around the jail issue in the same way but doesn't overwrite the specified host with a reverse dns lookup.
Anyone have any suggestions on how to handle this issue better? or maybe there's a way to force different behavior which I haven't yet found in the documentation?
Any help is much appreciated.
Best Regards.
Hello, I'm trying to work through an issue that cropped up on a server I've been working on and haven't found a very good workaround.
Dovecot is operating in a jailed environment.
The configuration in dovecot-sql.conf.ext has been set appropriately with the host=127.0.0.1 (which works from a jailed environment) and when dovecot attempts to auth it appears to perform a reverse dns lookup and overwrites the host with the result localhost before using the msql credentials 'user'@'localhost' which then fails after timing out.
I'm currently running version 2.2.33.2 of dovecot.
The workaround seems to be to have two credentials for the same user. 'user@'localhost' and 'user'@'127.0.0.1'; postfix operates in a jail as well and works around the jail issue in the same way but doesn't overwrite the specified host with a reverse dns lookup.
Anyone have any suggestions on how to handle this issue better? or maybe there's a way to force different behavior which I haven't yet found in the documentation?
Any help is much appreciated.
Best Regards.
Hi!
The 127.0.0.1 issue is actually a MySQL issue, unfortunately.
" On Unix, MySQL programs treat the host name localhost specially, in a way that is likely different from what you expect compared to other network-based programs: the client connects using a Unix socket file. The --socket option or the MYSQL_UNIX_PORT environment variable may be used to specify the socket name. "
So not really something we could do much about.
Aki
On 30/06/2019 13:36, Aki Tuomi via dovecot wrote:
Hello, I'm trying to work through an issue that cropped up on a server I've been working on and haven't found a very good workaround.
Dovecot is operating in a jailed environment.
The configuration in dovecot-sql.conf.ext has been set appropriately with the host=127.0.0.1 (which works from a jailed environment) and when dovecot attempts to auth it appears to perform a reverse dns lookup and overwrites the host with the result localhost before using the msql credentials 'user'@'localhost' which then fails after timing out.
I'm currently running version 2.2.33.2 of dovecot.
The workaround seems to be to have two credentials for the same user. 'user@'localhost' and 'user'@'127.0.0.1'; postfix operates in a jail as well and works around the jail issue in the same way but doesn't overwrite the specified host with a reverse dns lookup.
Anyone have any suggestions on how to handle this issue better? or maybe there's a way to force different behavior which I haven't yet found in the documentation?
Any help is much appreciated.
Best Regards.
Hi!
The 127.0.0.1 issue is actually a MySQL issue, unfortunately.
" On Unix, MySQL programs treat the host name localhost specially, in a way that is likely different from what you expect compared to other network-based programs: the client connects using a Unix socket file. The --socket option or the MYSQL_UNIX_PORT environment variable may be used to specify the socket name. "
So not really something we could do much about.
Aki
Aki
it is possible to force the use of tcp socket instead of unix socket by specifying an ip instead of the hostname localhost. As I understood it Nathan is specifying 127.0.0.1 so that will connect with tcp socket.
John
Actually, it seems I may have been wrong in initial assumption that the issue with the client was that it was being identified to mysql as coming from localhost when connecting via tcp. This is what syslog indicated as a reason for the failure but its not the whole picture.
As John mentioned I am trying to have dovecot connect over TCP to mysql (not using the socket), and the issue looked like the cause was the identified by portion of mysql being read by either mysql incorrectly or the domain portion being overwritten on dovecot's end (I don't know about the internals enough to say for sure where).
Just as due dilligence, I added credentials for a mysql user identified by localhost and removed the jail since the dovecot error was stating that it failed for connection by user@'localhost' (where there weren't credentials). After adding the credentials, I performed all the usual mysql tests before moving testing up to dovecot and still get an auth failure. The log seems to be a bit of a red herring or at the minimum doesn't show the whole picture.
Replacing the connection string host with the socket (host=localhost) and everything works, and using an external IP that's not 127.0.0.1 works as expected as well. (confirmed by standing up two isolated mysql and dovecot containers and setting auth up over the bridge).
If the issue was caused by user@'localhost' creating the credentials should have resolved it, and it didn't. So something weird is going on. I've got the environment built up in a dockerfile I can provide if anyone wants to dig into what's causing it.
In the meantime due to time constraints, I'll just be working with the socket file from now for hosts running most of the mail stack all in one.
Best Regards, Lorek.
On Sun, Jun 30, 2019 at 10:09 PM John Fawcett via dovecot < dovecot@dovecot.org> wrote:
On 30/06/2019 13:36, Aki Tuomi via dovecot wrote:
Hello, I'm trying to work through an issue that cropped up on a server
I've been working on and haven't found a very good workaround.
Dovecot is operating in a jailed environment.
The configuration in dovecot-sql.conf.ext has been set appropriately
with the host=127.0.0.1 (which works from a jailed environment) and when dovecot attempts to auth it appears to perform a reverse dns lookup and overwrites the host with the result localhost before using the msql credentials 'user'@'localhost' which then fails after timing out.
I'm currently running version 2.2.33.2 of dovecot.
The workaround seems to be to have two credentials for the same user. 'user@'localhost' and 'user'@'127.0.0.1'; postfix operates in a jail as well and works around the jail issue in
the same way but doesn't overwrite the specified host with a reverse dns lookup.
Anyone have any suggestions on how to handle this issue better? or
maybe there's a way to force different behavior which I haven't yet found in the documentation?
Any help is much appreciated.
Best Regards.
Hi!
The 127.0.0.1 issue is actually a MySQL issue, unfortunately.
" On Unix, MySQL programs treat the host name localhost specially, in a
way that is likely different from what you expect compared to other network-based programs: the client connects using a Unix socket file. The --socket option or the MYSQL_UNIX_PORT environment variable may be used to specify the socket name. "
So not really something we could do much about.
Aki
Aki
it is possible to force the use of tcp socket instead of unix socket by specifying an ip instead of the hostname localhost. As I understood it Nathan is specifying 127.0.0.1 so that will connect with tcp socket.
John
On 01/07/2019 09:48, lorek via dovecot wrote:
Actually, it seems I may have been wrong in initial assumption that the issue with the client was that it was being identified to mysql as coming from localhost when connecting via tcp. This is what syslog indicated as a reason for the failure but its not the whole picture.
As John mentioned I am trying to have dovecot connect over TCP to mysql (not using the socket), and the issue looked like the cause was the identified by portion of mysql being read by either mysql incorrectly or the domain portion being overwritten on dovecot's end (I don't know about the internals enough to say for sure where).
Just as due dilligence, I added credentials for a mysql user identified by localhost and removed the jail since the dovecot error was stating that it failed for connection by user@'localhost' (where there weren't credentials). After adding the credentials, I performed all the usual mysql tests before moving testing up to dovecot and still get an auth failure. The log seems to be a bit of a red herring or at the minimum doesn't show the whole picture.
Replacing the connection string host with the socket (host=localhost) and everything works, and using an external IP that's not 127.0.0.1 works as expected as well. (confirmed by standing up two isolated mysql and dovecot containers and setting auth up over the bridge).
If the issue was caused by user@'localhost' creating the credentials should have resolved it, and it didn't. So something weird is going on. I've got the environment built up in a dockerfile I can provide if anyone wants to dig into what's causing it.
In the meantime due to time constraints, I'll just be working with the socket file from now for hosts running most of the mail stack all in one.
Best Regards, Lorek.
Lorek
If you have user@localhost as a user in mysql you will be able to connect with either of these options:
a tcp socket via an ip address that resolves to localhost
a unix socket via hostname localhost.
If you have user@127.0.0.1 as a user in mysql you will be able to connect only via:
- a tcp socket using ip address 127.0.0.1.
John
John,
I was having an issue with the tcp socket @ 127.0.0.1. I'd tried various different modifications based on the logs and the issue had remained regardless. I had built up the environment in a dockerfile and worked on it through the weekend so the problem was replicable across instances up until this morning.
I ended up switching over to using the socket yesterday and only checked out to the previous commit this morning to run a few additional tests on the problematic part. For whatever reason, the issue is now completely gone.
At this point, I'm thinking this must be a docker issue.
The only notable (minor) differences between this past weekend and today are an AER/ASPM kernel error that was logging on the host. The paste is here: https://pastebin.com/idBWKDq2
The main difference between the weekend and today is the additional line included saying it can't find device of ID 0008. That wasn't showing up over the weekend. I think given this experience I may have to revisit my testing methodology when working with docker.
Best Regards, Lorek
On Tue, Jul 2, 2019 at 12:38 PM John Fawcett via dovecot < dovecot@dovecot.org> wrote:
On 01/07/2019 09:48, lorek via dovecot wrote:
Actually, it seems I may have been wrong in initial assumption that the issue with the client was that it was being identified to mysql as coming from localhost when connecting via tcp. This is what syslog indicated as a reason for the failure but its not the whole picture.
As John mentioned I am trying to have dovecot connect over TCP to mysql (not using the socket), and the issue looked like the cause was the identified by portion of mysql being read by either mysql incorrectly or the domain portion being overwritten on dovecot's end (I don't know about the internals enough to say for sure where).
Just as due dilligence, I added credentials for a mysql user identified by localhost and removed the jail since the dovecot error was stating that it failed for connection by user@'localhost' (where there weren't credentials). After adding the credentials, I performed all the usual mysql tests before moving testing up to dovecot and still get an auth failure. The log seems to be a bit of a red herring or at the minimum doesn't show the whole picture.
Replacing the connection string host with the socket (host=localhost) and everything works, and using an external IP that's not 127.0.0.1 works as expected as well. (confirmed by standing up two isolated mysql and dovecot containers and setting auth up over the bridge).
If the issue was caused by user@'localhost' creating the credentials should have resolved it, and it didn't. So something weird is going on. I've got the environment built up in a dockerfile I can provide if anyone wants to dig into what's causing it.
In the meantime due to time constraints, I'll just be working with the socket file from now for hosts running most of the mail stack all in one.
Best Regards, Lorek.
Lorek
If you have user@localhost as a user in mysql you will be able to connect with either of these options:
a tcp socket via an ip address that resolves to localhost
a unix socket via hostname localhost.
If you have user@127.0.0.1 as a user in mysql you will be able to connect only via:
- a tcp socket using ip address 127.0.0.1.
John
participants (4)
-
Aki Tuomi
-
John Fawcett
-
lorek
-
Nathan