IMAP preauth and stats-writer

older
BUG 'doveadm mailbox' list does...

Mark Hills

5 Jan 2019 5 Jan '19

4:49 p.m.

I use IMAP preauth; I connect with Alpine over SSH which is very useful.

The last few upgrades this has become more difficult to to. Last time (moving 2.2 -> 2.3, I think) I had to put in a workaround:

stats_writer_socket_path =

It prevented /usr/local/libexec/dovecot/imap attempting to connect to a central stats service.

As of an upgrade today (2.3.2.1_1 -> 2.3.4_3 on FreeBSD) it looks like that 'fix' stopped working, and I get:

imap(mark,)Error: net_connect_unix() failed: Permission denied

It goes to stderr, which breaks Alpine.

"()" is actually the filename. It seems that the empty string is no longer an indication to disable it. Here it is with the default configuration:

imap(mark,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied

I do also have a dovecot running as a system daemon, and, interestingly, disable this and it's 'fixed'; no attempt to connect. However, disabling the service is not an option (needed for smartphone)

The best I've come up with so far is when using preauth to hack it to send stderr to /dev/null. And yes, probably Alpine is at fault for interpreting stderr content (separate issue)

Is there a way to cleanly disable reporting to the stats service?
Previously, running as preauth was all very clean.
Just wanted to highlight that IMAP preauth is really useful. Even though it might not be mainstream, it seems healthy to be able to easily install dovecot as an unprivilidged user in a "unixy" way.

I'm on FreeBSD 11.2, with dovecot from ports. dovecot.conf below.

-- Mark

# # Dovecot configuration #

mail_location = maildir:~/Maildir postmaster_address = postmaster

namespace { inbox = yes }

ssl = required ssl_cert =

# # No writing of statistics to a priviledged service; # this is troublesome for a user to run the IMAP client #

#stats_writer_socket_path =

# # Authentication: only allow passwords sent over TLS, and # check against the system password database #

passdb { driver = pam }

userdb { driver = passwd }

# # Disable all except standard IMAP with TLS logins #

service imap-login { inet_listener imaps { port = 0 } }

service pop3-login { inet_listener pop3 { port = 0 }

inet_listener pop3s {
    port = 0
}

}

# # An authentication service to allow Exim to use the # same credentials as Dovecot #

service auth { unix_listener auth-client { mode = 0600 user = mailnull } }

protocol imap { mail_max_userip_connections = 40 }

# END

Show replies by date

John Fawcett

6 Jan 6 Jan

3:26 a.m.

On 05/01/2019 15:49, Mark Hills wrote:

...

I use IMAP preauth; I connect with Alpine over SSH which is very useful.

The last few upgrades this has become more difficult to to. Last time (moving 2.2 -> 2.3, I think) I had to put in a workaround:

stats_writer_socket_path =

It prevented /usr/local/libexec/dovecot/imap attempting to connect to a central stats service.

As of an upgrade today (2.3.2.1_1 -> 2.3.4_3 on FreeBSD) it looks like that 'fix' stopped working, and I get:

imap(mark,)Error: net_connect_unix() failed: Permission denied

It goes to stderr, which breaks Alpine.

"()" is actually the filename. It seems that the empty string is no longer an indication to disable it. Here it is with the default configuration:

imap(mark,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied

I do also have a dovecot running as a system daemon, and, interestingly, disable this and it's 'fixed'; no attempt to connect. However, disabling the service is not an option (needed for smartphone)

The best I've come up with so far is when using preauth to hack it to send stderr to /dev/null. And yes, probably Alpine is at fault for interpreting stderr content (separate issue)

Is there a way to cleanly disable reporting to the stats service?
Previously, running as preauth was all very clean.

Just wanted to highlight that IMAP preauth is really useful. Even though it might not be mainstream, it seems healthy to be able to easily install dovecot as an unprivilidged user in a "unixy" way.

I'm on FreeBSD 11.2, with dovecot from ports. dovecot.conf below.

Can't see anything in the Dovecot 2.3.4 code that would give this problem, setting

stats_writer_socket_path =

will overwrite the default value and dovecot does not attempt to open a socket in that case.

Using your configuration (though not FreeBSD) I don't get the net_connect_unix error whether I use a blank setting or leave the default. In the case of leaving the default I do get an additional process (dovecot/stats). No errors on connecting to the imap service or by running preauth (with the dovecot daemon already running).

The net_connect_unix() error with a zero length socket name is inexplicable to me, unless it's got a non printing character in it or there is something different happening on FreeBSD.

One suggestion is to run with the default setting, but look at resolving the permission problem for the default socket creation at /var/run/dovecot/stats-writer rather than working round it.

John

John Fawcett

4:35 a.m.

On 06/01/2019 02:26, John Fawcett wrote:

...

On 05/01/2019 15:49, Mark Hills wrote:

...
I use IMAP preauth; I connect with Alpine over SSH which is very useful.

The last few upgrades this has become more difficult to to. Last time (moving 2.2 -> 2.3, I think) I had to put in a workaround:

stats_writer_socket_path =

It prevented /usr/local/libexec/dovecot/imap attempting to connect to a central stats service.

As of an upgrade today (2.3.2.1_1 -> 2.3.4_3 on FreeBSD) it looks like that 'fix' stopped working, and I get:

imap(mark,)Error: net_connect_unix() failed: Permission denied

It goes to stderr, which breaks Alpine.

"()" is actually the filename. It seems that the empty string is no longer an indication to disable it. Here it is with the default configuration:

imap(mark,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied

I do also have a dovecot running as a system daemon, and, interestingly, disable this and it's 'fixed'; no attempt to connect. However, disabling the service is not an option (needed for smartphone)

The best I've come up with so far is when using preauth to hack it to send stderr to /dev/null. And yes, probably Alpine is at fault for interpreting stderr content (separate issue)

Is there a way to cleanly disable reporting to the stats service?
Previously, running as preauth was all very clean.

Just wanted to highlight that IMAP preauth is really useful. Even though it might not be mainstream, it seems healthy to be able to easily install dovecot as an unprivilidged user in a "unixy" way.

I'm on FreeBSD 11.2, with dovecot from ports. dovecot.conf below.

Can't see anything in the Dovecot 2.3.4 code that would give this problem, setting

stats_writer_socket_path =

will overwrite the default value and dovecot does not attempt to open a socket in that case.

Using your configuration (though not FreeBSD) I don't get the net_connect_unix error whether I use a blank setting or leave the default. In the case of leaving the default I do get an additional process (dovecot/stats). No errors on connecting to the imap service or by running preauth (with the dovecot daemon already running).

The net_connect_unix() error with a zero length socket name is inexplicable to me, unless it's got a non printing character in it or there is something different happening on FreeBSD.

One suggestion is to run with the default setting, but look at resolving the permission problem for the default socket creation at /var/run/dovecot/stats-writer rather than working round it.

John

Just following up, I don't get the error when I run preauth as root with -u parameter. I do get something similar when I run as the user (this wil the socket path set to blank)

Centos 7:

Error: net_connect_unix() failed: Connection refused

FreeBSD 11.2:

Error: net_connect_unix() failed: No such file or directory

So it's close. If I get time I'll see if I can track it down.

John

Stephan Bosch

12:37 p.m.

Op 06/01/2019 om 03:35 schreef John Fawcett:

...

On 06/01/2019 02:26, John Fawcett wrote:

...
Can't see anything in the Dovecot 2.3.4 code that would give this problem, setting

stats_writer_socket_path =

will overwrite the default value and dovecot does not attempt to open a socket in that case.

Using your configuration (though not FreeBSD) I don't get the net_connect_unix error whether I use a blank setting or leave the default. In the case of leaving the default I do get an additional process (dovecot/stats). No errors on connecting to the imap service or by running preauth (with the dovecot daemon already running).

The net_connect_unix() error with a zero length socket name is inexplicable to me, unless it's got a non printing character in it or there is something different happening on FreeBSD.

One suggestion is to run with the default setting, but look at resolving the permission problem for the default socket creation at /var/run/dovecot/stats-writer rather than working round it.

John

Just following up, I don't get the error when I run preauth as root with -u parameter. I do get something similar when I run as the user (this wil the socket path set to blank)

Centos 7:

Error: net_connect_unix() failed: Connection refused

FreeBSD 11.2:

Error: net_connect_unix() failed: No such file or directory

So it's close. If I get time I'll see if I can track it down.

Does this fix it? diff --git a/src/lib-master/master-service.c b/src/lib-master/master-service.c index 3de11fa1b..3c60a7a39 100644 --- a/src/lib-master/master-service.c +++ b/src/lib-master/master-service.c @@ -341,7 +341,7 @@ master_service_init(const char *name, enum master_service_flags flags, if ((flags & MASTER_SERVICE_FLAG_DONT_SEND_STATS) == 0) { /* Initialize stats-client early so it can see all events. */ value = getenv(DOVECOT_STATS_WRITER_SOCKET_PATH); - if (value != NULL) + if (value != NULL && *value != '\0') service->stats_client = stats_client_init(value, FALSE); } Regards, Stephan.

John Fawcett

7:12 p.m.

On 06/01/2019 11:37, Stephan Bosch wrote:

...

Op 06/01/2019 om 03:35 schreef John Fawcett:

...
On 06/01/2019 02:26, John Fawcett wrote:

...
Can't see anything in the Dovecot 2.3.4 code that would give this problem, setting

stats_writer_socket_path =

will overwrite the default value and dovecot does not attempt to open a socket in that case.

Using your configuration (though not FreeBSD) I don't get the net_connect_unix error whether I use a blank setting or leave the default. In the case of leaving the default I do get an additional process (dovecot/stats). No errors on connecting to the imap service or by running preauth (with the dovecot daemon already running).

The net_connect_unix() error with a zero length socket name is inexplicable to me, unless it's got a non printing character in it or there is something different happening on FreeBSD.

One suggestion is to run with the default setting, but look at resolving the permission problem for the default socket creation at /var/run/dovecot/stats-writer rather than working round it.

John

Just following up, I don't get the error when I run preauth as root with -u parameter. I do get something similar when I run as the user (this wil the socket path set to blank)

Centos 7:

Error: net_connect_unix() failed: Connection refused

FreeBSD 11.2:

Error: net_connect_unix() failed: No such file or directory

So it's close. If I get time I'll see if I can track it down.

Does this fix it?

diff --git a/src/lib-master/master-service.c b/src/lib-master/master-service.c index 3de11fa1b..3c60a7a39 100644 --- a/src/lib-master/master-service.c +++ b/src/lib-master/master-service.c @@ -341,7 +341,7 @@ master_service_init(const char *name, enum master_service_flags flags, if ((flags & MASTER_SERVICE_FLAG_DONT_SEND_STATS) == 0) { /* Initialize stats-client early so it can see all events. */ value = getenv(DOVECOT_STATS_WRITER_SOCKET_PATH); - if (value != NULL) + if (value != NULL && *value != '\0') service->stats_client = stats_client_init(value, FALSE); }

Regards,

Stephan.

Hi Stephan that fixes the issue that I reproduced. For the OP he will probably need to wait for this to be picked up for FreeBSD ports. This code seems safer than the original but it is still a mystery as to why DOVECOT_STATS_WRITER_SOCKET_PATH is being put into the environment as an empty string (changed behaviour reported by OP compared to 2.3.2.1_1). The function that is doing the env_put call with the empty string is config_request_putenv from src/config/doveconf.c. John

Stephan Bosch

7:39 p.m.

Op 06/01/2019 om 18:12 schreef John Fawcett:

...

On 06/01/2019 11:37, Stephan Bosch wrote:

...
Op 06/01/2019 om 03:35 schreef John Fawcett:

...
On 06/01/2019 02:26, John Fawcett wrote:

...
Can't see anything in the Dovecot 2.3.4 code that would give this problem, setting

stats_writer_socket_path =

will overwrite the default value and dovecot does not attempt to open a socket in that case.

Using your configuration (though not FreeBSD) I don't get the net_connect_unix error whether I use a blank setting or leave the default. In the case of leaving the default I do get an additional process (dovecot/stats). No errors on connecting to the imap service or by running preauth (with the dovecot daemon already running).

The net_connect_unix() error with a zero length socket name is inexplicable to me, unless it's got a non printing character in it or there is something different happening on FreeBSD.

One suggestion is to run with the default setting, but look at resolving the permission problem for the default socket creation at /var/run/dovecot/stats-writer rather than working round it.

John

Just following up, I don't get the error when I run preauth as root with -u parameter. I do get something similar when I run as the user (this wil the socket path set to blank)

Centos 7:

Error: net_connect_unix() failed: Connection refused

FreeBSD 11.2:

Error: net_connect_unix() failed: No such file or directory

So it's close. If I get time I'll see if I can track it down. Does this fix it?

diff --git a/src/lib-master/master-service.c b/src/lib-master/master-service.c index 3de11fa1b..3c60a7a39 100644 --- a/src/lib-master/master-service.c +++ b/src/lib-master/master-service.c @@ -341,7 +341,7 @@ master_service_init(const char *name, enum master_service_flags flags, if ((flags & MASTER_SERVICE_FLAG_DONT_SEND_STATS) == 0) { /* Initialize stats-client early so it can see all events. */ value = getenv(DOVECOT_STATS_WRITER_SOCKET_PATH); - if (value != NULL) + if (value != NULL && *value != '\0') service->stats_client = stats_client_init(value, FALSE); }

Regards,

Stephan.

Hi Stephan

that fixes the issue that I reproduced. For the OP he will probably need to wait for this to be picked up for FreeBSD ports.

This code seems safer than the original but it is still a mystery as to why DOVECOT_STATS_WRITER_SOCKET_PATH is being put into the environment as an empty string (changed behaviour reported by OP compared to 2.3.2.1_1).

Behavior changed because the code I patched didn't exist in 2.3.2.1. Regards, Stephan.

Larry Rosenman

7:47 p.m.

I'm the FreeBSD port maintainer for dovecot and I just added this patch to the port in SVN. 489515. On Sun, Jan 6, 2019 at 11:39 AM Stephan Bosch wrote:

...

Op 06/01/2019 om 18:12 schreef John Fawcett:

...
On 06/01/2019 11:37, Stephan Bosch wrote:

...
Op 06/01/2019 om 03:35 schreef John Fawcett:

...
On 06/01/2019 02:26, John Fawcett wrote:

...
Can't see anything in the Dovecot 2.3.4 code that would give this problem, setting

stats_writer_socket_path =

will overwrite the default value and dovecot does not attempt to open a socket in that case.

Using your configuration (though not FreeBSD) I don't get the net_connect_unix error whether I use a blank setting or leave the default. In the case of leaving the default I do get an additional process (dovecot/stats). No errors on connecting to the imap service or by running preauth (with the dovecot daemon already running).

The net_connect_unix() error with a zero length socket name is inexplicable to me, unless it's got a non printing character in it or there is something different happening on FreeBSD.

One suggestion is to run with the default setting, but look at resolving the permission problem for the default socket creation at /var/run/dovecot/stats-writer rather than working round it.

John

Just following up, I don't get the error when I run preauth as root with -u parameter. I do get something similar when I run as the user (this wil the socket path set to blank)

Centos 7:

Error: net_connect_unix() failed: Connection refused

FreeBSD 11.2:

Error: net_connect_unix() failed: No such file or directory

So it's close. If I get time I'll see if I can track it down. Does this fix it?

diff --git a/src/lib-master/master-service.c b/src/lib-master/master-service.c index 3de11fa1b..3c60a7a39 100644 --- a/src/lib-master/master-service.c +++ b/src/lib-master/master-service.c @@ -341,7 +341,7 @@ master_service_init(const char *name, enum master_service_flags flags, if ((flags & MASTER_SERVICE_FLAG_DONT_SEND_STATS) == 0) { /* Initialize stats-client early so it can see all events. */ value = getenv(DOVECOT_STATS_WRITER_SOCKET_PATH); - if (value != NULL) + if (value != NULL && *value != '\0') service->stats_client = stats_client_init(value, FALSE); }

Regards,

Stephan.

Hi Stephan

that fixes the issue that I reproduced. For the OP he will probably need to wait for this to be picked up for FreeBSD ports.

This code seems safer than the original but it is still a mystery as to why DOVECOT_STATS_WRITER_SOCKET_PATH is being put into the environment as an empty string (changed behaviour reported by OP compared to 2.3.2.1_1).

Behavior changed because the code I patched didn't exist in 2.3.2.1.

Regards,

Stephan.

-- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx@gmail.com US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

Larry Rosenman

8:09 p.m.

and stupid me missed a character, the full fix is in SVN r489516. On Sun, Jan 6, 2019 at 11:47 AM Larry Rosenman wrote:

...

I'm the FreeBSD port maintainer for dovecot and I just added this patch to the port in SVN. 489515.

On Sun, Jan 6, 2019 at 11:39 AM Stephan Bosch wrote:

...
Op 06/01/2019 om 18:12 schreef John Fawcett:

...
On 06/01/2019 11:37, Stephan Bosch wrote:

...
Op 06/01/2019 om 03:35 schreef John Fawcett:

...
On 06/01/2019 02:26, John Fawcett wrote:

...
Can't see anything in the Dovecot 2.3.4 code that would give this problem, setting

stats_writer_socket_path =

will overwrite the default value and dovecot does not attempt to open a socket in that case.

Using your configuration (though not FreeBSD) I don't get the net_connect_unix error whether I use a blank setting or leave the default. In the case of leaving the default I do get an additional process (dovecot/stats). No errors on connecting to the imap service or by running preauth (with the dovecot daemon already running).

The net_connect_unix() error with a zero length socket name is inexplicable to me, unless it's got a non printing character in it or there is something different happening on FreeBSD.

One suggestion is to run with the default setting, but look at resolving the permission problem for the default socket creation at /var/run/dovecot/stats-writer rather than working round it.

John

Just following up, I don't get the error when I run preauth as root with -u parameter. I do get something similar when I run as the user (this wil the socket path set to blank)

Centos 7:

Error: net_connect_unix() failed: Connection refused

FreeBSD 11.2:

Error: net_connect_unix() failed: No such file or directory

So it's close. If I get time I'll see if I can track it down. Does this fix it?

diff --git a/src/lib-master/master-service.c b/src/lib-master/master-service.c index 3de11fa1b..3c60a7a39 100644 --- a/src/lib-master/master-service.c +++ b/src/lib-master/master-service.c @@ -341,7 +341,7 @@ master_service_init(const char *name, enum master_service_flags flags, if ((flags & MASTER_SERVICE_FLAG_DONT_SEND_STATS) == 0) { /* Initialize stats-client early so it can see all events. */ value = getenv(DOVECOT_STATS_WRITER_SOCKET_PATH); - if (value != NULL) + if (value != NULL && *value != '\0') service->stats_client = stats_client_init(value, FALSE); }

Regards,

Stephan.

Hi Stephan

that fixes the issue that I reproduced. For the OP he will probably need to wait for this to be picked up for FreeBSD ports.

This code seems safer than the original but it is still a mystery as to why DOVECOT_STATS_WRITER_SOCKET_PATH is being put into the environment as an empty string (changed behaviour reported by OP compared to 2.3.2.1_1).

Behavior changed because the code I patched didn't exist in 2.3.2.1.

Regards,

Stephan.

-- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx@gmail.com US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

-- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx@gmail.com US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

Mark Hills

17 Jan 17 Jan

12:56 a.m.

Hi, many thanks for the quick replies and patch. So quick that I'm not able to respond in the same timeframe.

Yes, I am working with FreeBSD pre-built packages; I still intend check out the ports from svn or do my own build, but have not had time yet.

To answer specific question:

On Sun, 6 Jan 2019, John Fawcett wrote:

...

One suggestion is to run with the default setting, but look at resolving the permission problem for the default socket creation at /var/run/dovecot/stats-writer rather than working round it.

Potentially, though it's less logical that the daemon dovecot (which runs as a system user / priviledged) should not be accepting stats from an unpriviledged source.

Yes, one could seek to get stats for all dovecot activity on the server. Assuming clients are trusted to feed valid stats and not cause some horrible DoS or similar.

So I am heading for the opoosite where dovecot is really the self-contained 'imap' command without crossing any priviledge boundary; just a process consuing CPU and RAM resources like any other. To date I haven't used any of the dovecot stats (not dismissing them, just haven't had a cause to)

Thanks again

-- Mark

Stephan Bosch

20 Jan 20 Jan

1:43 p.m.

Op 06/01/2019 om 18:12 schreef John Fawcett:

...

On 06/01/2019 11:37, Stephan Bosch wrote:

...
Op 06/01/2019 om 03:35 schreef John Fawcett:

...
On 06/01/2019 02:26, John Fawcett wrote:

...
Can't see anything in the Dovecot 2.3.4 code that would give this problem, setting

stats_writer_socket_path =

will overwrite the default value and dovecot does not attempt to open a socket in that case.

Using your configuration (though not FreeBSD) I don't get the net_connect_unix error whether I use a blank setting or leave the default. In the case of leaving the default I do get an additional process (dovecot/stats). No errors on connecting to the imap service or by running preauth (with the dovecot daemon already running).

The net_connect_unix() error with a zero length socket name is inexplicable to me, unless it's got a non printing character in it or there is something different happening on FreeBSD.

One suggestion is to run with the default setting, but look at resolving the permission problem for the default socket creation at /var/run/dovecot/stats-writer rather than working round it.

John

Just following up, I don't get the error when I run preauth as root with -u parameter. I do get something similar when I run as the user (this wil the socket path set to blank)

Centos 7:

Error: net_connect_unix() failed: Connection refused

FreeBSD 11.2:

Error: net_connect_unix() failed: No such file or directory

So it's close. If I get time I'll see if I can track it down. Does this fix it?

diff --git a/src/lib-master/master-service.c b/src/lib-master/master-service.c index 3de11fa1b..3c60a7a39 100644 --- a/src/lib-master/master-service.c +++ b/src/lib-master/master-service.c @@ -341,7 +341,7 @@ master_service_init(const char *name, enum master_service_flags flags, if ((flags & MASTER_SERVICE_FLAG_DONT_SEND_STATS) == 0) { /* Initialize stats-client early so it can see all events. */ value = getenv(DOVECOT_STATS_WRITER_SOCKET_PATH); - if (value != NULL) + if (value != NULL && *value != '\0') service->stats_client = stats_client_init(value, FALSE); }

Regards,

Stephan.

Hi Stephan

that fixes the issue that I reproduced. For the OP he will probably need to wait for this to be picked up for FreeBSD ports.

This code seems safer than the original but it is still a mystery as to why DOVECOT_STATS_WRITER_SOCKET_PATH is being put into the environment as an empty string (changed behaviour reported by OP compared to 2.3.2.1_1).

The function that is doing the env_put call with the empty string is config_request_putenv from src/config/doveconf.c.

Tracked internally as DOP-838. Regards, Stephan.

2081

Age (days ago)

2096

Last active (days ago)

List overview

9 comments

4 participants

participants (4)

John Fawcett
Larry Rosenman
Mark Hills
Stephan Bosch

IMAP preauth and stats-writer

tags

participants (4)