From pigeonhole at rename-it.nl Wed Jul 3 23:05:20 2013 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Wed, 03 Jul 2013 22:05:20 +0200 Subject: dovecot-2.2-pigeonhole: Fixed deprecated INCLUDE= assignment in ... Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/53ee9655e78e changeset: 1783:53ee9655e78e user: Stephan Bosch date: Tue Jun 18 22:04:50 2013 +0200 description: Fixed deprecated INCLUDE= assignment in Makefile.am diffstat: src/lib-sievestorage/Makefile.am | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (11 lines): diff -r d4ae981567e1 -r 53ee9655e78e src/lib-sievestorage/Makefile.am --- a/src/lib-sievestorage/Makefile.am Fri May 24 13:07:23 2013 +0200 +++ b/src/lib-sievestorage/Makefile.am Tue Jun 18 22:04:50 2013 +0200 @@ -1,6 +1,6 @@ noinst_LIBRARIES = libsievestorage.a -INCLUDES = \ +AM_CPPFLAGS = \ $(LIBDOVECOT_INCLUDE) \ -I$(top_srcdir) \ -I$(top_srcdir)/src/lib-sieve From pigeonhole at rename-it.nl Wed Jul 3 23:05:21 2013 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Wed, 03 Jul 2013 22:05:21 +0200 Subject: dovecot-2.2-pigeonhole: Released v0.4.1 for Dovecot v2.2.4. Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/f1535e2255cd changeset: 1785:f1535e2255cd user: Stephan Bosch date: Wed Jul 03 22:05:04 2013 +0200 description: Released v0.4.1 for Dovecot v2.2.4. diffstat: NEWS | 17 +++++++++++++++++ configure.ac | 2 +- 2 files changed, 18 insertions(+), 1 deletions(-) diffs (33 lines): diff -r 597c1dcbda11 -r f1535e2255cd NEWS --- a/NEWS Wed Jul 03 21:35:12 2013 +0200 +++ b/NEWS Wed Jul 03 22:05:04 2013 +0200 @@ -1,3 +1,20 @@ +v0.4.1 03-06-2013 Stephan Bosch + + + Added support for handling temporary failures. These are passed back to + LDA/LTMP to produce an appropriate response towards the MTA. + - Sieve storage: Removed PATH_MAX limitation for active symlink. This caused + problems for GNU/Hurd. + - Fixed line endings in X-Sieve headers added by redirect command. + - ManageSieve: Fixed '[' ']' stupidity for response codes (only happened + before login). + - Fixed setting name in example-config/conf.d/20-managesieve.conf. + - Sieve extprograms plugin: Fixed interaction between pipe command and remote + script service. The output from the script service was never read, causing a + broken pipe error at the script service. Apparently, this was broken since + the I/O handling for extprograms was last revised. + - Fixed assertion failure due to datastack problem in message header + composition. + v0.4.0 09-05-2013 Stephan Bosch + Added doveadm-sieve plugin that provides the possibility to synch Sieve diff -r 597c1dcbda11 -r f1535e2255cd configure.ac --- a/configure.ac Wed Jul 03 21:35:12 2013 +0200 +++ b/configure.ac Wed Jul 03 22:05:04 2013 +0200 @@ -1,4 +1,4 @@ -AC_INIT([Pigeonhole], [0.4.0], [dovecot at dovecot.org], [dovecot-2.2-pigeonhole]) +AC_INIT([Pigeonhole], [0.4.1], [dovecot at dovecot.org], [dovecot-2.2-pigeonhole]) AC_CONFIG_AUX_DIR([.]) AC_CONFIG_SRCDIR([src]) AC_CONFIG_MACRO_DIR([m4]) From pigeonhole at rename-it.nl Wed Jul 3 23:05:21 2013 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Wed, 03 Jul 2013 22:05:21 +0200 Subject: dovecot-2.2-pigeonhole: Merged changes from Pigeonhole v0.3 tree. Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/597c1dcbda11 changeset: 1784:597c1dcbda11 user: Stephan Bosch date: Wed Jul 03 21:35:12 2013 +0200 description: Merged changes from Pigeonhole v0.3 tree. diffstat: src/lib-sievestorage/Makefile.am | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (11 lines): diff -r 1b1a0c271383 -r 597c1dcbda11 src/lib-sievestorage/Makefile.am --- a/src/lib-sievestorage/Makefile.am Tue Jun 18 08:34:26 2013 +0200 +++ b/src/lib-sievestorage/Makefile.am Wed Jul 03 21:35:12 2013 +0200 @@ -1,6 +1,6 @@ noinst_LTLIBRARIES = libsievestorage.la -INCLUDES = \ +AM_CPPFLAGS = \ $(LIBDOVECOT_INCLUDE) \ -I$(top_srcdir) \ -I$(top_srcdir)/src/lib-sieve From pigeonhole at rename-it.nl Wed Jul 3 23:05:21 2013 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Wed, 03 Jul 2013 22:05:21 +0200 Subject: dovecot-2.2-pigeonhole: Added tag 0.4.1 for changeset f1535e2255cd Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/931f00c28603 changeset: 1786:931f00c28603 user: Stephan Bosch date: Wed Jul 03 22:05:13 2013 +0200 description: Added tag 0.4.1 for changeset f1535e2255cd diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r f1535e2255cd -r 931f00c28603 .hgtags --- a/.hgtags Wed Jul 03 22:05:04 2013 +0200 +++ b/.hgtags Wed Jul 03 22:05:13 2013 +0200 @@ -16,3 +16,4 @@ 4932026768454443d87d2e6445552b331589dbb1 0.3.4 374ec93999588b1acc554f81f61c93bf9fad7037 0.3.5 3a8dc1250e9b850044acbcd9d63d63597e67c7cb 0.4.0 +f1535e2255cd84849d8b4fad6d82bdd6e01f810f 0.4.1 From pigeonhole at rename-it.nl Wed Jul 3 23:07:10 2013 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Wed, 03 Jul 2013 22:07:10 +0200 Subject: dovecot-2.2-pigeonhole: Added signature for changeset f1535e2255cd Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/612d91d3b745 changeset: 1787:612d91d3b745 user: Stephan Bosch date: Wed Jul 03 22:06:58 2013 +0200 description: Added signature for changeset f1535e2255cd diffstat: .hgsigs | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 931f00c28603 -r 612d91d3b745 .hgsigs --- a/.hgsigs Wed Jul 03 22:05:13 2013 +0200 +++ b/.hgsigs Wed Jul 03 22:06:58 2013 +0200 @@ -10,3 +10,4 @@ 4932026768454443d87d2e6445552b331589dbb1 0 iQEcBAABAgAGBQJRYIePAAoJEATWKx49+7T0p2oH/ROhjt/m+wZmT9+2NxEDwnaOoQ8m9TxkZiZ50mmi/k4L7OQe/xffxM2T3NTTgRkaLCK2+MEz0pSLJXL+n/AjTBtiynwSxYY+W0wtYKBIs0tcQHaSafN2u5LjtQZ2RHg+Fi1szhJQu/jy31w12KGTqdyVw05JuEPgyfM7fKpqKh8CQZJucEyn9Vf8boGPQMFJ7N6o4wpOeW8RuVcBAEToHMpkDI1OQmB22cEQJfZrdOMPaucUOG6Abfw0FDwwV4bHa1nmxiPZXF8DpW31SVDoZgyvi08TSWFHS9t8Pij+XyWIdXhbCzqdpkvLGEsvMJjhiro9agf2tSj9jP+D4xiFRe8= 374ec93999588b1acc554f81f61c93bf9fad7037 0 iQEcBAABAgAGBQJRi6LgAAoJEATWKx49+7T0OPgH+gPOOYqdODGONhZTwxjKmvbVNzb428t/fS/pi2bSIeOXVzLone/P7NYstdM67eQsfxY4Kn8qN/eFkjdkozbc+3xx0VFvptFTd9NXGM57O84umgrvKXQDQM333TVMxMG5u6TPPkCmFcWmggcc1IdQxZ66dpgfETdh+IgJXdLQg4oc0IqHtqx19dVyYXRbtgY+GfA9ovFqOkzcWoXgSHRMmUf2gCGWPc3jSZPVHd3ObRvrwn3NcWULK4WCt5QPc6fU53eDrEvkSu6C5U0IV5G3jakrUK0PYtQ3jvxXGm8J3XQ+ea79Bp+mG/21Woz30Ey6AlFbuAHobGesGafjPLDLSNs= 3a8dc1250e9b850044acbcd9d63d63597e67c7cb 0 iQEcBAABAgAGBQJRi6eKAAoJEATWKx49+7T0+3sIAKj87rGC9kakJPL4RyFDc762HueCez8mUTRhe62K9+2LzVJL+2inGPY33PL5F7xZd0H/pxIfNfk/WGA78q6BkOaJSlGtz+wb5H/iZzSc+6GRfzPavlXlENgOKtnmzf3AoEKc0iNnieB5oYJTkJ2GtbPv2GD4u/Y6w7eTISfh/HlJeGvybR3L4XT6scNCr90/iQWjoZMNZSbYvKInSmrN9SGgtg0exqFfdkFmMahhSO/PGBq4hi5ZyFKqsnYEZpkLOduKW86nsn1GZiPD5/bEUD7lxJeeoRk3aobZoMQUlz4VRma6ag0MPRLpmzqUQFhpvDzpdg3dZSw/8NUQBa78w1M= +f1535e2255cd84849d8b4fad6d82bdd6e01f810f 0 iQEcBAABAgAGBQJR1IRdAAoJEATWKx49+7T0uKMIANauBvbSO72bhrg8EI3Cvr2qy5nZrgqdgtZI0ZtIHbRBa9aBmxbX5lGqjWNLZaXwLc9dk7aeDoZds73kjiCYB+ac3UaXBaRscKNArkigTKFE0L2UZ5X5T/76iJ6diPFs8Eo/4ajYERDa3UQgUqHJ/8Fe2884bFDXMRmjbtVu2ceCw8Dlkee4kiVfjzuPRWud2jjtXcCoJXkFIQSbGcKsjlhd2yAGToJ9iw06tPNAFjupht9/wozyaZiZqJj7AnNvflHGMO1FDhOQJYxjz9JYA61h9F1vytGwfm1n3BVANpWSBJtkkuM/Tjp9GxeDB2jNrUbMikicSUceNtRLg9i0b+A= From dovecot at dovecot.org Wed Jul 10 01:45:51 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 01:45:51 +0300 Subject: dovecot-2.2: lib-http: Always try to connect to host's all IPs w... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/dee7822501b0 changeset: 16577:dee7822501b0 user: Timo Sirainen date: Wed Jul 10 01:44:40 2013 +0300 description: lib-http: Always try to connect to host's all IPs when connections fail. Previously this was done only when the new connections started from the first IP. diffstat: src/lib-http/http-client-host.c | 44 +++++++++++++++++++++++++++++++------ src/lib-http/http-client-private.h | 4 +++ 2 files changed, 41 insertions(+), 7 deletions(-) diffs (121 lines): diff -r 1fbac590b9d4 -r dee7822501b0 src/lib-http/http-client-host.c --- a/src/lib-http/http-client-host.c Fri Jun 28 19:48:37 2013 +0300 +++ b/src/lib-http/http-client-host.c Wed Jul 10 01:44:40 2013 +0300 @@ -114,6 +114,20 @@ } } +static bool +http_client_hport_is_last_connect_ip(struct http_client_host_port *hport) +{ + i_assert(hport->ips_connect_idx < hport->host->ips_count); + i_assert(hport->ips_connect_start_idx < hport->host->ips_count); + + /* we'll always go through all the IPs. we don't necessarily start + connecting from the first IP, so we'll need to treat the IPs as + a ring buffer where we automatically wrap back to the first IP + when necessary. */ + return (hport->ips_connect_idx + 1) % hport->host->ips_count == + hport->ips_connect_start_idx; +} + static void http_client_host_port_soft_connect_timeout(struct http_client_host_port *hport) { @@ -122,7 +136,7 @@ if (hport->to_connect != NULL) timeout_remove(&hport->to_connect); - if (hport->ips_connect_idx + 1 >= host->ips_count) + if (http_client_hport_is_last_connect_ip(hport)) return; /* if our our previous connection attempt takes longer than the @@ -135,7 +149,7 @@ hport->https_name == NULL ? "" : t_strdup_printf(" (SSL=%s)", hport->https_name)); - hport->ips_connect_idx++; + hport->ips_connect_idx = (hport->ips_connect_idx + 1) % host->ips_count; http_client_host_port_connection_setup(hport); } @@ -161,7 +175,7 @@ /* start soft connect time-out (but only if we have another IP left) */ msecs = host->client->set.soft_connect_timeout_msecs; - if (host->ips_count - hport->ips_connect_idx > 1 && msecs > 0 && + if (!http_client_hport_is_last_connect_ip(hport) && msecs > 0 && hport->to_connect == NULL) { hport->to_connect = timeout_add(msecs, http_client_host_port_soft_connect_timeout, hport); @@ -200,11 +214,26 @@ } } +static unsigned int +http_client_host_get_ip_idx(struct http_client_host *host, + const struct ip_addr *ip) +{ + unsigned int i; + + for (i = 0; i < host->ips_count; i++) { + if (net_ip_compare(&host->ips[i], ip)) + return i; + } + i_unreached(); +} + static void http_client_host_port_connection_success(struct http_client_host_port *hport, const struct http_client_peer_addr *addr) { /* we achieved at least one connection the the addr->ip */ + hport->ips_connect_start_idx = + http_client_host_get_ip_idx(hport->host, &addr->ip); /* stop soft connect time-out */ if (hport->to_connect != NULL) @@ -237,15 +266,16 @@ if (hport->to_connect != NULL) timeout_remove(&hport->to_connect); - i_assert(hport->ips_connect_idx < host->ips_count); - if (++hport->ips_connect_idx == host->ips_count) { + if (http_client_hport_is_last_connect_ip(hport)) { /* all IPs failed, but retry all of them again on the next request. */ - hport->ips_connect_idx = 0; + hport->ips_connect_idx = hport->ips_connect_start_idx = + (hport->ips_connect_idx + 1) % host->ips_count; http_client_host_port_error(hport, HTTP_CLIENT_REQUEST_ERROR_CONNECT_FAILED, reason); return FALSE; } + hport->ips_connect_idx = (hport->ips_connect_idx + 1) % host->ips_count; http_client_host_port_connection_setup(hport); return TRUE; @@ -329,7 +359,7 @@ /* make connections to requested ports */ array_foreach_modifiable(&host->ports, hport) { unsigned int count = array_count(&hport->request_queue); - hport->ips_connect_idx = 0; + hport->ips_connect_idx = hport->ips_connect_start_idx = 0; if (count > 0) http_client_host_port_connection_setup(hport); requests += count; diff -r 1fbac590b9d4 -r dee7822501b0 src/lib-http/http-client-private.h --- a/src/lib-http/http-client-private.h Fri Jun 28 19:48:37 2013 +0300 +++ b/src/lib-http/http-client-private.h Wed Jul 10 01:44:40 2013 +0300 @@ -78,6 +78,10 @@ /* current index in host->ips */ unsigned int ips_connect_idx; + /* the first IP that started the current round of connection attempts. + initially 0, and later set to the (ip_idx+1) of the last successful + connected IP */ + unsigned int ips_connect_start_idx; /* number of connections trying to connect for this host+port */ unsigned int pending_connection_count; From dovecot at dovecot.org Wed Jul 10 02:31:18 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 02:31:18 +0300 Subject: dovecot-2.2: lib-http: Don't assert-crash in case of somewhat un... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/843208699cbc changeset: 16578:843208699cbc user: Timo Sirainen date: Wed Jul 10 02:31:05 2013 +0300 description: lib-http: Don't assert-crash in case of somewhat unexpected failing connections. diffstat: src/lib-http/http-client-host.c | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) diffs (21 lines): diff -r dee7822501b0 -r 843208699cbc src/lib-http/http-client-host.c --- a/src/lib-http/http-client-host.c Wed Jul 10 01:44:40 2013 +0300 +++ b/src/lib-http/http-client-host.c Wed Jul 10 02:31:05 2013 +0300 @@ -256,9 +256,14 @@ { struct http_client_host *host = hport->host; - i_assert(hport->pending_connection_count > 0); - if (--hport->pending_connection_count > 0) - return TRUE; + if (hport->pending_connection_count > 0) { + /* we're still doing the initial connections to this hport. if + we're also doing parallel connections with soft timeouts + (pending_connection_count>1), wait for them to finish + first. */ + if (--hport->pending_connection_count > 0) + return TRUE; + } /* one of the connections failed. if we're not using soft timeouts, we need to try to connect to the next IP. if we are using soft From dovecot at dovecot.org Wed Jul 10 02:50:09 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 02:50:09 +0300 Subject: dovecot-2.2: lib-http: Fixed tracking number of pending connecti... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/129ae6342b28 changeset: 16579:129ae6342b28 user: Timo Sirainen date: Wed Jul 10 02:50:04 2013 +0300 description: lib-http: Fixed tracking number of pending connections to host-port. diffstat: src/lib-http/http-client-host.c | 3 ++- src/lib-http/http-client-peer.c | 16 +++++++++++----- src/lib-http/http-client-private.h | 2 +- 3 files changed, 14 insertions(+), 7 deletions(-) diffs (71 lines): diff -r 843208699cbc -r 129ae6342b28 src/lib-http/http-client-host.c --- a/src/lib-http/http-client-host.c Wed Jul 10 02:31:05 2013 +0300 +++ b/src/lib-http/http-client-host.c Wed Jul 10 02:50:04 2013 +0300 @@ -171,7 +171,8 @@ peer = http_client_peer_get(host->client, &addr); http_client_peer_add_host(peer, host); - hport->pending_connection_count++; + if (http_client_peer_handle_requests(peer)) + hport->pending_connection_count++; /* start soft connect time-out (but only if we have another IP left) */ msecs = host->client->set.soft_connect_timeout_msecs; diff -r 843208699cbc -r 129ae6342b28 src/lib-http/http-client-peer.c --- a/src/lib-http/http-client-peer.c Wed Jul 10 02:31:05 2013 +0300 +++ b/src/lib-http/http-client-peer.c Wed Jul 10 02:50:04 2013 +0300 @@ -93,7 +93,8 @@ } static bool -http_client_peer_next_request(struct http_client_peer *peer) +http_client_peer_next_request(struct http_client_peer *peer, + bool *created_connections) { struct http_client_connection *const *conn_idx; struct http_client_connection *conn = NULL; @@ -161,15 +162,21 @@ "(already %u usable, connecting to %u, closing %u)", new_connections, working_conn_count - connecting, connecting, closing); - http_client_peer_connect(peer, new_connections); + if (new_connections > 0) { + *created_connections = TRUE; + http_client_peer_connect(peer, new_connections); + } /* now we wait until it is connected */ return FALSE; } -void http_client_peer_handle_requests(struct http_client_peer *peer) +bool http_client_peer_handle_requests(struct http_client_peer *peer) { - while (http_client_peer_next_request(peer)) ; + bool created_connections = FALSE; + + while (http_client_peer_next_request(peer, &created_connections)) ; + return created_connections; } static struct http_client_peer * @@ -259,7 +266,6 @@ { if (!http_client_peer_have_host(peer, host)) array_append(&peer->hosts, &host, 1); - http_client_peer_handle_requests(peer); } struct http_client_request * diff -r 843208699cbc -r 129ae6342b28 src/lib-http/http-client-private.h --- a/src/lib-http/http-client-private.h Wed Jul 10 02:31:05 2013 +0300 +++ b/src/lib-http/http-client-private.h Wed Jul 10 02:50:04 2013 +0300 @@ -246,7 +246,7 @@ struct http_client_request * http_client_peer_claim_request(struct http_client_peer *peer, bool no_urgent); -void http_client_peer_handle_requests(struct http_client_peer *peer); +bool http_client_peer_handle_requests(struct http_client_peer *peer); void http_client_peer_connection_success(struct http_client_peer *peer); void http_client_peer_connection_failure(struct http_client_peer *peer, const char *reason); From dovecot at dovecot.org Wed Jul 10 03:28:04 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 03:28:04 +0300 Subject: dovecot-2.2: lib-http: Fix/cleanup to handling connect failures. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/4707096101a4 changeset: 16580:4707096101a4 user: Timo Sirainen date: Wed Jul 10 03:27:52 2013 +0300 description: lib-http: Fix/cleanup to handling connect failures. There was some kind of double-error handling which messed up things. diffstat: src/lib-http/http-client-connection.c | 5 +++-- src/lib-http/http-client-peer.c | 13 ++++++++----- src/lib-http/http-client-private.h | 1 + 3 files changed, 12 insertions(+), 7 deletions(-) diffs (64 lines): diff -r 129ae6342b28 -r 4707096101a4 src/lib-http/http-client-connection.c --- a/src/lib-http/http-client-connection.c Wed Jul 10 02:50:04 2013 +0300 +++ b/src/lib-http/http-client-connection.c Wed Jul 10 03:27:52 2013 +0300 @@ -694,6 +694,7 @@ struct stat st; conn->connected = TRUE; + conn->connect_succeeded = TRUE; if (conn->to_connect != NULL && (conn->ssl_iostream == NULL || ssl_iostream_is_handshaked(conn->ssl_iostream))) @@ -952,10 +953,10 @@ } } + if (conn->connect_succeeded) + http_client_peer_connection_lost(peer); i_free(conn); *_conn = NULL; - - http_client_peer_connection_lost(peer); } void http_client_connection_switch_ioloop(struct http_client_connection *conn) diff -r 129ae6342b28 -r 4707096101a4 src/lib-http/http-client-peer.c --- a/src/lib-http/http-client-peer.c Wed Jul 10 02:50:04 2013 +0300 +++ b/src/lib-http/http-client-peer.c Wed Jul 10 03:27:52 2013 +0300 @@ -329,17 +329,20 @@ { unsigned int num_urgent; + /* we get here when an already connected connection fails. if the + connect itself fails, http_client_peer_connection_failure() is + called instead. */ + if (peer->destroyed) return; http_client_peer_debug(peer, "Lost a connection (%d connections left)", array_count(&peer->conns)); - if (!peer->last_connect_failed) { - /* if there are pending requests, create a new - connection for them. */ - http_client_peer_handle_requests(peer); - } + /* if there are pending requests for this peer, create a new connection + for them. */ + http_client_peer_handle_requests(peer); + if (array_count(&peer->conns) == 0 && http_client_peer_requests_pending(peer, &num_urgent) == 0) http_client_peer_free(&peer); diff -r 129ae6342b28 -r 4707096101a4 src/lib-http/http-client-private.h --- a/src/lib-http/http-client-private.h Wed Jul 10 02:50:04 2013 +0300 +++ b/src/lib-http/http-client-private.h Wed Jul 10 03:27:52 2013 +0300 @@ -159,6 +159,7 @@ ARRAY_TYPE(http_client_request) request_wait_list; unsigned int connected:1; /* connection is connected */ + unsigned int connect_succeeded:1; unsigned int closing:1; unsigned int close_indicated:1; unsigned int output_locked:1; /* output is locked; no pipelining */ From dovecot at dovecot.org Wed Jul 10 05:19:35 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 05:19:35 +0300 Subject: dovecot-2.2: pop3: Avoid assert-crash if client disconnects duri... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/debbcfe5577a changeset: 16581:debbcfe5577a user: Timo Sirainen date: Wed Jul 10 05:19:22 2013 +0300 description: pop3: Avoid assert-crash if client disconnects during LIST. diffstat: src/pop3/pop3-commands.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (21 lines): diff -r 4707096101a4 -r debbcfe5577a src/pop3/pop3-commands.c --- a/src/pop3/pop3-commands.c Wed Jul 10 03:27:52 2013 +0300 +++ b/src/pop3/pop3-commands.c Wed Jul 10 05:19:22 2013 +0300 @@ -132,6 +132,8 @@ struct cmd_list_context *ctx = client->cmd_context; for (; ctx->msgnum != client->messages_count; ctx->msgnum++) { + if (client->output->closed) + break; if (POP3_CLIENT_OUTPUT_FULL(client)) { /* buffer full */ return; @@ -145,8 +147,6 @@ client_send_line(client, "%u %"PRIuUOFF_T, ctx->msgnum+1, client->message_sizes[ctx->msgnum]); - if (client->output->closed) - break; } client_send_line(client, "."); From dovecot at dovecot.org Wed Jul 10 05:29:58 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 05:29:58 +0300 Subject: dovecot-2.2: auth: If passdb sql returned only NULLs for userdb_... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9091d0f2d971 changeset: 16582:9091d0f2d971 user: Timo Sirainen date: Wed Jul 10 05:29:45 2013 +0300 description: auth: If passdb sql returned only NULLs for userdb_ fields, use userdb prefetch anyway. diffstat: src/auth/auth-request.c | 10 ++++++++++ src/auth/auth-request.h | 1 + src/auth/passdb-sql.c | 6 +++++- 3 files changed, 16 insertions(+), 1 deletions(-) diffs (47 lines): diff -r debbcfe5577a -r 9091d0f2d971 src/auth/auth-request.c --- a/src/auth/auth-request.c Wed Jul 10 05:19:22 2013 +0300 +++ b/src/auth/auth-request.c Wed Jul 10 05:29:45 2013 +0300 @@ -1397,6 +1397,16 @@ } } +void auth_request_set_null_field(struct auth_request *request, const char *name) +{ + if (strncmp(name, "userdb_", 7) == 0) { + /* make sure userdb prefetch is used even if all the fields + were returned as NULL. */ + if (request->userdb_reply == NULL) + auth_request_init_userdb_reply(request); + } +} + void auth_request_set_field_keyvalue(struct auth_request *request, const char *field, const char *default_scheme) diff -r debbcfe5577a -r 9091d0f2d971 src/auth/auth-request.h --- a/src/auth/auth-request.h Wed Jul 10 05:19:22 2013 +0300 +++ b/src/auth/auth-request.h Wed Jul 10 05:29:45 2013 +0300 @@ -192,6 +192,7 @@ void auth_request_set_field(struct auth_request *request, const char *name, const char *value, const char *default_scheme) ATTR_NULL(4); +void auth_request_set_null_field(struct auth_request *request, const char *name); void auth_request_set_field_keyvalue(struct auth_request *request, const char *field, const char *default_scheme) ATTR_NULL(3); diff -r debbcfe5577a -r 9091d0f2d971 src/auth/passdb-sql.c --- a/src/auth/passdb-sql.c Wed Jul 10 05:19:22 2013 +0300 +++ b/src/auth/passdb-sql.c Wed Jul 10 05:29:45 2013 +0300 @@ -45,7 +45,11 @@ name = sql_result_get_field_name(result, i); value = sql_result_get_field_value(result, i); - if (*name != '\0' && value != NULL) { + if (*name == '\0') + ; + else if (value == NULL) + auth_request_set_null_field(auth_request, name); + else { auth_request_set_field(auth_request, name, value, module->conn->set.default_pass_scheme); } From dovecot at dovecot.org Wed Jul 10 05:37:16 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 05:37:16 +0300 Subject: dovecot-2.2: auth: If passdb ldap returned no values for userdb_... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/939aa051e3f1 changeset: 16583:939aa051e3f1 user: Timo Sirainen date: Wed Jul 10 05:37:01 2013 +0300 description: auth: If passdb ldap returned no values for userdb_ fields, use userdb prefetch anyway. diffstat: src/auth/db-ldap.c | 18 ++++++++++++------ src/auth/db-ldap.h | 2 +- src/auth/passdb-ldap.c | 6 +++++- src/auth/userdb-ldap.c | 5 +++-- 4 files changed, 21 insertions(+), 10 deletions(-) diffs (122 lines): diff -r 9091d0f2d971 -r 939aa051e3f1 src/auth/db-ldap.c --- a/src/auth/db-ldap.c Wed Jul 10 05:29:45 2013 +0300 +++ b/src/auth/db-ldap.c Wed Jul 10 05:37:01 2013 +0300 @@ -68,6 +68,7 @@ const char *val_1_arr[2]; string_t *var, *debug; + bool skip_null_values; bool iter_dn_values; }; @@ -159,7 +160,8 @@ struct db_ldap_result_iterate_context * db_ldap_result_iterate_init_full(struct ldap_connection *conn, struct ldap_request_search *ldap_request, - LDAPMessage *res, bool iter_dn_values); + LDAPMessage *res, bool skip_null_values, + bool iter_dn_values); static int deref2str(const char *str) { @@ -556,7 +558,8 @@ struct db_ldap_result_iterate_context *ldap_iter; const char *name, *const *values; - ldap_iter = db_ldap_result_iterate_init_full(conn, request, res, TRUE); + ldap_iter = db_ldap_result_iterate_init_full(conn, request, res, + TRUE, TRUE); while (db_ldap_result_iterate_next(ldap_iter, &name, &values)) { if (values[1] != NULL) { auth_request_log_warning(auth_request, "ldap", @@ -1476,7 +1479,8 @@ struct db_ldap_result_iterate_context * db_ldap_result_iterate_init_full(struct ldap_connection *conn, struct ldap_request_search *ldap_request, - LDAPMessage *res, bool iter_dn_values) + LDAPMessage *res, bool skip_null_values, + bool iter_dn_values) { struct db_ldap_result_iterate_context *ctx; const struct ldap_request_named_result *named_res; @@ -1488,6 +1492,7 @@ ctx->pool = pool; ctx->auth_request = ldap_request->request.auth_request; ctx->attr_map = ldap_request->attr_map; + ctx->skip_null_values = skip_null_values; ctx->iter_dn_values = iter_dn_values; hash_table_create(&ctx->ldap_attrs, pool, 0, strcase_hash, strcasecmp); if (ctx->auth_request->set->debug) @@ -1507,9 +1512,10 @@ struct db_ldap_result_iterate_context * db_ldap_result_iterate_init(struct ldap_connection *conn, struct ldap_request_search *ldap_request, - LDAPMessage *res) + LDAPMessage *res, bool skip_null_values) { - return db_ldap_result_iterate_init_full(conn, ldap_request, res, FALSE); + return db_ldap_result_iterate_init_full(conn, ldap_request, res, + skip_null_values, FALSE); } static const char *db_ldap_field_get_default(const char *data) @@ -1644,7 +1650,7 @@ *name_r = field->name; *values_r = db_ldap_result_return_value(ctx, field, ldap_value); - if ((*values_r)[0] == NULL) { + if (ctx->skip_null_values && (*values_r)[0] == NULL) { /* no values. don't confuse the caller with this reply. */ return db_ldap_result_iterate_next(ctx, name_r, values_r); } diff -r 9091d0f2d971 -r 939aa051e3f1 src/auth/db-ldap.h --- a/src/auth/db-ldap.h Wed Jul 10 05:29:45 2013 +0300 +++ b/src/auth/db-ldap.h Wed Jul 10 05:37:01 2013 +0300 @@ -199,7 +199,7 @@ struct db_ldap_result_iterate_context * db_ldap_result_iterate_init(struct ldap_connection *conn, struct ldap_request_search *ldap_request, - LDAPMessage *res); + LDAPMessage *res, bool skip_null_values); bool db_ldap_result_iterate_next(struct db_ldap_result_iterate_context *ctx, const char **name_r, const char *const **values_r); diff -r 9091d0f2d971 -r 939aa051e3f1 src/auth/passdb-ldap.c --- a/src/auth/passdb-ldap.c Wed Jul 10 05:29:45 2013 +0300 +++ b/src/auth/passdb-ldap.c Wed Jul 10 05:37:01 2013 +0300 @@ -47,8 +47,12 @@ struct db_ldap_result_iterate_context *ldap_iter; const char *name, *const *values; - ldap_iter = db_ldap_result_iterate_init(conn, ldap_request, res); + ldap_iter = db_ldap_result_iterate_init(conn, ldap_request, res, FALSE); while (db_ldap_result_iterate_next(ldap_iter, &name, &values)) { + if (values[0] == NULL) { + auth_request_set_null_field(auth_request, name); + continue; + } if (values[1] != NULL) { auth_request_log_warning(auth_request, "ldap", "Multiple values found for '%s', " diff -r 9091d0f2d971 -r 939aa051e3f1 src/auth/userdb-ldap.c --- a/src/auth/userdb-ldap.c Wed Jul 10 05:29:45 2013 +0300 +++ b/src/auth/userdb-ldap.c Wed Jul 10 05:37:01 2013 +0300 @@ -52,7 +52,7 @@ auth_request_init_userdb_reply(auth_request); - ldap_iter = db_ldap_result_iterate_init(conn, ldap_request, res); + ldap_iter = db_ldap_result_iterate_init(conn, ldap_request, res, TRUE); while (db_ldap_result_iterate_next(ldap_iter, &name, &values)) { auth_request_set_userdb_field_values(auth_request, name, values); @@ -169,7 +169,8 @@ request->create_time = ioloop_time; ctx->in_callback = TRUE; - ldap_iter = db_ldap_result_iterate_init(conn, &urequest->request, res); + ldap_iter = db_ldap_result_iterate_init(conn, &urequest->request, + res, TRUE); while (db_ldap_result_iterate_next(ldap_iter, &name, &values)) { if (strcmp(name, "user") != 0) { i_warning("ldap: iterate: " From dovecot at dovecot.org Wed Jul 10 06:05:30 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 06:05:30 +0300 Subject: dovecot-2.2: imapc: Minor code cleanup. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/673648c6fb4b changeset: 16584:673648c6fb4b user: Timo Sirainen date: Wed Jul 10 05:57:36 2013 +0300 description: imapc: Minor code cleanup. diffstat: src/lib-storage/index/imapc/imapc-storage.c | 103 +++++++++++++++------------ 1 files changed, 57 insertions(+), 46 deletions(-) diffs (122 lines): diff -r 939aa051e3f1 -r 673648c6fb4b src/lib-storage/index/imapc/imapc-storage.c --- a/src/lib-storage/index/imapc/imapc-storage.c Wed Jul 10 05:37:01 2013 +0300 +++ b/src/lib-storage/index/imapc/imapc-storage.c Wed Jul 10 05:57:36 2013 +0300 @@ -240,61 +240,72 @@ } static int +imapc_client_create(struct mail_namespace *ns, + const struct imapc_settings *imapc_set, + const struct mail_storage_settings *mail_set, + struct imapc_client **client_r, const char **error_r) +{ + struct imapc_client_settings set; + string_t *str; + + memset(&set, 0, sizeof(set)); + set.host = imapc_set->imapc_host; + if (*set.host == '\0') { + *error_r = "missing imapc_host"; + return -1; + } + set.port = imapc_set->imapc_port; + if (imapc_set->imapc_user[0] != '\0') + set.username = imapc_set->imapc_user; + else if (ns->owner != NULL) + set.username = ns->owner->username; + else + set.username = ns->user->username; + set.master_user = imapc_set->imapc_master_user; + set.password = imapc_set->imapc_password; + if (*set.password == '\0') { + *error_r = "missing imapc_password"; + return -1; + } + set.max_idle_time = imapc_set->imapc_max_idle_time; + set.dns_client_socket_path = *ns->user->set->base_dir == '\0' ? "" : + t_strconcat(ns->user->set->base_dir, "/", + DNS_CLIENT_SOCKET_NAME, NULL); + set.debug = mail_set->mail_debug; + set.rawlog_dir = mail_user_home_expand(ns->user, + imapc_set->imapc_rawlog_dir); + + str = t_str_new(128); + mail_user_set_get_temp_prefix(str, ns->user->set); + set.temp_path_prefix = str_c(str); + + set.ssl_ca_dir = mail_set->ssl_client_ca_dir; + set.ssl_ca_file = mail_set->ssl_client_ca_file; + set.ssl_verify = imapc_set->imapc_ssl_verify; + if (strcmp(imapc_set->imapc_ssl, "imaps") == 0) + set.ssl_mode = IMAPC_CLIENT_SSL_MODE_IMMEDIATE; + else if (strcmp(imapc_set->imapc_ssl, "starttls") == 0) + set.ssl_mode = IMAPC_CLIENT_SSL_MODE_STARTTLS; + else + set.ssl_mode = IMAPC_CLIENT_SSL_MODE_NONE; + set.ssl_crypto_device = mail_set->ssl_crypto_device; + + *client_r = imapc_client_init(&set); + return 0; +} + +static int imapc_storage_create(struct mail_storage *_storage, struct mail_namespace *ns, const char **error_r) { struct imapc_storage *storage = (struct imapc_storage *)_storage; - struct imapc_client_settings set; - string_t *str; char sep; storage->set = mail_storage_get_driver_settings(_storage); - - memset(&set, 0, sizeof(set)); - set.host = storage->set->imapc_host; - if (*set.host == '\0') { - *error_r = "missing imapc_host"; + if (imapc_client_create(ns, storage->set, _storage->set, + &storage->client, error_r) < 0) return -1; - } - set.port = storage->set->imapc_port; - if (storage->set->imapc_user[0] != '\0') - set.username = storage->set->imapc_user; - else if (ns->owner != NULL) - set.username = ns->owner->username; - else - set.username = ns->user->username; - set.master_user = storage->set->imapc_master_user; - set.password = storage->set->imapc_password; - if (*set.password == '\0') { - *error_r = "missing imapc_password"; - return -1; - } - set.max_idle_time = storage->set->imapc_max_idle_time; - set.dns_client_socket_path = - *_storage->user->set->base_dir == '\0' ? "" : - t_strconcat(_storage->user->set->base_dir, "/", - DNS_CLIENT_SOCKET_NAME, NULL); - set.debug = _storage->set->mail_debug; - set.rawlog_dir = mail_user_home_expand(_storage->user, - storage->set->imapc_rawlog_dir); - - str = t_str_new(128); - mail_user_set_get_temp_prefix(str, _storage->user->set); - set.temp_path_prefix = str_c(str); - - set.ssl_ca_dir = _storage->set->ssl_client_ca_dir; - set.ssl_ca_file = _storage->set->ssl_client_ca_file; - set.ssl_verify = storage->set->imapc_ssl_verify; - if (strcmp(storage->set->imapc_ssl, "imaps") == 0) - set.ssl_mode = IMAPC_CLIENT_SSL_MODE_IMMEDIATE; - else if (strcmp(storage->set->imapc_ssl, "starttls") == 0) - set.ssl_mode = IMAPC_CLIENT_SSL_MODE_STARTTLS; - else - set.ssl_mode = IMAPC_CLIENT_SSL_MODE_NONE; - set.ssl_crypto_device = _storage->set->ssl_crypto_device; - - storage->client = imapc_client_init(&set); p_array_init(&storage->remote_namespaces, _storage->pool, 4); p_array_init(&storage->untagged_callbacks, _storage->pool, 16); From dovecot at dovecot.org Wed Jul 10 06:05:30 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 06:05:30 +0300 Subject: dovecot-2.2: imapc: Code cleanup: Don't access imapc settings vi... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/e45ad5524f1a changeset: 16585:e45ad5524f1a user: Timo Sirainen date: Wed Jul 10 06:05:20 2013 +0300 description: imapc: Code cleanup: Don't access imapc settings via storage. diffstat: src/lib-storage/index/imapc/imapc-list.c | 40 +++++++++++++++++++------------ src/lib-storage/index/imapc/imapc-list.h | 1 + 2 files changed, 25 insertions(+), 16 deletions(-) diffs (132 lines): diff -r 673648c6fb4b -r e45ad5524f1a src/lib-storage/index/imapc/imapc-list.c --- a/src/lib-storage/index/imapc/imapc-list.c Wed Jul 10 05:57:36 2013 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.c Wed Jul 10 06:05:20 2013 +0300 @@ -56,6 +56,17 @@ return &list->list; } +static int +imapc_list_init(struct mailbox_list *_list, const char **error_r ATTR_UNUSED) +{ + struct imapc_mailbox_list *list = (struct imapc_mailbox_list *)_list; + + list->set = mail_user_set_get_driver_settings(_list->ns->user->set_info, + _list->ns->user->set, + IMAPC_STORAGE_NAME); + return 0; +} + static void imapc_list_deinit(struct mailbox_list *_list) { struct imapc_mailbox_list *list = (struct imapc_mailbox_list *)_list; @@ -205,7 +216,7 @@ imapc_list_get_storage_name(struct mailbox_list *_list, const char *vname) { struct imapc_mailbox_list *list = (struct imapc_mailbox_list *)_list; - const char *prefix = list->storage->set->imapc_list_prefix; + const char *prefix = list->set->imapc_list_prefix; const char *storage_name; storage_name = mailbox_list_default_get_storage_name(_list, vname); @@ -222,7 +233,7 @@ imapc_list_get_vname(struct mailbox_list *_list, const char *storage_name) { struct imapc_mailbox_list *list = (struct imapc_mailbox_list *)_list; - const char *prefix = list->storage->set->imapc_list_prefix; + const char *prefix = list->set->imapc_list_prefix; unsigned int prefix_len; if (*storage_name == '\0') { @@ -285,7 +296,7 @@ return NULL; vname = mailbox_list_get_vname(&list->list, name); - if (list->storage->set->imapc_list_prefix[0] != '\0') { + if (list->set->imapc_list_prefix[0] != '\0') { /* put back the prefix, so it gets included in the filesystem. */ unsigned int vname_len = strlen(vname); @@ -301,10 +312,10 @@ } if (vname[0] == '\0') { vname = t_strconcat(ns->prefix, - list->storage->set->imapc_list_prefix, NULL); + list->set->imapc_list_prefix, NULL); } else { vname = t_strdup_printf("%s%s%c%s", ns->prefix, - list->storage->set->imapc_list_prefix, + list->set->imapc_list_prefix, ns_sep, vname); } } @@ -366,7 +377,7 @@ struct mailbox_list *fs_list = imapc_list_get_fs(list); struct mailbox_list_iterate_context *iter; const struct mailbox_info *info; - const char *imapc_list_prefix = list->storage->set->imapc_list_prefix; + const char *imapc_list_prefix = list->set->imapc_list_prefix; unsigned int imapc_list_prefix_len = strlen(imapc_list_prefix); const char *fs_name, *vname; @@ -417,14 +428,13 @@ if (list->refreshed_mailboxes) return 0; - if (*list->storage->set->imapc_list_prefix == '\0') + if (*list->set->imapc_list_prefix == '\0') pattern = "*"; else { /* list "prefix*" instead of "prefix.*". this may return a bit more than we want, but we're also interested in the flags of the prefix itself. */ - pattern = t_strdup_printf("%s*", - list->storage->set->imapc_list_prefix); + pattern = t_strdup_printf("%s*", list->set->imapc_list_prefix); } cmd = imapc_list_simple_context_init(&ctx, list); @@ -440,7 +450,7 @@ imapc_list_prefix doesn't match it */ bool created; node = mailbox_tree_get(list->mailboxes, "INBOX", &created); - if (*list->storage->set->imapc_list_prefix != '\0') { + if (*list->set->imapc_list_prefix != '\0') { /* this listing didn't include the INBOX itself, but might have included its children. make sure there aren't any extra flags in it (especially @@ -640,12 +650,10 @@ mailbox_tree_init(mail_namespace_get_sep(_src_list->ns)); cmd = imapc_list_simple_context_init(&ctx, src_list); - if (*src_list->storage->set->imapc_list_prefix == '\0') + if (*src_list->set->imapc_list_prefix == '\0') pattern = "*"; - else { - pattern = t_strdup_printf("%s*", - src_list->storage->set->imapc_list_prefix); - } + else + pattern = t_strdup_printf("%s*", src_list->set->imapc_list_prefix); imapc_command_sendf(cmd, "LSUB \"\" %s", pattern); imapc_simple_run(&ctx); @@ -789,7 +797,7 @@ { imapc_list_alloc, - NULL, + imapc_list_init, imapc_list_deinit, NULL, imapc_list_get_hierarchy_sep, diff -r 673648c6fb4b -r e45ad5524f1a src/lib-storage/index/imapc/imapc-list.h --- a/src/lib-storage/index/imapc/imapc-list.h Wed Jul 10 05:57:36 2013 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.h Wed Jul 10 06:05:20 2013 +0300 @@ -9,6 +9,7 @@ struct imapc_mailbox_list { struct mailbox_list list; + const struct imapc_settings *set; struct imapc_storage *storage; struct mailbox_list *index_list; From dovecot at dovecot.org Wed Jul 10 06:55:08 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 06:55:08 +0300 Subject: dovecot-2.2: imapc: Reorganize code so that imapc_list works wit... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/620876853f6f changeset: 16586:620876853f6f user: Timo Sirainen date: Wed Jul 10 06:54:57 2013 +0300 description: imapc: Reorganize code so that imapc_list works without imapc_storage. Most importantly fixes crashes when imapc_list is trying to lookup hierarchy separator before storage is created. diffstat: src/lib-storage/index/imapc/imapc-list.c | 150 ++++++++++++-- src/lib-storage/index/imapc/imapc-list.h | 7 +- src/lib-storage/index/imapc/imapc-mail-fetch.c | 4 +- src/lib-storage/index/imapc/imapc-save.c | 13 +- src/lib-storage/index/imapc/imapc-storage.c | 241 +++++++++--------------- src/lib-storage/index/imapc/imapc-storage.h | 43 +++- src/lib-storage/index/imapc/imapc-sync.c | 6 +- 7 files changed, 260 insertions(+), 204 deletions(-) diffs (truncated from 931 to 300 lines): diff -r e45ad5524f1a -r 620876853f6f src/lib-storage/index/imapc/imapc-list.c --- a/src/lib-storage/index/imapc/imapc-list.c Wed Jul 10 06:05:20 2013 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.c Wed Jul 10 06:54:57 2013 +0300 @@ -41,6 +41,12 @@ extern struct mailbox_list imapc_mailbox_list; +static void imapc_list_send_hierarcy_sep_lookup(struct imapc_mailbox_list *list); +static void imapc_untagged_list(const struct imapc_untagged_reply *reply, + struct imapc_storage_client *client); +static void imapc_untagged_lsub(const struct imapc_untagged_reply *reply, + struct imapc_storage_client *client); + static struct mailbox_list *imapc_list_alloc(void) { struct imapc_mailbox_list *list; @@ -50,20 +56,41 @@ list = p_new(pool, struct imapc_mailbox_list, 1); list->list = imapc_mailbox_list; list->list.pool = pool; - /* separator is set when storage is created */ + /* separator is set lazily */ list->mailboxes = mailbox_tree_init('\0'); mailbox_tree_set_parents_nonexistent(list->mailboxes); return &list->list; } -static int -imapc_list_init(struct mailbox_list *_list, const char **error_r ATTR_UNUSED) +static int imapc_list_init(struct mailbox_list *_list, const char **error_r) { struct imapc_mailbox_list *list = (struct imapc_mailbox_list *)_list; + char sep; list->set = mail_user_set_get_driver_settings(_list->ns->user->set_info, _list->ns->user->set, IMAPC_STORAGE_NAME); + if (imapc_storage_client_create(_list->ns, list->set, _list->mail_set, + &list->client, error_r) < 0) + return -1; + list->client->_list = list; + + imapc_storage_client_register_untagged(list->client, "LIST", + imapc_untagged_list); + imapc_storage_client_register_untagged(list->client, "LSUB", + imapc_untagged_lsub); + imapc_list_send_hierarcy_sep_lookup(list); + if ((_list->ns->flags & NAMESPACE_FLAG_INBOX_USER) != 0) { + /* we're using imapc for the INBOX namespace. wait and make + sure we can successfully access the IMAP server (so if the + username is invalid we don't just keep failing every + command). */ + if (imapc_list_try_get_root_sep(list, &sep) < 0) { + imapc_storage_client_unref(&list->client); + *error_r = "Failed to access imapc backend"; + return -1; + } + } return 0; } @@ -76,21 +103,43 @@ mailbox_tree_deinit(&list->mailboxes); if (list->tmp_subscriptions != NULL) mailbox_tree_deinit(&list->tmp_subscriptions); + imapc_storage_client_unref(&list->client); pool_unref(&list->list.pool); } +static void +imapc_list_copy_error_from_reply(struct imapc_mailbox_list *list, + enum mail_error default_error, + const struct imapc_command_reply *reply) +{ + enum mail_error error; + + if (imap_resp_text_code_parse(reply->resp_text_key, &error)) { + mailbox_list_set_error(&list->list, error, + reply->text_without_resp); + } else { + mailbox_list_set_error(&list->list, default_error, + reply->text_without_resp); + } +} + static void imapc_list_simple_callback(const struct imapc_command_reply *reply, void *context) { struct imapc_simple_context *ctx = context; - const char *str; - enum mail_error error; - imapc_simple_callback(reply, context); - if (ctx->ret < 0) { - str = mail_storage_get_last_error(&ctx->storage->storage, &error); - mailbox_list_set_error(&ctx->storage->list->list, error, str); + if (reply->state == IMAPC_COMMAND_STATE_OK) + ctx->ret = 0; + else if (reply->state == IMAPC_COMMAND_STATE_NO) { + imapc_list_copy_error_from_reply(ctx->client->_list, + MAIL_ERROR_PARAMS, reply); + ctx->ret = -1; + } else { + mailbox_list_set_critical(&ctx->client->_list->list, + "imapc: Command failed: %s", reply->text_full); + ctx->ret = -1; } + imapc_client_stop(ctx->client->client); } static bool @@ -144,13 +193,13 @@ } static void imapc_untagged_list(const struct imapc_untagged_reply *reply, - struct imapc_storage *storage) + struct imapc_storage_client *client) { - struct imapc_mailbox_list *list = storage->list; + struct imapc_mailbox_list *list = client->_list; const struct imap_arg *args = reply->args; const char *sep, *name; - if (storage->root_sep == '\0') { + if (list->root_sep == '\0') { /* we haven't asked for the separator yet. lets see if this is the reply for its request. */ if (args[0].type == IMAP_ARG_EOL || @@ -159,21 +208,21 @@ return; /* we can't handle NIL separator yet */ - storage->root_sep = sep == NULL ? '/' : sep[0]; - mailbox_tree_set_separator(list->mailboxes, storage->root_sep); + list->root_sep = sep == NULL ? '/' : sep[0]; + mailbox_tree_set_separator(list->mailboxes, list->root_sep); } else { (void)imapc_list_update_tree(list, list->mailboxes, args); } } static void imapc_untagged_lsub(const struct imapc_untagged_reply *reply, - struct imapc_storage *storage) + struct imapc_storage_client *client) { - struct imapc_mailbox_list *list = storage->list; + struct imapc_mailbox_list *list = client->_list; const struct imap_arg *args = reply->args; struct mailbox_node *node; - if (storage->root_sep == '\0') { + if (list->root_sep == '\0') { /* we haven't asked for the separator yet */ return; } @@ -191,12 +240,61 @@ } } -void imapc_list_register_callbacks(struct imapc_mailbox_list *list) +static void imapc_list_sep_verify(struct imapc_mailbox_list *list) { - imapc_storage_register_untagged(list->storage, "LIST", - imapc_untagged_list); - imapc_storage_register_untagged(list->storage, "LSUB", - imapc_untagged_lsub); + const char *imapc_list_prefix = list->set->imapc_list_prefix; + + if (list->root_sep == '\0') { + mailbox_list_set_critical(&list->list, + "imapc: LIST didn't return hierarchy separator"); + } else if (imapc_list_prefix[0] != '\0' && + imapc_list_prefix[strlen(imapc_list_prefix)-1] == list->root_sep) { + mailbox_list_set_critical(&list->list, + "imapc_list_prefix must not end with hierarchy separator"); + } +} + +static void imapc_storage_sep_callback(const struct imapc_command_reply *reply, + void *context) +{ + struct imapc_mailbox_list *list = context; + + list->root_sep_pending = FALSE; + if (reply->state == IMAPC_COMMAND_STATE_OK) + imapc_list_sep_verify(list); + else if (reply->state == IMAPC_COMMAND_STATE_NO) + imapc_list_copy_error_from_reply(list, MAIL_ERROR_PARAMS, reply); + else { + mailbox_list_set_critical(&list->list, + "imapc: Command failed: %s", reply->text_full); + } + imapc_client_stop(list->client->client); +} + +static void imapc_list_send_hierarcy_sep_lookup(struct imapc_mailbox_list *list) +{ + struct imapc_command *cmd; + + if (list->root_sep_pending) + return; + list->root_sep_pending = TRUE; + + cmd = imapc_client_cmd(list->client->client, + imapc_storage_sep_callback, list); + imapc_command_send(cmd, "LIST \"\" \"\""); +} + +int imapc_list_try_get_root_sep(struct imapc_mailbox_list *list, char *sep_r) +{ + if (list->root_sep == '\0') { + imapc_list_send_hierarcy_sep_lookup(list); + while (list->root_sep_pending) + imapc_client_run(list->client->client); + if (list->root_sep == '\0') + return -1; + } + *sep_r = list->root_sep; + return 0; } static char imapc_list_get_hierarchy_sep(struct mailbox_list *_list) @@ -204,7 +302,7 @@ struct imapc_mailbox_list *list = (struct imapc_mailbox_list *)_list; char sep; - if (imapc_storage_try_get_root_sep(list->storage, &sep) < 0) { + if (imapc_list_try_get_root_sep(list, &sep) < 0) { /* we can't really fail here. just return a common separator and keep failing all list commands until it succeeds. */ return '/'; @@ -367,8 +465,8 @@ imapc_list_simple_context_init(struct imapc_simple_context *ctx, struct imapc_mailbox_list *list) { - imapc_simple_context_init(ctx, list->storage); - return imapc_client_cmd(list->storage->client, + imapc_simple_context_init(ctx, list->client); + return imapc_client_cmd(list->client->client, imapc_list_simple_callback, ctx); } @@ -690,7 +788,7 @@ struct imapc_command *cmd; struct imapc_simple_context ctx; - capa = imapc_client_get_capabilities(list->storage->client); + capa = imapc_client_get_capabilities(list->client->client); cmd = imapc_list_simple_context_init(&ctx, list); imapc_command_set_flags(cmd, IMAPC_COMMAND_FLAG_SELECT); diff -r e45ad5524f1a -r 620876853f6f src/lib-storage/index/imapc/imapc-list.h --- a/src/lib-storage/index/imapc/imapc-list.h Wed Jul 10 06:05:20 2013 +0300 +++ b/src/lib-storage/index/imapc/imapc-list.h Wed Jul 10 06:54:57 2013 +0300 @@ -10,21 +10,22 @@ struct imapc_mailbox_list { struct mailbox_list list; const struct imapc_settings *set; - struct imapc_storage *storage; + struct imapc_storage_client *client; struct mailbox_list *index_list; struct mailbox_tree_context *mailboxes, *tmp_subscriptions; + char root_sep; unsigned int iter_count; unsigned int refreshed_subscriptions:1; unsigned int refreshed_mailboxes:1; unsigned int index_list_failed:1; + unsigned int root_sep_pending:1; }; int imapc_list_get_mailbox_flags(struct mailbox_list *list, const char *name, enum mailbox_info_flags *flags_r); - -void imapc_list_register_callbacks(struct imapc_mailbox_list *list); +int imapc_list_try_get_root_sep(struct imapc_mailbox_list *list, char *sep_r); #endif diff -r e45ad5524f1a -r 620876853f6f src/lib-storage/index/imapc/imapc-mail-fetch.c --- a/src/lib-storage/index/imapc/imapc-mail-fetch.c Wed Jul 10 06:05:20 2013 +0300 +++ b/src/lib-storage/index/imapc/imapc-mail-fetch.c Wed Jul 10 06:54:57 2013 +0300 @@ -48,7 +48,7 @@ "imapc: Mail prefetch failed: %s", reply->text_full); } pool_unref(&mail->imail.mail.pool); - imapc_client_stop(mbox->storage->client); + imapc_client_stop(mbox->storage->client->client); } static int @@ -407,6 +407,6 @@ if (!match) { /* this is only a FETCH FLAGS update for the wanted mail */ } else { - imapc_client_stop(mbox->storage->client); + imapc_client_stop(mbox->storage->client->client); } } From dovecot at dovecot.org Wed Jul 10 08:37:47 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 08:37:47 +0300 Subject: dovecot-2.2: login-common: Add support for ECDH/ECDHE cipher suites Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/331d0a4fe772 changeset: 16587:331d0a4fe772 user: David Hicks date: Sat Jul 06 14:46:16 2013 +1000 description: login-common: Add support for ECDH/ECDHE cipher suites ECDH temporary key parameter selection must be performed during OpenSSL context initialisation before ECDH and ECDHE cipher suites can be used. OpenSSL >= 1.0.2 automatically handles ECDH temporary key parameter selection. For OpenSSL < 1.0.2 we must manually specify a named elliptic curve that Dovecot will use to generate an ephemeral key pair. By default we try to use the same named curve as that used in the server's private EC key file. If this attempt fails, a fall back curve of NIST P-384 (secp384r1) is used instead. RFC 6460 states that NIST P-384 MUST be used for cipher suites that include AES-256. For cipher suites that include AES-128, RFC 6460 states that NIST P-256 MUST be used. No matter which curve is used as a fall back option, Dovecot will be non-compliant. The reason for selecting NIST P-384 as a fall back curve is to ensure that the non-compliance is in the form of providing too great a level of security for AES-128 cipher suites rather than too little security for AES-256 cipher suites. diffstat: src/login-common/ssl-proxy-openssl.c | 76 +++++++++++++++++++++++++++++++++++- 1 files changed, 74 insertions(+), 2 deletions(-) diffs (111 lines): diff -r 620876853f6f -r 331d0a4fe772 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Wed Jul 10 06:54:57 2013 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Sat Jul 06 14:46:16 2013 +1000 @@ -119,6 +119,10 @@ const struct master_service_ssl_settings *ssl_set); static void ssl_server_context_deinit(struct ssl_server_context **_ctx); +static void ssl_proxy_ctx_set_crypto_params(SSL_CTX *ssl_ctx, + const struct master_service_ssl_settings *set); +static int ssl_proxy_ctx_get_pkey_ec_curve_name(const struct master_service_ssl_settings *set); + static unsigned int ssl_server_context_hash(const struct ssl_server_context *ctx) { unsigned int i, g, h = 0; @@ -993,11 +997,58 @@ store = SSL_CTX_get_cert_store(ssl_ctx); load_ca(store, set->ssl_ca, load_xnames ? &xnames : NULL); } + ssl_proxy_ctx_set_crypto_params(ssl_ctx, set); SSL_CTX_set_info_callback(ssl_ctx, ssl_info_callback); + return xnames; +} + +static void +ssl_proxy_ctx_set_crypto_params(SSL_CTX *ssl_ctx, + const struct master_service_ssl_settings *set) +{ +#if !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10000000L && OPENSSL_VERSION_NUMBER < 0x10002000L + EC_KEY *ecdh; + int nid; + const char *curve_name; +#endif if (SSL_CTX_need_tmp_RSA(ssl_ctx)) SSL_CTX_set_tmp_rsa_callback(ssl_ctx, ssl_gen_rsa_key); SSL_CTX_set_tmp_dh_callback(ssl_ctx, ssl_tmp_dh_callback); - return xnames; +#if !defined(OPENSSL_NO_ECDH) + /* In the non-recommended situation where ECDH cipher suites are being + used instead of ECDHE, do not reuse the same ECDH key pair for + different sessions. This option improves forward secrecy. */ + SSL_CTX_set_options(ssl_ctx, SSL_OP_SINGLE_ECDH_USE); +#endif +#if !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10002000L + /* OpenSSL >= 1.0.2 automatically handles ECDH temporary key parameter + selection. */ + SSL_CTX_set_ecdh_auto(ssl_ctx, 1); +#elif !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10000000L + /* For OpenSSL < 1.0.2, ECDH temporary key parameter selection must be + performed manually. Attempt to select the same curve as that used + in the server's private EC key file. Otherwise fall back to the + NIST P-384 (secp384r1) curve to be compliant with RFC 6460 when + AES-256 TLS cipher suites are in use. This fall back option does + however make Dovecot non-compliant with RFC 6460 which requires + curve NIST P-256 (prime256v1) be used when AES-128 TLS cipher + suites are in use. At least the non-compliance is in the form of + providing too much security rather than too little. */ + nid = ssl_proxy_ctx_get_pkey_ec_curve_name(set); + ecdh = EC_KEY_new_by_curve_name(nid); + if (ecdh == NULL) { + /* Fall back option */ + nid = NID_secp384r1; + ecdh = EC_KEY_new_by_curve_name(nid); + } + if ((curve_name = OBJ_nid2sn(nid)) != NULL && set->verbose_ssl) + i_debug("SSL: elliptic curve %s will be used for ECDH and" + " ECDHE key exchanges", curve_name); + if (ecdh != NULL) { + SSL_CTX_set_tmp_ecdh(ssl_ctx, ecdh); + EC_KEY_free(ecdh); + } +#endif } static void @@ -1084,6 +1135,28 @@ } static int +ssl_proxy_ctx_get_pkey_ec_curve_name(const struct master_service_ssl_settings *set) +{ + int nid = 0; +#if !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10000000L && OPENSSL_VERSION_NUMBER < 0x10002000L + EVP_PKEY *pkey; + const char *password; + EC_KEY *eckey; + const EC_GROUP *ecgrp; + + password = *set->ssl_key_password != '\0' ? set->ssl_key_password : + getenv(MASTER_SSL_KEY_PASSWORD_ENV); + pkey = ssl_proxy_load_key(set->ssl_key, password); + if (pkey != NULL && + (eckey = EVP_PKEY_get1_EC_KEY(pkey)) != NULL && + (ecgrp = EC_KEY_get0_group(eckey)) != NULL) + nid = EC_GROUP_get_curve_name(ecgrp); + EVP_PKEY_free(pkey); +#endif + return nid; +} + +static int ssl_proxy_ctx_use_certificate_chain(SSL_CTX *ctx, const char *cert) { /* mostly just copy&pasted from SSL_CTX_use_certificate_chain_file() */ @@ -1209,7 +1282,6 @@ #endif ssl_proxy_ctx_use_key(ctx->ctx, ssl_set); - SSL_CTX_set_info_callback(ctx->ctx, ssl_info_callback); if (ctx->verify_client_cert) ssl_proxy_ctx_verify_client(ctx->ctx, xnames); From dovecot at dovecot.org Wed Jul 10 08:42:13 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 08:42:13 +0300 Subject: dovecot-2.2: lib-http: Minor comment update. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/f296a05246fb changeset: 16588:f296a05246fb user: Timo Sirainen date: Wed Jul 10 08:42:09 2013 +0300 description: lib-http: Minor comment update. diffstat: src/lib-http/http-client-private.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 331d0a4fe772 -r f296a05246fb src/lib-http/http-client-private.h --- a/src/lib-http/http-client-private.h Sat Jul 06 14:46:16 2013 +1000 +++ b/src/lib-http/http-client-private.h Wed Jul 10 08:42:09 2013 +0300 @@ -79,7 +79,7 @@ /* current index in host->ips */ unsigned int ips_connect_idx; /* the first IP that started the current round of connection attempts. - initially 0, and later set to the (ip_idx+1) of the last successful + initially 0, and later set to the ip index of the last successful connected IP */ unsigned int ips_connect_start_idx; /* number of connections trying to connect for this host+port */ From dovecot at dovecot.org Wed Jul 10 08:58:48 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 08:58:48 +0300 Subject: dovecot-2.2: login-common: Compiler warning fix with OpenSSL v1.... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ac0dd2fbdbf0 changeset: 16589:ac0dd2fbdbf0 user: Timo Sirainen date: Wed Jul 10 08:47:49 2013 +0300 description: login-common: Compiler warning fix with OpenSSL v1.0.2+ diffstat: src/login-common/ssl-proxy-openssl.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (27 lines): diff -r f296a05246fb -r ac0dd2fbdbf0 src/login-common/ssl-proxy-openssl.c --- a/src/login-common/ssl-proxy-openssl.c Wed Jul 10 08:42:09 2013 +0300 +++ b/src/login-common/ssl-proxy-openssl.c Wed Jul 10 08:47:49 2013 +0300 @@ -1134,11 +1134,11 @@ EVP_PKEY_free(pkey); } +#if !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10000000L && OPENSSL_VERSION_NUMBER < 0x10002000L static int ssl_proxy_ctx_get_pkey_ec_curve_name(const struct master_service_ssl_settings *set) { int nid = 0; -#if !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10000000L && OPENSSL_VERSION_NUMBER < 0x10002000L EVP_PKEY *pkey; const char *password; EC_KEY *eckey; @@ -1152,9 +1152,9 @@ (ecgrp = EC_KEY_get0_group(eckey)) != NULL) nid = EC_GROUP_get_curve_name(ecgrp); EVP_PKEY_free(pkey); -#endif return nid; } +#endif static int ssl_proxy_ctx_use_certificate_chain(SSL_CTX *ctx, const char *cert) From dovecot at dovecot.org Wed Jul 10 08:58:48 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 08:58:48 +0300 Subject: dovecot-2.2: lib-ssl-iostream: Added support for ECDH/ECDHE ciph... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/dc99912e5047 changeset: 16590:dc99912e5047 user: Timo Sirainen date: Wed Jul 10 08:58:36 2013 +0300 description: lib-ssl-iostream: Added support for ECDH/ECDHE cipher suites Based on the login-common patch by David Hicks. diffstat: src/lib-ssl-iostream/iostream-openssl-context.c | 83 ++++++++++++++++++++++++- 1 files changed, 80 insertions(+), 3 deletions(-) diffs (100 lines): diff -r ac0dd2fbdbf0 -r dc99912e5047 src/lib-ssl-iostream/iostream-openssl-context.c --- a/src/lib-ssl-iostream/iostream-openssl-context.c Wed Jul 10 08:47:49 2013 +0300 +++ b/src/lib-ssl-iostream/iostream-openssl-context.c Wed Jul 10 08:58:36 2013 +0300 @@ -406,6 +406,84 @@ return 0; } +#if !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10000000L && OPENSSL_VERSION_NUMBER < 0x10002000L +static int +ssl_proxy_ctx_get_pkey_ec_curve_name(const struct ssl_iostream_settings *set, + int *nid_r, const char **error_r) +{ + int nid = 0; + EVP_PKEY *pkey; + const char *password; + EC_KEY *eckey; + const EC_GROUP *ecgrp; + + password = set->key_password[0] != '\0' ? set->key_password : NULL; + if (openssl_iostream_load_key(set, &pkey, error_r) < 0) + return -1; + + if ((eckey = EVP_PKEY_get1_EC_KEY(pkey)) != NULL && + (ecgrp = EC_KEY_get0_group(eckey)) != NULL) + nid = EC_GROUP_get_curve_name(ecgrp); + EVP_PKEY_free(pkey); + + *nid_r = nid; + return 0; +} +#endif + +static int +ssl_proxy_ctx_set_crypto_params(SSL_CTX *ssl_ctx, + const struct ssl_iostream_settings *set, + const char **error_r) +{ +#if !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10000000L && OPENSSL_VERSION_NUMBER < 0x10002000L + EC_KEY *ecdh; + int nid; + const char *curve_name; +#endif + if (SSL_CTX_need_tmp_RSA(ssl_ctx)) + SSL_CTX_set_tmp_rsa_callback(ssl_ctx, ssl_gen_rsa_key); + SSL_CTX_set_tmp_dh_callback(ssl_ctx, ssl_tmp_dh_callback); +#if !defined(OPENSSL_NO_ECDH) + /* In the non-recommended situation where ECDH cipher suites are being + used instead of ECDHE, do not reuse the same ECDH key pair for + different sessions. This option improves forward secrecy. */ + SSL_CTX_set_options(ssl_ctx, SSL_OP_SINGLE_ECDH_USE); +#endif +#if !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10002000L + /* OpenSSL >= 1.0.2 automatically handles ECDH temporary key parameter + selection. */ + SSL_CTX_set_ecdh_auto(ssl_ctx, 1); +#elif !defined(OPENSSL_NO_ECDH) && OPENSSL_VERSION_NUMBER >= 0x10000000L + /* For OpenSSL < 1.0.2, ECDH temporary key parameter selection must be + performed manually. Attempt to select the same curve as that used + in the server's private EC key file. Otherwise fall back to the + NIST P-384 (secp384r1) curve to be compliant with RFC 6460 when + AES-256 TLS cipher suites are in use. This fall back option does + however make Dovecot non-compliant with RFC 6460 which requires + curve NIST P-256 (prime256v1) be used when AES-128 TLS cipher + suites are in use. At least the non-compliance is in the form of + providing too much security rather than too little. */ + if (ssl_proxy_ctx_get_pkey_ec_curve_name(set, &nid, error_r) < 0) + return -1; + ecdh = EC_KEY_new_by_curve_name(nid); + if (ecdh == NULL) { + /* Fall back option */ + nid = NID_secp384r1; + ecdh = EC_KEY_new_by_curve_name(nid); + } + if ((curve_name = OBJ_nid2sn(nid)) != NULL && set->verbose) { + i_debug("SSL: elliptic curve %s will be used for ECDH and" + " ECDHE key exchanges", curve_name); + } + if (ecdh != NULL) { + SSL_CTX_set_tmp_ecdh(ssl_ctx, ecdh); + EC_KEY_free(ecdh); + } +#endif + return 0; +} + static int ssl_iostream_context_init_common(struct ssl_iostream_context *ctx, const struct ssl_iostream_settings *set, @@ -417,9 +495,8 @@ makes SSL more vulnerable against attacks */ SSL_CTX_set_options(ctx->ssl_ctx, SSL_OP_NO_SSLv2 | (SSL_OP_ALL & ~SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS)); - if (SSL_CTX_need_tmp_RSA(ctx->ssl_ctx)) - SSL_CTX_set_tmp_rsa_callback(ctx->ssl_ctx, ssl_gen_rsa_key); - SSL_CTX_set_tmp_dh_callback(ctx->ssl_ctx, ssl_tmp_dh_callback); + if (ssl_proxy_ctx_set_crypto_params(ctx->ssl_ctx, set, error_r) < 0) + return -1; return ssl_iostream_context_set(ctx, set, error_r); } From dovecot at dovecot.org Wed Jul 10 09:37:17 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 09:37:17 +0300 Subject: dovecot-2.2: Maildir++ quota: When recreating, don't use maildir... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/24d6708cea76 changeset: 16591:24d6708cea76 user: Timo Sirainen date: Wed Jul 10 09:37:06 2013 +0300 description: Maildir++ quota: When recreating, don't use maildirsize.lock file. The point was never to even try to lock it, it just used to be an easier API to create the file. Nowadays safe_mkstemp_hostpid_group() works just as easily. This normally doesn't matter, but apparently Exim+CPanel leaves maildirsize.lock files lying around, causing Dovecot to not update maildirsize. diffstat: src/plugins/quota/quota-maildir.c | 46 ++++++++++++++++++-------------------- 1 files changed, 22 insertions(+), 24 deletions(-) diffs (91 lines): diff -r dc99912e5047 -r 24d6708cea76 src/plugins/quota/quota-maildir.c --- a/src/plugins/quota/quota-maildir.c Wed Jul 10 08:58:36 2013 +0300 +++ b/src/plugins/quota/quota-maildir.c Wed Jul 10 09:37:06 2013 +0300 @@ -4,7 +4,7 @@ #include "array.h" #include "ioloop.h" #include "nfs-workarounds.h" -#include "file-dotlock.h" +#include "safe-mkstemp.h" #include "mkdir-parents.h" #include "read-full.h" #include "write-full.h" @@ -234,10 +234,9 @@ struct quota_root *_root = &root->root; struct mail_namespace *const *namespaces; unsigned int i, count; - struct dotlock *dotlock; struct mailbox_permissions perm; const char *p, *dir; - string_t *str; + string_t *str, *temp_path; int fd; i_assert(root->fd == -1); @@ -260,11 +259,12 @@ dotlock_settings.use_excl_lock = set->dotlock_use_excl; dotlock_settings.nfs_flush = set->mail_nfs_storage; - fd = file_dotlock_open_group(&dotlock_settings, path, - DOTLOCK_CREATE_FLAG_NONBLOCK, - perm.file_create_mode, - perm.file_create_gid, - perm.file_create_gid_origin, &dotlock); + + temp_path = t_str_new(128); + str_append(temp_path, path); + fd = safe_mkstemp_hostpid_group(temp_path, perm.file_create_mode, + perm.file_create_gid, + perm.file_create_gid_origin); if (fd == -1 && errno == ENOENT) { /* the control directory doesn't exist yet? create it */ p = strrchr(path, '/'); @@ -276,20 +276,13 @@ i_error("mkdir_parents(%s) failed: %m", dir); return -1; } - fd = file_dotlock_open_group(&dotlock_settings, path, - DOTLOCK_CREATE_FLAG_NONBLOCK, - perm.file_create_mode, - perm.file_create_gid, - perm.file_create_gid_origin, - &dotlock); + fd = safe_mkstemp_hostpid_group(temp_path, + perm.file_create_mode, + perm.file_create_gid, + perm.file_create_gid_origin); } if (fd == -1) { - if (errno == EAGAIN) { - /* someone's just in the middle of updating it */ - return 1; - } - - i_error("file_dotlock_open(%s) failed: %m", path); + i_error("safe_mkstemp(%s) failed: %m", path); return -1; } @@ -309,13 +302,18 @@ (unsigned long long)root->total_bytes, (unsigned long long)root->total_count); if (write_full(fd, str_data(str), str_len(str)) < 0) { - i_error("write_full(%s) failed: %m", path); - file_dotlock_delete(&dotlock); + i_error("write_full(%s) failed: %m", str_c(temp_path)); + i_close_fd(&fd); + if (unlink(str_c(temp_path)) < 0) + i_error("unlink(%s) failed: %m", str_c(temp_path)); return -1; } + i_close_fd(&fd); - if (file_dotlock_replace(&dotlock, 0) < 0) { - i_error("file_dotlock_replace(%s) failed: %m", path); + if (rename(str_c(temp_path), path) < 0) { + i_error("rename(%s, %s) failed: %m", str_c(temp_path), path); + if (unlink(str_c(temp_path)) < 0 && errno != ENOENT) + i_error("unlink(%s) failed: %m", str_c(temp_path)); return -1; } return 0; From dovecot at dovecot.org Wed Jul 10 09:43:34 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 09:43:34 +0300 Subject: dovecot-2.2: auth: Fixed crash for some lookups that didn't spec... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/aff54366b1b6 changeset: 16592:aff54366b1b6 user: Timo Sirainen date: Wed Jul 10 09:43:19 2013 +0300 description: auth: Fixed crash for some lookups that didn't specify service name. diffstat: src/auth/auth.c | 20 +++++++++++--------- 1 files changed, 11 insertions(+), 9 deletions(-) diffs (30 lines): diff -r 24d6708cea76 -r aff54366b1b6 src/auth/auth.c --- a/src/auth/auth.c Wed Jul 10 09:37:06 2013 +0300 +++ b/src/auth/auth.c Wed Jul 10 09:43:19 2013 +0300 @@ -268,15 +268,17 @@ unsigned int i, count; a = array_get(&auths, &count); - for (i = 1; i < count; i++) { - if (strcmp(a[i]->service, name) == 0) - return a[i]; - } - /* not found. maybe we can instead find a !service */ - for (i = 1; i < count; i++) { - if (a[i]->service[0] == '!' && - strcmp(a[i]->service + 1, name) != 0) - return a[i]; + if (name != NULL) { + for (i = 1; i < count; i++) { + if (strcmp(a[i]->service, name) == 0) + return a[i]; + } + /* not found. maybe we can instead find a !service */ + for (i = 1; i < count; i++) { + if (a[i]->service[0] == '!' && + strcmp(a[i]->service + 1, name) != 0) + return a[i]; + } } return a[0]; } From dovecot at dovecot.org Wed Jul 10 10:01:43 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 10 Jul 2013 10:01:43 +0300 Subject: dovecot-2.2: lib-ssl-iostreams: ssl_protocols setting supports n... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3a08bd30d180 changeset: 16593:3a08bd30d180 user: Timo Sirainen date: Wed Jul 10 10:01:26 2013 +0300 description: lib-ssl-iostreams: ssl_protocols setting supports now TLSv1.1 and TLSv1.2 values. diffstat: src/lib-ssl-iostream/iostream-openssl-common.c | 24 ++++++++++++++++++++---- 1 files changed, 20 insertions(+), 4 deletions(-) diffs (48 lines): diff -r aff54366b1b6 -r 3a08bd30d180 src/lib-ssl-iostream/iostream-openssl-common.c --- a/src/lib-ssl-iostream/iostream-openssl-common.c Wed Jul 10 09:43:19 2013 +0300 +++ b/src/lib-ssl-iostream/iostream-openssl-common.c Wed Jul 10 10:01:26 2013 +0300 @@ -6,10 +6,12 @@ #include enum { - DOVECOT_SSL_PROTO_SSLv2 = 0x01, - DOVECOT_SSL_PROTO_SSLv3 = 0x02, - DOVECOT_SSL_PROTO_TLSv1 = 0x04, - DOVECOT_SSL_PROTO_ALL = 0x07 + DOVECOT_SSL_PROTO_SSLv2 = 0x01, + DOVECOT_SSL_PROTO_SSLv3 = 0x02, + DOVECOT_SSL_PROTO_TLSv1 = 0x04, + DOVECOT_SSL_PROTO_TLSv1_1 = 0x08, + DOVECOT_SSL_PROTO_TLSv1_2 = 0x10, + DOVECOT_SSL_PROTO_ALL = 0x1f }; int openssl_get_protocol_options(const char *protocols) @@ -34,6 +36,14 @@ proto = DOVECOT_SSL_PROTO_SSLv3; else if (strcasecmp(name, SSL_TXT_TLSV1) == 0) proto = DOVECOT_SSL_PROTO_TLSv1; +#ifdef SSL_TXT_TLSV1_1 + else if (strcasecmp(name, SSL_TXT_TLSV1_1) == 0) + proto = DOVECOT_SSL_PROTO_TLSv1_1; +#endif +#ifdef SSL_TXT_TLSV1_2 + else if (strcasecmp(name, SSL_TXT_TLSV1_2) == 0) + proto = DOVECOT_SSL_PROTO_TLSv1_2; +#endif else { i_fatal("Invalid ssl_protocols setting: " "Unknown protocol '%s'", name); @@ -51,6 +61,12 @@ if ((exclude & DOVECOT_SSL_PROTO_SSLv2) != 0) op |= SSL_OP_NO_SSLv2; if ((exclude & DOVECOT_SSL_PROTO_SSLv3) != 0) op |= SSL_OP_NO_SSLv3; if ((exclude & DOVECOT_SSL_PROTO_TLSv1) != 0) op |= SSL_OP_NO_TLSv1; +#ifdef SSL_OP_NO_TLSv1_1 + if ((exclude & DOVECOT_SSL_PROTO_TLSv1_1) != 0) op |= SSL_OP_NO_TLSv1_1; +#endif +#ifdef SSL_OP_NO_TLSv1_2 + if ((exclude & DOVECOT_SSL_PROTO_TLSv1_2) != 0) op |= SSL_OP_NO_TLSv1_2; +#endif return op; } From dovecot at dovecot.org Thu Jul 11 00:29:00 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 00:29:00 +0300 Subject: dovecot-2.2: lib-ssl-iostream: Compiler warning fix Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/dff200f433d2 changeset: 16594:dff200f433d2 user: Timo Sirainen date: Thu Jul 11 00:28:54 2013 +0300 description: lib-ssl-iostream: Compiler warning fix diffstat: src/lib-ssl-iostream/iostream-openssl-context.c | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-) diffs (15 lines): diff -r 3a08bd30d180 -r dff200f433d2 src/lib-ssl-iostream/iostream-openssl-context.c --- a/src/lib-ssl-iostream/iostream-openssl-context.c Wed Jul 10 10:01:26 2013 +0300 +++ b/src/lib-ssl-iostream/iostream-openssl-context.c Thu Jul 11 00:28:54 2013 +0300 @@ -413,11 +413,9 @@ { int nid = 0; EVP_PKEY *pkey; - const char *password; EC_KEY *eckey; const EC_GROUP *ecgrp; - password = set->key_password[0] != '\0' ? set->key_password : NULL; if (openssl_iostream_load_key(set, &pkey, error_r) < 0) return -1; From dovecot at dovecot.org Thu Jul 11 00:30:40 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 00:30:40 +0300 Subject: dovecot-2.2: lib-http: Http requests weren't freed when connecti... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a99b211f0409 changeset: 16595:a99b211f0409 user: Timo Sirainen date: Thu Jul 11 00:30:28 2013 +0300 description: lib-http: Http requests weren't freed when connection was aborted. Patch by Stephan Bosch. diffstat: src/lib-http/http-client-connection.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r dff200f433d2 -r a99b211f0409 src/lib-http/http-client-connection.c --- a/src/lib-http/http-client-connection.c Thu Jul 11 00:28:54 2013 +0300 +++ b/src/lib-http/http-client-connection.c Thu Jul 11 00:30:28 2013 +0300 @@ -119,6 +119,7 @@ array_foreach_modifiable(&conn->request_wait_list, req) { i_assert((*req)->submitted); http_client_request_error(*req, status, error); + http_client_request_unref(req); } array_clear(&conn->request_wait_list); http_client_connection_unref(_conn); From pigeonhole at rename-it.nl Thu Jul 11 00:34:42 2013 From: pigeonhole at rename-it.nl (pigeonhole at rename-it.nl) Date: Wed, 10 Jul 2013 23:34:42 +0200 Subject: dovecot-2.2-pigeonhole: Updated Sieve extension status in comments. Message-ID: details: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/a6a1c041c209 changeset: 1788:a6a1c041c209 user: Stephan Bosch date: Wed Jul 10 23:34:34 2013 +0200 description: Updated Sieve extension status in comments. diffstat: src/lib-sieve/plugins/editheader/ext-editheader.c | 4 ++-- src/lib-sieve/plugins/environment/ext-environment.c | 4 ++-- src/lib-sieve/plugins/ihave/ext-ihave.c | 2 +- src/lib-sieve/plugins/include/ext-include.c | 5 ++--- src/lib-sieve/plugins/mailbox/ext-mailbox.c | 2 +- src/lib-sieve/plugins/spamvirustest/ext-spamvirustest.c | 2 +- src/lib-sieve/plugins/vacation/ext-vacation-seconds.c | 2 +- src/lib-sieve/plugins/vnd.dovecot/debug/ext-debug.c | 2 +- src/lib-sieve/plugins/vnd.dovecot/duplicate/ext-duplicate.c | 2 +- src/plugins/sieve-extprograms/ext-execute.c | 10 +++++----- src/plugins/sieve-extprograms/ext-filter.c | 10 +++++----- src/plugins/sieve-extprograms/ext-pipe.c | 10 +++++----- 12 files changed, 27 insertions(+), 28 deletions(-) diffs (220 lines): diff -r 612d91d3b745 -r a6a1c041c209 src/lib-sieve/plugins/editheader/ext-editheader.c --- a/src/lib-sieve/plugins/editheader/ext-editheader.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/lib-sieve/plugins/editheader/ext-editheader.c Wed Jul 10 23:34:34 2013 +0200 @@ -6,8 +6,8 @@ * * Authors: Stephan Bosch * Specification: RFC 5293 - * Implementation: partial - * Status: experimental + * Implementation: full + * Status: testing * */ diff -r 612d91d3b745 -r a6a1c041c209 src/lib-sieve/plugins/environment/ext-environment.c --- a/src/lib-sieve/plugins/environment/ext-environment.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/lib-sieve/plugins/environment/ext-environment.c Wed Jul 10 23:34:34 2013 +0200 @@ -6,8 +6,8 @@ * * Authors: Stephan Bosch * Specification: RFC 5183 - * Implementation: basic - * Status: experimental, not thoroughly tested + * Implementation: full + * Status: testing * */ diff -r 612d91d3b745 -r a6a1c041c209 src/lib-sieve/plugins/ihave/ext-ihave.c --- a/src/lib-sieve/plugins/ihave/ext-ihave.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/lib-sieve/plugins/ihave/ext-ihave.c Wed Jul 10 23:34:34 2013 +0200 @@ -7,7 +7,7 @@ * Authors: Stephan Bosch * Specification: RFC 5463 * Implementation: full - * Status: experimental + * Status: testing * */ diff -r 612d91d3b745 -r a6a1c041c209 src/lib-sieve/plugins/include/ext-include.c --- a/src/lib-sieve/plugins/include/ext-include.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/lib-sieve/plugins/include/ext-include.c Wed Jul 10 23:34:34 2013 +0200 @@ -5,9 +5,8 @@ * ----------------- * * Authors: Stephan Bosch - * Specification: draft-ietf-sieve-include-01 - * Implementation: mostly full, but required ManageSieve behavior is not - * implemented + * Specification: RFC 6609 + * Implementation: full * Status: testing * */ diff -r 612d91d3b745 -r a6a1c041c209 src/lib-sieve/plugins/mailbox/ext-mailbox.c --- a/src/lib-sieve/plugins/mailbox/ext-mailbox.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/lib-sieve/plugins/mailbox/ext-mailbox.c Wed Jul 10 23:34:34 2013 +0200 @@ -6,7 +6,7 @@ * * Authors: Stephan Bosch * Specification: RFC 5490 - * Implementation: almost full; acl support is missing for mailboxexists + * Implementation: full * Status: testing * */ diff -r 612d91d3b745 -r a6a1c041c209 src/lib-sieve/plugins/spamvirustest/ext-spamvirustest.c --- a/src/lib-sieve/plugins/spamvirustest/ext-spamvirustest.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/lib-sieve/plugins/spamvirustest/ext-spamvirustest.c Wed Jul 10 23:34:34 2013 +0200 @@ -7,7 +7,7 @@ * Authors: Stephan Bosch * Specification: RFC 5235 * Implementation: full - * Status: experimental + * Status: testing * */ diff -r 612d91d3b745 -r a6a1c041c209 src/lib-sieve/plugins/vacation/ext-vacation-seconds.c --- a/src/lib-sieve/plugins/vacation/ext-vacation-seconds.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/lib-sieve/plugins/vacation/ext-vacation-seconds.c Wed Jul 10 23:34:34 2013 +0200 @@ -5,7 +5,7 @@ * -------------------------- * * Authors: Stephan Bosch - * Specification: draft-ietf-sieve-vacation-seconds-03 + * Specification: RFC 6131 * Implementation: full * Status: testing * diff -r 612d91d3b745 -r a6a1c041c209 src/lib-sieve/plugins/vnd.dovecot/debug/ext-debug.c --- a/src/lib-sieve/plugins/vnd.dovecot/debug/ext-debug.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/lib-sieve/plugins/vnd.dovecot/debug/ext-debug.c Wed Jul 10 23:34:34 2013 +0200 @@ -5,7 +5,7 @@ * --------------- * * Authors: Stephan Bosch - * Specification: vendor-defined + * Specification: vendor-defined; spec-bosch-sieve-debug * Implementation: full * Status: experimental * diff -r 612d91d3b745 -r a6a1c041c209 src/lib-sieve/plugins/vnd.dovecot/duplicate/ext-duplicate.c --- a/src/lib-sieve/plugins/vnd.dovecot/duplicate/ext-duplicate.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/lib-sieve/plugins/vnd.dovecot/duplicate/ext-duplicate.c Wed Jul 10 23:34:34 2013 +0200 @@ -5,7 +5,7 @@ * ------------------------------- * * Authors: Stephan Bosch - * Specification: spec-bosch-sieve-duplicate + * Specification: vendor-defined; spec-bosch-sieve-duplicate * Implementation: full * Status: experimental * diff -r 612d91d3b745 -r a6a1c041c209 src/plugins/sieve-extprograms/ext-execute.c --- a/src/plugins/sieve-extprograms/ext-execute.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/plugins/sieve-extprograms/ext-execute.c Wed Jul 10 23:34:34 2013 +0200 @@ -5,17 +5,17 @@ * ----------------------------- * * Authors: Stephan Bosch - * Specification: spec-bosch-sieve-extprograms + * Specification: vendor-defined; spec-bosch-sieve-extprograms * Implementation: full * Status: experimental * */ - + #include "lib.h" #include "sieve-extensions.h" #include "sieve-commands.h" -#include "sieve-binary.h" +#include "sieve-binary.h" #include "sieve-validator.h" #include "sieve-interpreter.h" @@ -24,8 +24,8 @@ #include "sieve-extprograms-common.h" -/* - * Extension +/* + * Extension */ static bool ext_execute_load(const struct sieve_extension *ext, void **context); diff -r 612d91d3b745 -r a6a1c041c209 src/plugins/sieve-extprograms/ext-filter.c --- a/src/plugins/sieve-extprograms/ext-filter.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/plugins/sieve-extprograms/ext-filter.c Wed Jul 10 23:34:34 2013 +0200 @@ -5,17 +5,17 @@ * ----------------------------- * * Authors: Stephan Bosch - * Specification: spec-bosch-sieve-extprograms + * Specification: vendor-defined; spec-bosch-sieve-extprograms * Implementation: full * Status: experimental * */ - + #include "lib.h" #include "sieve-extensions.h" #include "sieve-commands.h" -#include "sieve-binary.h" +#include "sieve-binary.h" #include "sieve-validator.h" #include "sieve-interpreter.h" @@ -24,8 +24,8 @@ #include "sieve-extprograms-common.h" -/* - * Extension +/* + * Extension */ static bool ext_filter_load(const struct sieve_extension *ext, void **context); diff -r 612d91d3b745 -r a6a1c041c209 src/plugins/sieve-extprograms/ext-pipe.c --- a/src/plugins/sieve-extprograms/ext-pipe.c Wed Jul 03 22:06:58 2013 +0200 +++ b/src/plugins/sieve-extprograms/ext-pipe.c Wed Jul 10 23:34:34 2013 +0200 @@ -5,17 +5,17 @@ * ----------------------------- * * Authors: Stephan Bosch - * Specification: spec-bosch-sieve-extprograms + * Specification: vendor-define; spec-bosch-sieve-extprograms * Implementation: full * Status: experimental * */ - + #include "lib.h" #include "sieve-extensions.h" #include "sieve-commands.h" -#include "sieve-binary.h" +#include "sieve-binary.h" #include "sieve-validator.h" #include "sieve-interpreter.h" @@ -24,8 +24,8 @@ #include "sieve-extprograms-common.h" -/* - * Extension +/* + * Extension */ static bool ext_pipe_load(const struct sieve_extension *ext, void **context); From dovecot at dovecot.org Thu Jul 11 09:27:04 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 09:27:04 +0300 Subject: dovecot-2.2: lib-http: Simplify http response status-line parsin... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d7244f7ceca4 changeset: 16596:d7244f7ceca4 user: Timo Sirainen date: Thu Jul 11 09:23:14 2013 +0300 description: lib-http: Simplify http response status-line parsing code. Also by not using a temporary string buffer the istream can at least in theory limit the maximum status-line length (=max memory usage). diffstat: src/lib-http/http-response-parser.c | 83 ++++++++++-------------------------- 1 files changed, 24 insertions(+), 59 deletions(-) diffs (165 lines): diff -r a99b211f0409 -r d7244f7ceca4 src/lib-http/http-response-parser.c --- a/src/lib-http/http-response-parser.c Thu Jul 11 00:30:28 2013 +0300 +++ b/src/lib-http/http-response-parser.c Thu Jul 11 09:23:14 2013 +0300 @@ -32,8 +32,6 @@ const unsigned char *begin, *cur, *end; const char *error; - string_t *strbuf; - enum http_response_parser_state state; struct http_header_parser *header_parser; @@ -51,7 +49,6 @@ parser = i_new(struct http_response_parser, 1); parser->input = input; - parser->strbuf = str_new(default_pool, 128); return parser; } @@ -59,7 +56,6 @@ { struct http_response_parser *parser = *_parser; - str_free(&parser->strbuf); if (parser->header_parser != NULL) http_header_parser_deinit(&parser->header_parser); if (parser->response_pool != NULL) @@ -75,7 +71,6 @@ i_assert(parser->payload == NULL); parser->content_length = 0; parser->transfer_encoding = NULL; - str_truncate(parser->strbuf, 0); if (parser->response_pool != NULL) pool_unref(&parser->response_pool); parser->response_pool = pool_alloconly_create("http_response", 4096); @@ -86,85 +81,54 @@ static int http_response_parse_version(struct http_response_parser *parser) { - const unsigned char *first = parser->cur; - const char *p; + const unsigned char *p = parser->cur; + const size_t size = parser->end - parser->cur; /* HTTP-version = HTTP-name "/" DIGIT "." DIGIT HTTP-name = %x48.54.54.50 ; "HTTP", case-sensitive */ - while (parser->cur < parser->end && http_char_is_value(*parser->cur)) - parser->cur++; - - if (str_len(parser->strbuf) + (parser->cur-first) > 8) + if (size < 8) + return 0; + if (memcmp(p, "HTTP/", 5) != 0 || + !i_isdigit(p[5]) || p[6] != '.' || !i_isdigit(p[7])) return -1; - - if ((parser->cur - first) > 0) - str_append_n(parser->strbuf, first, parser->cur-first); - if (parser->cur == parser->end) - return 0; - - if (str_len(parser->strbuf) != 8) - return -1; - if (strncmp(str_c(parser->strbuf), "HTTP/",5) != 0) - return -1; - p = str_c(parser->strbuf) + 5; - if (!i_isdigit(*p)) - return -1; - parser->response->version_major = *p - '0'; - p++; - if (*(p++) != '.') - return -1; - if (!i_isdigit(*p)) - return -1; - parser->response->version_minor = *p - '0'; - str_truncate(parser->strbuf, 0); + parser->response->version_major = p[5] - '0'; + parser->response->version_minor = p[7] - '0'; + parser->cur += 8; return 1; } static int http_response_parse_status(struct http_response_parser *parser) { - const unsigned char *first = parser->cur; - const unsigned char *p; + const unsigned char *p = parser->cur; + const size_t size = parser->end - parser->cur; /* status-code = 3DIGIT */ - while (parser->cur < parser->end && i_isdigit(*parser->cur)) { - parser->cur++; - if ((parser->cur - first) > 3) - return -1; - } - - if (str_len(parser->strbuf) + (parser->cur - first) > 3) + if (size < 3) + return 0; + if (!i_isdigit(p[0]) || !i_isdigit(p[1]) || !i_isdigit(p[2])) return -1; - if ((parser->cur - first) > 0) - str_append_n(parser->strbuf, first, parser->cur-first); - if (parser->cur == parser->end) - return 0; - if (str_len(parser->strbuf) != 3) - return -1; - p = str_data(parser->strbuf); parser->response->status = (p[0] - '0')*100 + (p[1] - '0')*10 + (p[2] - '0'); - str_truncate(parser->strbuf, 0); + parser->cur += 3; return 1; } static int http_response_parse_reason(struct http_response_parser *parser) { - const unsigned char *first = parser->cur; + const unsigned char *p = parser->cur; /* reason-phrase = *( HTAB / SP / VCHAR / obs-text ) */ - while (parser->cur < parser->end && http_char_is_text(*parser->cur)) - parser->cur++; + while (p < parser->end && http_char_is_text(*p)) + p++; - if ((parser->cur - first) > 0) - str_append_n(parser->strbuf, first, parser->cur-first); - if (parser->cur == parser->end) + if (p == parser->end) return 0; parser->response->reason = - p_strdup(parser->response_pool, str_c(parser->strbuf)); - str_truncate(parser->strbuf, 0); + p_strdup_until(parser->response_pool, parser->cur, p); + parser->cur = p; return 1; } @@ -270,11 +234,11 @@ static int http_response_parse_status_line(struct http_response_parser *parser) { - size_t size; + size_t size, old_bytes = 0; int ret; while ((ret = i_stream_read_data(parser->input, - &parser->begin, &size, 0)) > 0) { + &parser->begin, &size, old_bytes)) > 0) { parser->cur = parser->begin; parser->end = parser->cur + size; @@ -284,6 +248,7 @@ i_stream_skip(parser->input, parser->cur - parser->begin); if (ret > 0) return 1; + old_bytes = i_stream_get_data_size(parser->input); } i_assert(ret != -2); From dovecot at dovecot.org Thu Jul 11 09:27:04 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 09:27:04 +0300 Subject: dovecot-2.2: lib-http: Fixed handling responses whose header arr... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/07fb7d4665c6 changeset: 16597:07fb7d4665c6 user: Timo Sirainen date: Thu Jul 11 09:25:53 2013 +0300 description: lib-http: Fixed handling responses whose header arrives in smaller pieces. diffstat: src/lib-http/http-response-parser.c | 11 +++++------ src/lib-http/test-http-response-parser.c | 16 ++++++++++++---- 2 files changed, 17 insertions(+), 10 deletions(-) diffs (65 lines): diff -r d7244f7ceca4 -r 07fb7d4665c6 src/lib-http/http-response-parser.c --- a/src/lib-http/http-response-parser.c Thu Jul 11 09:23:14 2013 +0300 +++ b/src/lib-http/http-response-parser.c Thu Jul 11 09:25:53 2013 +0300 @@ -376,14 +376,13 @@ *error_r = parser->error; return ret; } + /* *( header-field CRLF ) CRLF */ + if (parser->header_parser == NULL) + parser->header_parser = http_header_parser_init(parser->input); + else + http_header_parser_reset(parser->header_parser); } - /* *( header-field CRLF ) CRLF */ - if (parser->header_parser == NULL) - parser->header_parser = http_header_parser_init(parser->input); - else - http_header_parser_reset(parser->header_parser); - while ((ret=http_header_parse_next_field (parser->header_parser, &field_name, &field_data, &field_size, &error)) > 0) { if (field_name == NULL) break; diff -r d7244f7ceca4 -r 07fb7d4665c6 src/lib-http/test-http-response-parser.c --- a/src/lib-http/test-http-response-parser.c Thu Jul 11 09:23:14 2013 +0300 +++ b/src/lib-http/test-http-response-parser.c Thu Jul 11 09:25:53 2013 +0300 @@ -99,19 +99,26 @@ struct ostream *output; const struct http_response_parse_test *test; struct http_response_parser *parser; - struct http_response *response; + struct http_response *response = NULL; const char *response_text, *payload, *error; - int ret; + unsigned int response_text_len; + int ret = 0; test = &valid_response_parse_tests[i]; response_text = test->response; - input = i_stream_create_from_data(response_text, strlen(response_text)); + response_text_len = strlen(response_text); + input = test_istream_create_data(response_text, response_text_len); parser = http_response_parser_init(input); test_begin(t_strdup_printf("http response valid [%d]", i)); payload = NULL; - while ((ret=http_response_parse_next(parser, FALSE, &response, &error)) > 0) { + for (i = 0; i < response_text_len && ret == 0; i++) { + test_istream_set_size(input, i); + ret = http_response_parse_next(parser, FALSE, &response, &error); + } + test_istream_set_size(input, response_text_len); + while (ret > 0) { if (response->payload != NULL) { buffer_set_used_size(payload_buffer, 0); output = o_stream_create_buffer(payload_buffer); @@ -122,6 +129,7 @@ } else { payload = NULL; } + ret = http_response_parse_next(parser, FALSE, &response, &error); } test_out("parse success", ret == 0); From dovecot at dovecot.org Thu Jul 11 09:29:51 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 09:29:51 +0300 Subject: dovecot-2.2: lib-http: Fixed previous test-http-response-parser ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/55ae488349d2 changeset: 16598:55ae488349d2 user: Timo Sirainen date: Thu Jul 11 09:29:36 2013 +0300 description: lib-http: Fixed previous test-http-response-parser change. diffstat: src/lib-http/test-http-response-parser.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diffs (23 lines): diff -r 07fb7d4665c6 -r 55ae488349d2 src/lib-http/test-http-response-parser.c --- a/src/lib-http/test-http-response-parser.c Thu Jul 11 09:25:53 2013 +0300 +++ b/src/lib-http/test-http-response-parser.c Thu Jul 11 09:29:36 2013 +0300 @@ -101,7 +101,7 @@ struct http_response_parser *parser; struct http_response *response = NULL; const char *response_text, *payload, *error; - unsigned int response_text_len; + unsigned int pos, response_text_len; int ret = 0; test = &valid_response_parse_tests[i]; @@ -113,8 +113,8 @@ test_begin(t_strdup_printf("http response valid [%d]", i)); payload = NULL; - for (i = 0; i < response_text_len && ret == 0; i++) { - test_istream_set_size(input, i); + for (pos = 0; pos < response_text_len && ret == 0; pos++) { + test_istream_set_size(input, pos); ret = http_response_parse_next(parser, FALSE, &response, &error); } test_istream_set_size(input, response_text_len); From dovecot at dovecot.org Thu Jul 11 09:33:19 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 09:33:19 +0300 Subject: dovecot-2.2: lib-http: test-http-header-parser makes sure the pa... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/feac6ec9fa6e changeset: 16599:feac6ec9fa6e user: Timo Sirainen date: Thu Jul 11 09:33:13 2013 +0300 description: lib-http: test-http-header-parser makes sure the parser works one byte at a time. diffstat: src/lib-http/test-http-header-parser.c | 16 ++++++++++++---- 1 files changed, 12 insertions(+), 4 deletions(-) diffs (36 lines): diff -r 55ae488349d2 -r feac6ec9fa6e src/lib-http/test-http-header-parser.c --- a/src/lib-http/test-http-header-parser.c Thu Jul 11 09:29:36 2013 +0300 +++ b/src/lib-http/test-http-header-parser.c Thu Jul 11 09:33:13 2013 +0300 @@ -127,20 +127,28 @@ const unsigned char *field_data; size_t field_size; int ret; - int j; + unsigned int j, pos, header_len; header = valid_header_parse_tests[i].header; - input = i_stream_create_from_data(header, strlen(header)); + header_len = strlen(header); + input = test_istream_create_data(header, header_len); parser = http_header_parser_init(input); test_begin(t_strdup_printf("http header valid [%d]", i)); - j = 0; + j = 0; pos = 0; test_istream_set_size(input, 0); while ((ret=http_header_parse_next_field - (parser, &field_name, &field_data, &field_size, &error)) > 0) { + (parser, &field_name, &field_data, &field_size, &error)) >= 0) { const struct http_header_parse_result *result; const char *field_value; + if (ret == 0) { + if (pos == header_len) + break; + test_istream_set_size(input, ++pos); + continue; + } + if (field_name == NULL) break; result = &valid_header_parse_tests[i].fields[j]; From dovecot at dovecot.org Thu Jul 11 10:18:03 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 10:18:03 +0300 Subject: dovecot-2.2: lib-http: Updated test-http-response-parser Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/1ab7d7dd3087 changeset: 16600:1ab7d7dd3087 user: Timo Sirainen date: Thu Jul 11 10:17:49 2013 +0300 description: lib-http: Updated test-http-response-parser diffstat: src/lib-http/test-http-response-parser.c | 27 +++++++++++++++++++++------ 1 files changed, 21 insertions(+), 6 deletions(-) diffs (56 lines): diff -r feac6ec9fa6e -r 1ab7d7dd3087 src/lib-http/test-http-response-parser.c --- a/src/lib-http/test-http-response-parser.c Thu Jul 11 09:33:13 2013 +0300 +++ b/src/lib-http/test-http-response-parser.c Thu Jul 11 10:17:49 2013 +0300 @@ -173,20 +173,25 @@ "Cache-Control: private\n\r" }; +static unsigned char invalid_response_with_nuls[] = + "HTTP/1.1 200 OK\r\n" + "Server: text\0server\r\n" + "\r\n"; + unsigned int invalid_response_parse_test_count = N_ELEMENTS(invalid_response_parse_tests); static void test_http_response_parse_invalid(void) { + struct http_response_parser *parser; + struct http_response *response; + const char *response_text, *error; + struct istream *input; + int ret; unsigned int i; for (i = 0; i < invalid_response_parse_test_count; i++) T_BEGIN { - struct istream *input; const char *test; - struct http_response_parser *parser; - struct http_response *response; - const char *response_text, *error; - int ret; test = invalid_response_parse_tests[i]; response_text = test; @@ -197,10 +202,20 @@ while ((ret=http_response_parse_next(parser, FALSE, &response, &error)) > 0); - test_out("parse failure", ret < 0); + test_assert(ret < 0); test_end(); http_response_parser_deinit(&parser); } T_END; + + /* parse failure guarantees http_response_header.size equals + strlen(http_response_header.value) */ + test_begin("http response with NULs"); + input = i_stream_create_from_data(invalid_response_with_nuls, + sizeof(invalid_response_with_nuls)-1); + parser = http_response_parser_init(input); + while ((ret=http_response_parse_next(parser, FALSE, &response, &error)) > 0); + test_assert(ret < 0); + test_end(); } int main(void) From dovecot at dovecot.org Thu Jul 11 10:20:02 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 10:20:02 +0300 Subject: dovecot-2.2: lib-http: Minor fix/cleanup to detecting stream err... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/dbda2f84aeaa changeset: 16601:dbda2f84aeaa user: Timo Sirainen date: Thu Jul 11 10:19:43 2013 +0300 description: lib-http: Minor fix/cleanup to detecting stream errors when skipping payload. diffstat: src/lib-http/http-response-parser.c | 22 ++++++++-------------- 1 files changed, 8 insertions(+), 14 deletions(-) diffs (37 lines): diff -r 1ab7d7dd3087 -r dbda2f84aeaa src/lib-http/http-response-parser.c --- a/src/lib-http/http-response-parser.c Thu Jul 11 10:17:49 2013 +0300 +++ b/src/lib-http/http-response-parser.c Thu Jul 11 10:19:43 2013 +0300 @@ -343,25 +343,19 @@ before we continue. */ if (parser->payload != NULL) { struct istream *payload = parser->payload; + const unsigned char *data; + size_t size; i_assert(parser->state == HTTP_RESPONSE_PARSE_STATE_INIT); - if (i_stream_have_bytes_left(payload)) { - do { - i_stream_skip(payload, i_stream_get_data_size(payload)); - } while ((ret=i_stream_read(payload)) > 0); - if (ret == 0) - return 0; - if (ret < 0 && !payload->eof) { + while ((ret = i_stream_read_data(payload, &data, &size, 0)) > 0) + i_stream_skip(payload, size); + if (ret == 0 || payload->stream_errno != 0) { + if (ret < 0) *error_r = "Stream error while skipping payload"; - return -1; - } + return ret; } - - if (payload->eof) { - i_stream_unref(&parser->payload); - parser->payload = NULL; - } + i_stream_unref(&parser->payload); } /* HTTP-message = start-line From dovecot at dovecot.org Thu Jul 11 10:26:07 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 10:26:07 +0300 Subject: dovecot-2.2: lib-http: Minor change to make sure http_response_h... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/69cc86583da2 changeset: 16602:69cc86583da2 user: Timo Sirainen date: Thu Jul 11 10:25:32 2013 +0300 description: lib-http: Minor change to make sure http_response_header.size is always correct. The current http-header-parser already guaranteed that it is, but this change just adds extra guarantees that it won't break in future. Besides, this change improves the performance slightly by avoiding strlen(). diffstat: src/lib-http/http-response-parser.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diffs (17 lines): diff -r dbda2f84aeaa -r 69cc86583da2 src/lib-http/http-response-parser.c --- a/src/lib-http/http-response-parser.c Thu Jul 11 10:19:43 2013 +0300 +++ b/src/lib-http/http-response-parser.c Thu Jul 11 10:25:32 2013 +0300 @@ -267,10 +267,12 @@ { struct http_response_header *hdr; struct http_parser hparser; + void *value; hdr = array_append_space(&parser->response->headers); hdr->key = p_strdup(parser->response_pool, name); - hdr->value = p_strndup(parser->response_pool, data, size); + hdr->value = value = p_malloc(parser->response_pool, size+1); + memcpy(value, data, size); hdr->size = size; switch (name[0]) { From dovecot at dovecot.org Thu Jul 11 14:19:03 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 14:19:03 +0300 Subject: dovecot-2.2: lib-http: Moved struct http_response* to http-respo... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c5952dd642c6 changeset: 16603:c5952dd642c6 user: Timo Sirainen date: Thu Jul 11 12:54:02 2013 +0300 description: lib-http: Moved struct http_response* to http-response.h Also http-client.h now #includes only http-response.h diffstat: src/lib-http/Makefile.am | 1 + src/lib-http/http-client.h | 2 +- src/lib-http/http-response-parser.h | 23 +---------------------- src/lib-http/http-response.h | 28 ++++++++++++++++++++++++++++ 4 files changed, 31 insertions(+), 23 deletions(-) diffs (88 lines): diff -r 69cc86583da2 -r c5952dd642c6 src/lib-http/Makefile.am --- a/src/lib-http/Makefile.am Thu Jul 11 10:25:32 2013 +0300 +++ b/src/lib-http/Makefile.am Thu Jul 11 12:54:02 2013 +0300 @@ -25,6 +25,7 @@ http-parser.h \ http-header-parser.h \ http-transfer.h \ + http-response.h \ http-response-parser.h \ http-client-private.h \ http-client.h diff -r 69cc86583da2 -r c5952dd642c6 src/lib-http/http-client.h --- a/src/lib-http/http-client.h Thu Jul 11 10:25:32 2013 +0300 +++ b/src/lib-http/http-client.h Thu Jul 11 12:54:02 2013 +0300 @@ -1,7 +1,7 @@ #ifndef HTTP_CLIENT_H #define HTTP_CLIENT_H -#include "http-response-parser.h" +#include "http-response.h" struct http_response; diff -r 69cc86583da2 -r c5952dd642c6 src/lib-http/http-response-parser.h --- a/src/lib-http/http-response-parser.h Thu Jul 11 10:25:32 2013 +0300 +++ b/src/lib-http/http-response-parser.h Thu Jul 11 12:54:02 2013 +0300 @@ -1,28 +1,7 @@ #ifndef HTTP_RESPONSE_PARSER_H #define HTTP_RESPONSE_PARSER_H -struct http_response_header { - const char *key; - const char *value; - size_t size; -}; - -struct http_response { - unsigned char version_major; - unsigned char version_minor; - - unsigned int status; - - const char *reason; - const char *location; - - time_t date; - struct istream *payload; - - ARRAY(struct http_response_header) headers; - - unsigned int connection_close:1; -}; +#include "http-response.h" struct http_response_parser; diff -r 69cc86583da2 -r c5952dd642c6 src/lib-http/http-response.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/lib-http/http-response.h Thu Jul 11 12:54:02 2013 +0300 @@ -0,0 +1,28 @@ +#ifndef HTTP_RESPONSE_H +#define HTTP_RESPONSE_H + +struct http_response_header { + const char *key; + const char *value; + size_t size; +}; +ARRAY_DEFINE_TYPE(http_response_header, struct http_response_header); + +struct http_response { + unsigned char version_major; + unsigned char version_minor; + + unsigned int status; + + const char *reason; + const char *location; + + time_t date; + struct istream *payload; + + ARRAY_TYPE(http_response_header) headers; + + unsigned int connection_close:1; +}; + +#endif From dovecot at dovecot.org Thu Jul 11 14:19:03 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 14:19:03 +0300 Subject: dovecot-2.2: lib-http: Implemented http-request-parser for HTTP ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c55bd41133a2 changeset: 16604:c55bd41133a2 user: Timo Sirainen date: Thu Jul 11 14:18:42 2013 +0300 description: lib-http: Implemented http-request-parser for HTTP servers. Moved code common with it and http-response-parser to http-message-parser. diffstat: src/lib-http/Makefile.am | 5 + src/lib-http/http-message-parser.c | 221 +++++++++++++++++++++++ src/lib-http/http-message-parser.h | 45 ++++ src/lib-http/http-request-parser.c | 267 ++++++++++++++++++++++++++++ src/lib-http/http-request-parser.h | 29 +++ src/lib-http/http-response-parser.c | 338 ++++++++--------------------------- 6 files changed, 651 insertions(+), 254 deletions(-) diffs (truncated from 1129 to 300 lines): diff -r c5952dd642c6 -r c55bd41133a2 src/lib-http/Makefile.am --- a/src/lib-http/Makefile.am Thu Jul 11 12:54:02 2013 +0300 +++ b/src/lib-http/Makefile.am Thu Jul 11 14:18:42 2013 +0300 @@ -12,6 +12,8 @@ http-parser.c \ http-header-parser.c \ http-transfer-chunked.c \ + http-message-parser.c \ + http-request-parser.c \ http-response-parser.c \ http-client-request.c \ http-client-connection.c \ @@ -25,6 +27,8 @@ http-parser.h \ http-header-parser.h \ http-transfer.h \ + http-message-parser.h \ + http-request-parser.h \ http-response.h \ http-response-parser.h \ http-client-private.h \ @@ -81,6 +85,7 @@ http-parser.lo \ http-header-parser.lo \ http-transfer-chunked.lo \ + http-message-parser.lo \ http-response-parser.lo \ $(test_libs) test_http_response_parser_DEPENDENCIES = $(test_deps) diff -r c5952dd642c6 -r c55bd41133a2 src/lib-http/http-message-parser.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/lib-http/http-message-parser.c Thu Jul 11 14:18:42 2013 +0300 @@ -0,0 +1,221 @@ +/* Copyright (c) 2013 Dovecot authors, see the included COPYING file */ + +#include "lib.h" +#include "array.h" +#include "istream.h" +#include "http-parser.h" +#include "http-header-parser.h" +#include "http-date.h" +#include "http-transfer.h" +#include "http-message-parser.h" + +#include + +void http_message_parser_init(struct http_message_parser *parser, + struct istream *input) +{ + memset(parser, 0, sizeof(*parser)); + parser->input = input; +} + +void http_message_parser_deinit(struct http_message_parser *parser) +{ + if (parser->header_parser != NULL) + http_header_parser_deinit(&parser->header_parser); + if (parser->msg_pool != NULL) + pool_unref(&parser->msg_pool); + if (parser->payload != NULL) + i_stream_unref(&parser->payload); +} + +void http_message_parser_restart(struct http_message_parser *parser) +{ + i_assert(parser->payload == NULL); + + if (parser->header_parser == NULL) + parser->header_parser = http_header_parser_init(parser->input); + else + http_header_parser_reset(parser->header_parser); + + if (parser->msg_pool != NULL) + pool_unref(&parser->msg_pool); + parser->msg_pool = pool_alloconly_create("http_message", 4096); + memset(&parser->msg, 0, sizeof(parser->msg)); + parser->msg.date = (time_t)-1; + p_array_init(&parser->msg.headers, parser->msg_pool, 32); +} + +int http_message_parse_version(struct http_message_parser *parser) +{ + const unsigned char *p = parser->cur; + const size_t size = parser->end - parser->cur; + + /* HTTP-version = HTTP-name "/" DIGIT "." DIGIT + HTTP-name = %x48.54.54.50 ; "HTTP", case-sensitive + */ + if (size < 8) + return 0; + if (memcmp(p, "HTTP/", 5) != 0 || + !i_isdigit(p[5]) || p[6] != '.' || !i_isdigit(p[7])) + return -1; + parser->msg.version_major = p[5] - '0'; + parser->msg.version_minor = p[7] - '0'; + parser->cur += 8; + return 1; +} + +int http_message_parse_finish_payload(struct http_message_parser *parser, + const char **error_r) +{ + const unsigned char *data; + size_t size; + int ret; + + if (parser->payload == NULL) + return 1; + + while ((ret = i_stream_read_data(parser->payload, &data, &size, 0)) > 0) + i_stream_skip(parser->payload, size); + if (ret == 0 || parser->payload->stream_errno != 0) { + if (ret < 0) + *error_r = "Stream error while skipping payload"; + return ret; + } + i_stream_unref(&parser->payload); + return 1; +} + +static int +http_message_parse_header(struct http_message_parser *parser, const char *name, + const unsigned char *data, size_t size, + const char **error_r) +{ + struct http_response_header *hdr; + struct http_parser hparser; + void *value; + + hdr = array_append_space(&parser->msg.headers); + hdr->key = p_strdup(parser->msg_pool, name); + hdr->value = value = p_malloc(parser->msg_pool, size+1); + memcpy(value, data, size); + hdr->size = size; + + switch (name[0]) { + case 'C': case 'c': + if (strcasecmp(name, "Connection") == 0) { + const char *option; + + /* Connection = 1#connection-option + connection-option = token + */ + http_parser_init(&hparser, data, size); + for (;;) { + if (http_parse_token_list_next(&hparser, &option) <= 0) + break; + if (strcasecmp(option, "close") == 0) { + parser->msg.connection_close = TRUE; + break; // not interested in any other options + } + } + return 0; + } + if (strcasecmp(name, "Content-Length") == 0) { + /* Content-Length = 1*DIGIT */ + if (str_to_uoff(hdr->value, &parser->msg.content_length) < 0) { + *error_r = "Invalid Content-Length header"; + return -1; + } + return 0; + } + break; + case 'D': case 'd': + if (strcasecmp(name, "Date") == 0) { + /* Date = HTTP-date */ + (void)http_date_parse(data, size, &parser->msg.date); + return 0; + } + break; + case 'L': case 'l': + if (strcasecmp(name, "Location") == 0) { + /* Location = URI-reference (not parsed here) */ + parser->msg.location = hdr->value; + return 0; + } + break; + case 'T': case 't': + if (strcasecmp(name, "Transfer-Encoding") == 0) { + /* Transfer-Encoding = 1#transfer-coding */ + parser->msg.transfer_encoding = hdr->value; + return 0; + } + break; + default: + break; + } + return 0; +} + +int http_message_parse_headers(struct http_message_parser *parser, + const char **error_r) +{ + const char *field_name, *error; + const unsigned char *field_data; + size_t field_size; + int ret; + + /* *( header-field CRLF ) CRLF */ + while ((ret=http_header_parse_next_field + (parser->header_parser, &field_name, &field_data, &field_size, &error)) > 0) { + if (field_name == NULL) { + /* EOH */ + return 1; + } + if (http_message_parse_header(parser, field_name, field_data, + field_size, error_r) < 0) + return -1; + } + + if (ret < 0) { + *error_r = t_strdup_printf( + "Failed to parse response header: %s", error); + } + return ret; +} + +int http_message_parse_body(struct http_message_parser *parser, + const char **error_r) +{ + struct http_parser hparser; + + if (parser->msg.content_length > 0) { + /* Got explicit message size from Content-Length: header */ + parser->payload = + i_stream_create_limit(parser->input, + parser->msg.content_length); + } else if (parser->msg.transfer_encoding != NULL) { + const char *tenc; + + /* Transfer-Encoding = 1#transfer-coding + transfer-coding = "chunked" / "compress" / "deflate" / "gzip" + / transfer-extension ; [FIXME] + transfer-extension = token *( OWS ";" OWS transfer-parameter ) + */ + http_parser_init(&hparser, + (const unsigned char *)parser->msg.transfer_encoding, + strlen(parser->msg.transfer_encoding)); + for (;;) { + if (http_parse_token_list_next(&hparser, &tenc) <= 0) + break; + if (strcasecmp(tenc, "chunked") == 0) { + parser->payload = + http_transfer_chunked_istream_create(parser->input); + break; // FIXME + } else { + *error_r = t_strdup_printf( + "Unknown Transfer-Encoding `%s'", tenc); + return -1; + } + } + } + return 0; +} diff -r c5952dd642c6 -r c55bd41133a2 src/lib-http/http-message-parser.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/lib-http/http-message-parser.h Thu Jul 11 14:18:42 2013 +0300 @@ -0,0 +1,45 @@ +#ifndef HTTP_MESSAGE_PARSER_H +#define HTTP_MESSAGE_PARSER_H + +#include "http-response.h" + +struct http_message { + unsigned int version_major; + unsigned int version_minor; + + ARRAY_TYPE(http_response_header) headers; + time_t date; + + uoff_t content_length; + const char *location; + const char *transfer_encoding; + + unsigned int connection_close:1; +}; + +struct http_message_parser { + struct istream *input; + + const unsigned char *cur, *end; + + struct http_header_parser *header_parser; + struct istream *payload; + + pool_t msg_pool; + struct http_message msg; +}; + +void http_message_parser_init(struct http_message_parser *parser, + struct istream *input); +void http_message_parser_deinit(struct http_message_parser *parser); +void http_message_parser_restart(struct http_message_parser *parser); + +int http_message_parse_finish_payload(struct http_message_parser *parser, + const char **error_r); +int http_message_parse_version(struct http_message_parser *parser); +int http_message_parse_headers(struct http_message_parser *parser, + const char **error_r); +int http_message_parse_body(struct http_message_parser *parser, From dovecot at dovecot.org Thu Jul 11 14:29:23 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Thu, 11 Jul 2013 14:29:23 +0300 Subject: dovecot-2.2: lib-http: Added a very dummy test-http-server Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/6d986d132fb8 changeset: 16605:6d986d132fb8 user: Timo Sirainen date: Thu Jul 11 14:29:05 2013 +0300 description: lib-http: Added a very dummy test-http-server diffstat: src/lib-http/Makefile.am | 16 ++++- src/lib-http/test-http-server.c | 131 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 146 insertions(+), 1 deletions(-) diffs (168 lines): diff -r c55bd41133a2 -r 6d986d132fb8 src/lib-http/Makefile.am --- a/src/lib-http/Makefile.am Thu Jul 11 14:18:42 2013 +0300 +++ b/src/lib-http/Makefile.am Thu Jul 11 14:29:05 2013 +0300 @@ -45,7 +45,8 @@ test-http-response-parser test_nocheck_programs = \ - test-http-client + test-http-client \ + test-http-server noinst_PROGRAMS = $(test_programs) $(test_nocheck_programs) @@ -103,6 +104,19 @@ ../lib-ssl-iostream/libssl_iostream.la \ $(test_deps) +test_http_server_SOURCES = test-http-server.c +test_http_server_LDFLAGS = -export-dynamic +test_http_server_LDADD = \ + libhttp.la \ + ../lib-dns/libdns.la \ + ../lib-ssl-iostream/libssl_iostream.la \ + $(test_libs) +test_http_server_DEPENDENCIES = \ + libhttp.la \ + ../lib-dns/libdns.la \ + ../lib-ssl-iostream/libssl_iostream.la \ + $(test_deps) + check: check-am check-test check-test: all-am for bin in $(test_programs); do \ diff -r c55bd41133a2 -r 6d986d132fb8 src/lib-http/test-http-server.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/lib-http/test-http-server.c Thu Jul 11 14:29:05 2013 +0300 @@ -0,0 +1,131 @@ +/* Copyright (c) 2013 Dovecot authors, see the included COPYING file */ + +#include "lib.h" +#include "str.h" +#include "ioloop.h" +#include "ostream.h" +#include "connection.h" +#include "http-date.h" +#include "http-request-parser.h" + +static struct connection_list *clients; +static int fd_listen; +struct io *io_listen; + +struct client { + struct connection conn; + struct http_request_parser *parser; +}; + +static void client_destroy(struct connection *conn) +{ + struct client *client = (struct client *)conn; + + http_request_parser_deinit(&client->parser); + connection_deinit(&client->conn); + i_free(client); +} + +static int +client_handle_request(struct client *client, struct http_request *request) +{ + string_t *str = t_str_new(128); + + if (strcmp(request->method, "GET") != 0) { + o_stream_send_str(client->conn.output, "HTTP/1.1 501 Not Implemented\r\nAllow: GET\r\n\r\n"); + return 0; + } + str_append(str, "HTTP/1.1 200 OK\r\n"); + str_printfa(str, "Date: %s\r\n", http_date_create(ioloop_time)); + str_printfa(str, "Content-Length: %d\r\n", (int)strlen(request->target)); + str_append(str, "Content-Type: text/plain\r\n"); + str_append(str, "\r\n"); + str_append(str, request->target); + o_stream_send(client->conn.output, str_data(str), str_len(str)); + return 0; +} + +static void client_input(struct connection *conn) +{ + struct client *client = (struct client *)conn; + struct http_request *request; + const char *error; + int ret; + + while ((ret = http_request_parse_next(client->parser, &request, &error)) > 0) { + if (client_handle_request(client, request) < 0 || + request->connection_close) { + client_destroy(conn); + return; + } + } + if (ret < 0) { + i_error("Client sent invalid request: %s", error); + client_destroy(conn); + } +} + +static struct connection_settings client_set = { + .input_max_size = (size_t)-1, + .output_max_size = (size_t)-1, + .client = FALSE +}; + +static const struct connection_vfuncs client_vfuncs = { + .destroy = client_destroy, + .input = client_input +}; + +static void client_init(int fd) +{ + struct client *client; + + client = i_new(struct client, 1); + connection_init_server(clients, &client->conn, + "(http client)", fd, fd); + client->parser = http_request_parser_init(client->conn.input); +} + +static void client_accept(void *context ATTR_UNUSED) +{ + int fd; + + fd = net_accept(fd_listen, NULL, NULL); + if (fd == -1) + return; + if (fd == -2) + i_fatal("accept() failed: %m"); + + client_init(fd); +} + +int main(int argc, char *argv[]) +{ + struct ip_addr my_ip; + struct ioloop *ioloop; + unsigned int port; + + lib_init(); + if (argc < 2 || str_to_uint(argv[1], &port) < 0) + i_fatal("Port parameter missing"); + if (argc < 3) + net_get_ip_any4(&my_ip); + else if (net_addr2ip(argv[2], &my_ip) < 0) + i_fatal("Invalid IP parameter"); + + ioloop = io_loop_create(); + clients = connection_list_init(&client_set, &client_vfuncs); + + fd_listen = net_listen(&my_ip, &port, 128); + if (fd_listen == -1) + i_fatal("listen(port=%u) failed: %m", port); + io_listen = io_add(fd_listen, IO_READ, client_accept, NULL); + + io_loop_run(ioloop); + + io_remove(&io_listen); + i_close_fd(&fd_listen); + connection_list_deinit(&clients); + io_loop_destroy(&ioloop); + lib_deinit(); +} From dovecot at dovecot.org Fri Jul 12 01:36:49 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 12 Jul 2013 01:36:49 +0300 Subject: dovecot-2.2: lib-ssl-iostream: Crashfix for recent change. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/50b1d59912eb changeset: 16606:50b1d59912eb user: Timo Sirainen date: Fri Jul 12 01:36:41 2013 +0300 description: lib-ssl-iostream: Crashfix for recent change. diffstat: src/lib-ssl-iostream/iostream-openssl-context.c | 14 ++++++++------ 1 files changed, 8 insertions(+), 6 deletions(-) diffs (25 lines): diff -r 6d986d132fb8 -r 50b1d59912eb src/lib-ssl-iostream/iostream-openssl-context.c --- a/src/lib-ssl-iostream/iostream-openssl-context.c Thu Jul 11 14:29:05 2013 +0300 +++ b/src/lib-ssl-iostream/iostream-openssl-context.c Fri Jul 12 01:36:41 2013 +0300 @@ -416,13 +416,15 @@ EC_KEY *eckey; const EC_GROUP *ecgrp; - if (openssl_iostream_load_key(set, &pkey, error_r) < 0) - return -1; + if (set->key != NULL) { + if (openssl_iostream_load_key(set, &pkey, error_r) < 0) + return -1; - if ((eckey = EVP_PKEY_get1_EC_KEY(pkey)) != NULL && - (ecgrp = EC_KEY_get0_group(eckey)) != NULL) - nid = EC_GROUP_get_curve_name(ecgrp); - EVP_PKEY_free(pkey); + if ((eckey = EVP_PKEY_get1_EC_KEY(pkey)) != NULL && + (ecgrp = EC_KEY_get0_group(eckey)) != NULL) + nid = EC_GROUP_get_curve_name(ecgrp); + EVP_PKEY_free(pkey); + } *nid_r = nid; return 0; From dovecot at dovecot.org Fri Jul 12 01:39:55 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 12 Jul 2013 01:39:55 +0300 Subject: dovecot-2.2: man: Updated doveadm-director man page. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a97caa1ce2b7 changeset: 16607:a97caa1ce2b7 user: Timo Sirainen date: Fri Jul 12 01:39:47 2013 +0300 description: man: Updated doveadm-director man page. diffstat: doc/man/doveadm-director.1.in | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diffs (22 lines): diff -r 50b1d59912eb -r a97caa1ce2b7 doc/man/doveadm-director.1.in --- a/doc/man/doveadm-director.1.in Fri Jul 12 01:36:41 2013 +0300 +++ b/doc/man/doveadm-director.1.in Fri Jul 12 01:39:47 2013 +0300 @@ -1,5 +1,5 @@ -.\" Copyright (c) 2010 Dovecot authors, see the included COPYING file -.TH DOVEADM\-DIRECTOR 1 "2011-05-11" "Dovecot v2.2" "Dovecot" +.\" Copyright (c) 2013 Dovecot authors, see the included COPYING file +.TH DOVEADM\-DIRECTOR 1 "2013-07-12" "Dovecot v2.2" "Dovecot" .SH NAME doveadm\-director \- Manage Dovecot directors .\"------------------------------------------------------------------------ @@ -121,7 +121,9 @@ .B doveadm director map is used to list current .IR user "\ \(->\ " host -mappings. +mappings. Note that the director works using 32bit hashes which makes +collisions quite likely, so this command can\(aqt reliably list exactly +which users have recently logged in. .PP .TP .BI \-f\ users_file From dovecot at dovecot.org Fri Jul 12 01:57:50 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 12 Jul 2013 01:57:50 +0300 Subject: dovecot-2.2: auth: Avoid wasting data stack when iterating users. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/7172b893101d changeset: 16608:7172b893101d user: Timo Sirainen date: Fri Jul 12 01:57:45 2013 +0300 description: auth: Avoid wasting data stack when iterating users. diffstat: src/auth/auth-worker-client.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diffs (29 lines): diff -r a97caa1ce2b7 -r 7172b893101d src/auth/auth-worker-client.c --- a/src/auth/auth-worker-client.c Fri Jul 12 01:39:47 2013 +0300 +++ b/src/auth/auth-worker-client.c Fri Jul 12 01:57:45 2013 +0300 @@ -455,8 +455,10 @@ do { ctx->sending = TRUE; ctx->sent = FALSE; - ctx->auth_request->userdb->userdb->iface-> - iterate_next(ctx->iter); + T_BEGIN { + ctx->auth_request->userdb->userdb->iface-> + iterate_next(ctx->iter); + } T_END; } while (ctx->sent && o_stream_get_buffer_used_size(ctx->client->output) == 0); ctx->sending = FALSE; @@ -472,10 +474,10 @@ list_iter_deinit(ctx); return 1; } - if (ret > 0) { + if (ret > 0) T_BEGIN { ctx->auth_request->userdb->userdb->iface-> iterate_next(ctx->iter); - } + } T_END; return 1; } From dovecot at dovecot.org Fri Jul 12 02:17:44 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 12 Jul 2013 02:17:44 +0300 Subject: dovecot-2.2: lib-storage: mail_storage_service_all_init() no lon... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/c0b0f46ff581 changeset: 16609:c0b0f46ff581 user: Timo Sirainen date: Fri Jul 12 02:17:31 2013 +0300 description: lib-storage: mail_storage_service_all_init() no longer returns total number of users. In preperation for removing the count functionality, since it requires reading all the users into memory for it to work. This also required removing the /total from verbosity counter with doveadm -A. diffstat: src/doveadm/doveadm-mail.c | 11 ++++------- src/lib-storage/mail-storage-service.c | 4 +--- src/lib-storage/mail-storage-service.h | 5 ++--- 3 files changed, 7 insertions(+), 13 deletions(-) diffs (70 lines): diff -r 7172b893101d -r c0b0f46ff581 src/doveadm/doveadm-mail.c --- a/src/doveadm/doveadm-mail.c Fri Jul 12 01:57:45 2013 +0300 +++ b/src/doveadm/doveadm-mail.c Fri Jul 12 02:17:31 2013 +0300 @@ -342,7 +342,7 @@ const char *wildcard_user) { struct mail_storage_service_input input; - unsigned int user_idx, user_count, interval, n; + unsigned int user_idx; const char *user, *error; int ret; @@ -359,10 +359,7 @@ ctx->v.init(ctx, (const void *)argv); - user_count = mail_storage_service_all_init(ctx->storage_service); - n = user_count / 10000; - for (interval = 10; n > 0 && interval < 1000; interval *= 10) - n /= 10; + mail_storage_service_all_init(ctx->storage_service); if (hook_doveadm_mail_init != NULL) hook_doveadm_mail_init(ctx); @@ -386,8 +383,8 @@ if (ret == -1) break; if (doveadm_verbose) { - if (++user_idx % interval == 0) { - printf("\r%d / %d", user_idx, user_count); + if (++user_idx % 100 == 0) { + printf("\r%d", user_idx); fflush(stdout); } } diff -r 7172b893101d -r c0b0f46ff581 src/lib-storage/mail-storage-service.c --- a/src/lib-storage/mail-storage-service.c Fri Jul 12 01:57:45 2013 +0300 +++ b/src/lib-storage/mail-storage-service.c Fri Jul 12 02:17:31 2013 +0300 @@ -1243,15 +1243,13 @@ pool_unref(&temp_pool); } -unsigned int -mail_storage_service_all_init(struct mail_storage_service_ctx *ctx) +void mail_storage_service_all_init(struct mail_storage_service_ctx *ctx) { if (ctx->auth_list != NULL) (void)auth_master_user_list_deinit(&ctx->auth_list); mail_storage_service_init_settings(ctx, NULL); ctx->auth_list = auth_master_user_list_init(ctx->conn, "", NULL); - return auth_master_user_list_count(ctx->auth_list); } int mail_storage_service_all_next(struct mail_storage_service_ctx *ctx, diff -r 7172b893101d -r c0b0f46ff581 src/lib-storage/mail-storage-service.h --- a/src/lib-storage/mail-storage-service.h Fri Jul 12 01:57:45 2013 +0300 +++ b/src/lib-storage/mail-storage-service.h Fri Jul 12 02:17:31 2013 +0300 @@ -98,9 +98,8 @@ struct mail_user **mail_user_r, const char **error_r); void mail_storage_service_user_free(struct mail_storage_service_user **user); -/* Initialize iterating through all users. Return the number of users. */ -unsigned int -mail_storage_service_all_init(struct mail_storage_service_ctx *ctx); +/* Initialize iterating through all users. */ +void mail_storage_service_all_init(struct mail_storage_service_ctx *ctx); /* Iterate through all usernames. Returns 1 if username was returned, 0 if there are no more users, -1 if error. */ int mail_storage_service_all_next(struct mail_storage_service_ctx *ctx, From dovecot at dovecot.org Fri Jul 12 02:26:55 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 12 Jul 2013 02:26:55 +0300 Subject: dovecot-2.2: lib-auth: auth_master_user_list_*() no longer reads... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/328bc770af54 changeset: 16610:328bc770af54 user: Timo Sirainen date: Fri Jul 12 02:26:44 2013 +0300 description: lib-auth: auth_master_user_list_*() no longer reads the entire user list into memory. diffstat: src/lib-auth/auth-master.c | 100 +++++++++++++++++++++++++------------------- src/lib-auth/auth-master.h | 1 - 2 files changed, 57 insertions(+), 44 deletions(-) diffs (229 lines): diff -r c0b0f46ff581 -r 328bc770af54 src/lib-auth/auth-master.c --- a/src/lib-auth/auth-master.c Fri Jul 12 02:17:31 2013 +0300 +++ b/src/lib-auth/auth-master.c Fri Jul 12 02:26:44 2013 +0300 @@ -31,7 +31,7 @@ enum auth_master_flags flags; int fd; - struct ioloop *ioloop; + struct ioloop *ioloop, *prev_ioloop; struct io *io; struct istream *input; struct ostream *output; @@ -61,10 +61,8 @@ struct auth_master_user_list_ctx { struct auth_master_connection *conn; - pool_t pool; - ARRAY_TYPE(const_string) users; - const char *const *user_strings; - unsigned int idx, user_count; + string_t *username; + bool finished; bool failed; }; @@ -338,6 +336,7 @@ if (conn->to != NULL) timeout_remove(&conn->to); + conn->prev_ioloop = current_ioloop; conn->ioloop = io_loop_create(); conn->input = i_stream_create_fd(conn->fd, MAX_INBUF_SIZE, FALSE); conn->output = o_stream_create_fd(conn->fd, MAX_OUTBUF_SIZE, FALSE); @@ -347,11 +346,10 @@ lib_signals_reset_ioloop(); } -static void auth_master_unset_io(struct auth_master_connection *conn, - struct ioloop *prev_ioloop) +static void auth_master_unset_io(struct auth_master_connection *conn) { - if (prev_ioloop != NULL) { - io_loop_set_current(prev_ioloop); + if (conn->prev_ioloop != NULL) { + io_loop_set_current(conn->prev_ioloop); lib_signals_reset_ioloop(); } io_loop_set_current(conn->ioloop); @@ -363,7 +361,7 @@ io_loop_destroy(&conn->ioloop); if ((conn->flags & AUTH_MASTER_FLAG_NO_IDLE_TIMEOUT) == 0) { - if (prev_ioloop == NULL) + if (conn->prev_ioloop == NULL) auth_connection_close(conn); else { conn->to = timeout_add(1000*AUTH_MASTER_IDLE_SECS, @@ -385,18 +383,15 @@ return TRUE; } -static int auth_master_run_cmd(struct auth_master_connection *conn, - const char *cmd) +static int auth_master_run_cmd_pre(struct auth_master_connection *conn, + const char *cmd) { - struct ioloop *prev_ioloop; const char *str; if (conn->fd == -1) { if (auth_master_connect(conn) < 0) return -1; } - - prev_ioloop = current_ioloop; auth_master_set_io(conn); o_stream_cork(conn->output); @@ -412,12 +407,16 @@ if (o_stream_nfinish(conn->output) < 0) { i_error("write(auth socket) failed: %m"); - conn->aborted = TRUE; - } else { - io_loop_run(conn->ioloop); + auth_master_unset_io(conn); + auth_connection_close(conn); + return -1; } + return 0; +} - auth_master_unset_io(conn, prev_ioloop); +static int auth_master_run_cmd_post(struct auth_master_connection *conn) +{ + auth_master_unset_io(conn); if (conn->aborted) { conn->aborted = FALSE; auth_connection_close(conn); @@ -426,6 +425,15 @@ return 0; } +static int auth_master_run_cmd(struct auth_master_connection *conn, + const char *cmd) +{ + if (auth_master_run_cmd_pre(conn, cmd) < 0) + return -1; + io_loop_run(conn->ioloop); + return auth_master_run_cmd_post(conn); +} + static unsigned int auth_master_next_request_id(struct auth_master_connection *conn) { @@ -617,26 +625,26 @@ void *context) { struct auth_master_user_list_ctx *ctx = context; - const char *user; timeout_reset(ctx->conn->to); + str_truncate(ctx->username, 0); + io_loop_stop(ctx->conn->ioloop); if (strcmp(cmd, "DONE") == 0) { - io_loop_stop(ctx->conn->ioloop); if (args[0] != NULL && strcmp(args[0], "fail") == 0) { i_error("User listing returned failure"); ctx->failed = TRUE; } - return TRUE; - } - if (strcmp(cmd, "LIST") == 0 && args[0] != NULL) { + ctx->finished = TRUE; + } else if (strcmp(cmd, "LIST") == 0 && args[0] != NULL) { /* we'll just read all the users into memory. otherwise we'd have to use a separate connection for listing and there's a higher chance of a failure since the connection could be open to dovecot-auth for a long time. */ - user = p_strdup(ctx->pool, args[0]); - array_append(&ctx->users, &user, 1); - return TRUE; + str_append(ctx->username, args[0]); + } else { + i_error("User listing returned invalid input"); + ctx->failed = TRUE; } return FALSE; } @@ -648,13 +656,10 @@ { struct auth_master_user_list_ctx *ctx; string_t *str; - pool_t pool; - pool = pool_alloconly_create("auth master user list", 10240); - ctx = p_new(pool, struct auth_master_user_list_ctx, 1); - ctx->pool = pool; + ctx = i_new(struct auth_master_user_list_ctx, 1); ctx->conn = conn; - i_array_init(&ctx->users, 128); + ctx->username = str_new(default_pool, 128); conn->reply_callback = auth_user_list_reply_callback; conn->reply_context = ctx; @@ -669,23 +674,31 @@ str_append_c(str, '\n'); conn->prefix = "userdb list"; - if (auth_master_run_cmd(conn, str_c(str)) < 0) + + if (auth_master_run_cmd_pre(conn, str_c(str)) < 0) ctx->failed = TRUE; - ctx->user_strings = array_get(&ctx->users, &ctx->user_count); conn->prefix = DEFAULT_USERDB_LOOKUP_PREFIX; return ctx; } const char *auth_master_user_list_next(struct auth_master_user_list_ctx *ctx) { - if (ctx->idx == ctx->user_count) + const char *line; + + /* try to read already buffered input */ + line = i_stream_next_line(ctx->conn->input); + if (line != NULL) { + T_BEGIN { + auth_handle_line(ctx->conn, line); + } T_END; + } else if (!ctx->failed) { + /* wait for more data */ + io_loop_run(ctx->conn->ioloop); + } + + if (ctx->finished || ctx->failed) return NULL; - return ctx->user_strings[ctx->idx++]; -} - -unsigned int auth_master_user_list_count(struct auth_master_user_list_ctx *ctx) -{ - return ctx->user_count; + return str_c(ctx->username); } int auth_master_user_list_deinit(struct auth_master_user_list_ctx **_ctx) @@ -694,7 +707,8 @@ int ret = ctx->failed ? -1 : 0; *_ctx = NULL; - array_free(&ctx->users); - pool_unref(&ctx->pool); + auth_master_run_cmd_post(ctx->conn); + str_free(&ctx->username); + i_free(ctx); return ret; } diff -r c0b0f46ff581 -r 328bc770af54 src/lib-auth/auth-master.h --- a/src/lib-auth/auth-master.h Fri Jul 12 02:17:31 2013 +0300 +++ b/src/lib-auth/auth-master.h Fri Jul 12 02:26:44 2013 +0300 @@ -59,7 +59,6 @@ const char *user_mask, const struct auth_user_info *info) ATTR_NULL(3); const char *auth_master_user_list_next(struct auth_master_user_list_ctx *ctx); -unsigned int auth_master_user_list_count(struct auth_master_user_list_ctx *ctx); /* Returns -1 if anything failed, 0 if ok */ int auth_master_user_list_deinit(struct auth_master_user_list_ctx **ctx); From dovecot at dovecot.org Fri Jul 12 02:30:21 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 12 Jul 2013 02:30:21 +0300 Subject: dovecot-2.2: auth: Use fewer syscalls for sending user list. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/86275093e7b3 changeset: 16611:86275093e7b3 user: Timo Sirainen date: Fri Jul 12 02:29:00 2013 +0300 description: auth: Use fewer syscalls for sending user list. diffstat: src/auth/auth-master-connection.c | 6 +++++- src/auth/auth-worker-client.c | 10 +++++++++- 2 files changed, 14 insertions(+), 2 deletions(-) diffs (70 lines): diff -r 328bc770af54 -r 86275093e7b3 src/auth/auth-master-connection.c --- a/src/auth/auth-master-connection.c Fri Jul 12 02:26:44 2013 +0300 +++ b/src/auth/auth-master-connection.c Fri Jul 12 02:29:00 2013 +0300 @@ -432,6 +432,7 @@ if (ctx->iter != NULL) (void)userdb_blocking_iter_deinit(&ctx->iter); + o_stream_uncork(ctx->conn->output); o_stream_unset_flush_callback(ctx->conn->output); auth_request_unref(&ctx->auth_request); auth_master_connection_unref(&ctx->conn); @@ -492,12 +493,14 @@ str_tabescape(user)); ret = o_stream_send_str(ctx->conn->output, str); } T_END; + if (o_stream_get_buffer_used_size(ctx->conn->output) >= MAX_OUTBUF_SIZE) + ret = o_stream_flush(ctx->conn->output); if (ret < 0) { /* disconnected, don't bother finishing */ master_input_list_finish(ctx); return; } - if (o_stream_get_buffer_used_size(ctx->conn->output) == 0) + if (o_stream_get_buffer_used_size(ctx->conn->output) < MAX_OUTBUF_SIZE) userdb_blocking_iter_next(ctx->iter); } @@ -569,6 +572,7 @@ ctx->auth_request->userdb = userdb; io_remove(&conn->io); + o_stream_cork(conn->output); o_stream_set_flush_callback(conn->output, master_output_list, ctx); ctx->iter = userdb_blocking_iter_init(auth_request, master_input_list_callback, ctx); diff -r 328bc770af54 -r 86275093e7b3 src/auth/auth-worker-client.c --- a/src/auth/auth-worker-client.c Fri Jul 12 02:26:44 2013 +0300 +++ b/src/auth/auth-worker-client.c Fri Jul 12 02:29:00 2013 +0300 @@ -421,6 +421,7 @@ auth_worker_send_reply(client, str); client->io = io_add(client->fd, IO_READ, auth_worker_input, client); + o_stream_uncork(ctx->client->output); o_stream_set_flush_callback(client->output, auth_worker_output, client); auth_request_unref(&ctx->auth_request); auth_worker_client_unref(&client); @@ -459,8 +460,14 @@ ctx->auth_request->userdb->userdb->iface-> iterate_next(ctx->iter); } T_END; + if (o_stream_get_buffer_used_size(ctx->client->output) > OUTBUF_THROTTLE_SIZE) { + if (o_stream_flush(ctx->client->output) < 0) { + ctx->done = TRUE; + break; + } + } } while (ctx->sent && - o_stream_get_buffer_used_size(ctx->client->output) == 0); + o_stream_get_buffer_used_size(ctx->client->output) <= OUTBUF_THROTTLE_SIZE); ctx->sending = FALSE; if (ctx->done) list_iter_deinit(ctx); @@ -513,6 +520,7 @@ } io_remove(&ctx->client->io); + o_stream_cork(ctx->client->output); o_stream_set_flush_callback(ctx->client->output, auth_worker_list_output, ctx); ctx->iter = ctx->auth_request->userdb->userdb->iface-> From dovecot at dovecot.org Fri Jul 12 02:43:58 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 12 Jul 2013 02:43:58 +0300 Subject: dovecot-2.2: auth: If blocking passdb returned only NULLs for us... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2be295a0b64f changeset: 16612:2be295a0b64f user: Timo Sirainen date: Fri Jul 12 02:43:43 2013 +0300 description: auth: If blocking passdb returned only NULLs for userdb_ fields, use userdb prefetch anyway. diffstat: src/auth/auth-request.c | 5 ++--- src/auth/auth-request.h | 2 ++ src/auth/auth-worker-client.c | 26 +++++++++++++++++--------- 3 files changed, 21 insertions(+), 12 deletions(-) diffs (91 lines): diff -r 86275093e7b3 -r 2be295a0b64f src/auth/auth-request.c --- a/src/auth/auth-request.c Fri Jul 12 02:29:00 2013 +0300 +++ b/src/auth/auth-request.c Fri Jul 12 02:43:43 2013 +0300 @@ -32,7 +32,6 @@ #define AUTH_DNS_DEFAULT_TIMEOUT_MSECS (1000*10) #define AUTH_DNS_WARN_MSECS 500 #define CACHED_PASSWORD_SCHEME "SHA1" -#define AUTH_REQUEST_KEY_IGNORE " " struct auth_request_proxy_dns_lookup_ctx { struct auth_request *request; @@ -928,7 +927,7 @@ if (str_len(str) == 0) { /* no userdb fields. but we can't save an empty string, since that means "user unknown". */ - str_append(str, AUTH_REQUEST_KEY_IGNORE); + str_append(str, AUTH_REQUEST_USER_KEY_IGNORE); } cache_value = str_c(str); } @@ -1520,7 +1519,7 @@ warned = TRUE; } name = "system_groups_user"; - } else if (strcmp(name, AUTH_REQUEST_KEY_IGNORE) == 0) { + } else if (strcmp(name, AUTH_REQUEST_USER_KEY_IGNORE) == 0) { return; } diff -r 86275093e7b3 -r 2be295a0b64f src/auth/auth-request.h --- a/src/auth/auth-request.h Fri Jul 12 02:29:00 2013 +0300 +++ b/src/auth/auth-request.h Fri Jul 12 02:43:43 2013 +0300 @@ -7,6 +7,8 @@ #include "userdb.h" #include "passdb.h" +#define AUTH_REQUEST_USER_KEY_IGNORE " " + struct auth_client_connection; enum auth_request_state { diff -r 86275093e7b3 -r 2be295a0b64f src/auth/auth-worker-client.c --- a/src/auth/auth-worker-client.c Fri Jul 12 02:29:00 2013 +0300 +++ b/src/auth/auth-worker-client.c Fri Jul 12 02:43:43 2013 +0300 @@ -109,6 +109,21 @@ o_stream_nsend(client->output, str_data(str), str_len(str)); } +static void +reply_append_extra_fields(string_t *str, struct auth_request *request) +{ + if (!auth_fields_is_empty(request->extra_fields)) { + str_append_c(str, '\t'); + auth_fields_append(request->extra_fields, str, 0, 0); + } + if (request->userdb_reply != NULL && + auth_fields_is_empty(request->userdb_reply)) { + /* all userdb_* fields had NULL values. we'll still + need to tell this to the master */ + str_append(str, "\tuserdb_"AUTH_REQUEST_USER_KEY_IGNORE); + } +} + static void verify_plain_callback(enum passdb_result result, struct auth_request *request) { @@ -131,10 +146,7 @@ str_append_c(str, '\t'); if (request->passdb_password != NULL) str_append_tabescaped(str, request->passdb_password); - if (!auth_fields_is_empty(request->extra_fields)) { - str_append_c(str, '\t'); - auth_fields_append(request->extra_fields, str, 0, 0); - } + reply_append_extra_fields(str, request); } str_append_c(str, '\n'); auth_worker_send_reply(client, str); @@ -215,11 +227,7 @@ str_append_tabescaped(str, request->user); str_printfa(str, "\t{%s.b64}", request->credentials_scheme); base64_encode(credentials, size, str); - - if (!auth_fields_is_empty(request->extra_fields)) { - str_append_c(str, '\t'); - auth_fields_append(request->extra_fields, str, 0, 0); - } + reply_append_extra_fields(str, request); } str_append_c(str, '\n'); auth_worker_send_reply(client, str); From dovecot at dovecot.org Fri Jul 12 03:01:44 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 12 Jul 2013 03:01:44 +0300 Subject: dovecot-2.2: auth: With multiple passdbs the previous passdb's u... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/3e6b136d936c changeset: 16613:3e6b136d936c user: Timo Sirainen date: Fri Jul 12 03:01:31 2013 +0300 description: auth: With multiple passdbs the previous passdb's userdb_* fields weren't cleared. diffstat: src/auth/auth-request.c | 10 ++++++++++ src/auth/auth-request.h | 1 + 2 files changed, 11 insertions(+), 0 deletions(-) diffs (35 lines): diff -r 2be295a0b64f -r 3e6b136d936c src/auth/auth-request.c --- a/src/auth/auth-request.c Fri Jul 12 02:43:43 2013 +0300 +++ b/src/auth/auth-request.c Fri Jul 12 03:01:31 2013 +0300 @@ -605,10 +605,20 @@ /* this passdb lookup succeeded, preserve its extra fields */ auth_fields_snapshot(request->extra_fields); + request->snapshot_has_userdb_reply = + request->userdb_reply != NULL; + if (request->userdb_reply != NULL) + auth_fields_snapshot(request->userdb_reply); } else { /* this passdb lookup failed, remove any extra fields it set */ auth_fields_rollback(request->extra_fields); + if (request->userdb_reply == NULL) + ; + else if (!request->snapshot_has_userdb_reply) + request->userdb_reply = NULL; + else + auth_fields_rollback(request->userdb_reply); } if (*result == PASSDB_RESULT_USER_UNKNOWN) { diff -r 2be295a0b64f -r 3e6b136d936c src/auth/auth-request.h --- a/src/auth/auth-request.h Fri Jul 12 02:43:43 2013 +0300 +++ b/src/auth/auth-request.h Fri Jul 12 03:01:31 2013 +0300 @@ -123,6 +123,7 @@ unsigned int prefer_plain_credentials:1; unsigned int in_delayed_failure_queue:1; unsigned int removed_from_handler:1; + unsigned int snapshot_has_userdb_reply:1; /* each passdb lookup can update the current success-status using the result_* rules. the authentication succeeds only if this is TRUE at the end. mechanisms that don't require passdb, but do a passdb From dovecot at dovecot.org Fri Jul 12 03:06:33 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Fri, 12 Jul 2013 03:06:33 +0300 Subject: dovecot-2.2: director: Don't log an error if login connection ge... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/ebbe00999da8 changeset: 16614:ebbe00999da8 user: Timo Sirainen date: Fri Jul 12 03:06:26 2013 +0300 description: director: Don't log an error if login connection gets closed with ECONNRESET. diffstat: src/director/login-connection.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 3e6b136d936c -r ebbe00999da8 src/director/login-connection.c --- a/src/director/login-connection.c Fri Jul 12 03:01:31 2013 +0300 +++ b/src/director/login-connection.c Fri Jul 12 03:06:26 2013 +0300 @@ -47,7 +47,8 @@ if (ret < 0) { if (errno == EAGAIN) return; - i_error("read(login connection) failed: %m"); + if (errno != ECONNRESET) + i_error("read(login connection) failed: %m"); } login_connection_deinit(&conn); return; From dovecot at dovecot.org Wed Jul 17 10:23:36 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Wed, 17 Jul 2013 10:23:36 +0300 Subject: dovecot-2.2: liblib: connection_disconnect() didn't close fd_out... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/50117c22151e changeset: 16615:50117c22151e user: Timo Sirainen date: Wed Jul 17 10:23:21 2013 +0300 description: liblib: connection_disconnect() didn't close fd_out if it was different from fd_in. diffstat: src/lib/connection.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r ebbe00999da8 -r 50117c22151e src/lib/connection.c --- a/src/lib/connection.c Fri Jul 12 03:06:26 2013 +0300 +++ b/src/lib/connection.c Wed Jul 17 10:23:21 2013 +0300 @@ -254,7 +254,7 @@ if (conn->fd_in != -1) { if (close(conn->fd_in) < 0) i_error("close(%s) failed: %m", conn->name); - if (conn->fd_in != conn->fd_out) + if (conn->fd_in != conn->fd_out && close(conn->fd_out) < 0) i_error("close(%s/out) failed: %m", conn->name); conn->fd_in = conn->fd_out = -1; } From dovecot at dovecot.org Mon Jul 29 22:19:13 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:19:13 +0300 Subject: dovecot-2.1: director: Don't allow director_user_expire to be le... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/fcdbadcb9801 changeset: 14982:fcdbadcb9801 user: Timo Sirainen date: Mon Jul 29 22:06:13 2013 +0300 description: director: Don't allow director_user_expire to be less than 10 seconds. diffstat: src/director/director-settings.c | 20 +++++++++++++++++++- 1 files changed, 19 insertions(+), 1 deletions(-) diffs (35 lines): diff -r edc20123f658 -r fcdbadcb9801 src/director/director-settings.c --- a/src/director/director-settings.c Wed Jun 26 22:04:48 2013 +0300 +++ b/src/director/director-settings.c Mon Jul 29 22:06:13 2013 +0300 @@ -7,6 +7,8 @@ #include "director-settings.h" /* */ +static bool director_settings_verify(void *_set, pool_t pool, const char **error_r); + static struct file_listener_settings director_unix_listeners_array[] = { { "login/director", 0, "", "" }, { "director-admin", 0600, "", "" } @@ -93,5 +95,21 @@ .type_offset = (size_t)-1, .struct_size = sizeof(struct director_settings), - .parent_offset = (size_t)-1 + .parent_offset = (size_t)-1, + + .check_func = director_settings_verify }; + +/* */ +static bool +director_settings_verify(void *_set, pool_t pool ATTR_UNUSED, const char **error_r) +{ + struct director_settings *set = _set; + + if (set->director_user_expire < 10) { + *error_r = "director_user_expire is too low"; + return FALSE; + } + return TRUE; +} +/* */ From dovecot at dovecot.org Mon Jul 29 22:19:13 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:19:13 +0300 Subject: dovecot-2.1: director: Expire users a bit more correctly. Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/df041f75b6da changeset: 14983:df041f75b6da user: Timo Sirainen date: Mon Jul 29 22:08:26 2013 +0300 description: director: Expire users a bit more correctly. Also make sure that the "user near expiring" interval is at least 3 seconds in case director_user_expire is very low. diffstat: src/director/user-directory.c | 20 +++++++++++++++----- 1 files changed, 15 insertions(+), 5 deletions(-) diffs (59 lines): diff -r fcdbadcb9801 -r df041f75b6da src/director/user-directory.c --- a/src/director/user-directory.c Mon Jul 29 22:06:13 2013 +0300 +++ b/src/director/user-directory.c Mon Jul 29 22:08:26 2013 +0300 @@ -11,7 +11,8 @@ /* n% of timeout_secs */ #define USER_NEAR_EXPIRING_PERCENTAGE 10 -/* but max. of this many secs */ +/* but min/max. of this many secs */ +#define USER_NEAR_EXPIRING_MIN 3 #define USER_NEAR_EXPIRING_MAX 30 struct user_directory_iter { @@ -65,7 +66,7 @@ { time_t expire_timestamp = user->timestamp + dir->timeout_secs; - if (expire_timestamp >= ioloop_time) + if (expire_timestamp > ioloop_time) return TRUE; if (user->kill_state != USER_KILL_STATE_NONE) { @@ -93,9 +94,15 @@ struct user *user_directory_lookup(struct user_directory *dir, unsigned int username_hash) { + struct user *user; + user_directory_drop_expired(dir); - - return hash_table_lookup(dir->hash, POINTER_CAST(username_hash)); + user = hash_table_lookup(dir->hash, POINTER_CAST(username_hash)); + if (user != NULL && !user_directory_user_has_connections(dir, user)) { + user_free(dir, user); + user = NULL; + } + return user; } static void @@ -230,6 +237,8 @@ { struct user_directory *dir; + i_assert(timeout_secs > USER_NEAR_EXPIRING_MIN); + dir = i_new(struct user_directory, 1); dir->timeout_secs = timeout_secs; dir->user_near_expiring_secs = @@ -237,7 +246,8 @@ dir->user_near_expiring_secs = I_MIN(dir->user_near_expiring_secs, USER_NEAR_EXPIRING_MAX); dir->user_near_expiring_secs = - I_MAX(dir->user_near_expiring_secs, 1); + I_MAX(dir->user_near_expiring_secs, USER_NEAR_EXPIRING_MIN); + i_assert(dir->timeout_secs/2 > dir->user_near_expiring_secs); dir->username_hash_fmt = i_strdup(username_hash_fmt); dir->hash = hash_table_create(default_pool, default_pool, From dovecot at dovecot.org Mon Jul 29 22:19:13 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:19:13 +0300 Subject: dovecot-2.1: director: Fixes to handling users near expiration. Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/966a9ee85d58 changeset: 14985:966a9ee85d58 user: Timo Sirainen date: Mon Jul 29 22:10:49 2013 +0300 description: director: Fixes to handling users near expiration. diffstat: src/director/director-connection.c | 46 +++++++++++++++++++++++++++++++------ 1 files changed, 38 insertions(+), 8 deletions(-) diffs (85 lines): diff -r a34c9e40d473 -r 966a9ee85d58 src/director/director-connection.c --- a/src/director/director-connection.c Mon Jul 29 22:10:01 2013 +0300 +++ b/src/director/director-connection.c Mon Jul 29 22:10:49 2013 +0300 @@ -476,7 +476,24 @@ dir_debug("user refresh: %u set weak", username_hash); user->weak = TRUE; ret = TRUE; - } else if (user->host != host) { + } else if (weak) { + dir_debug("user refresh: %u weak update to %s ignored, " + "we recently changed it to %s", + username_hash, net_ip2addr(&host->ip), + net_ip2addr(&user->host->ip)); + host = user->host; + ret = TRUE; + } else if (user->host == host) { + /* update to the same host */ + } else if (user_directory_user_is_near_expiring(dir->users, user)) { + /* host conflict for a user that is already near expiring. we can + assume that the other director had already dropped this user + and we should have as well. use the new host. */ + dir_debug("user refresh: %u is nearly expired, " + "replacing host %s with %s", username_hash, + net_ip2addr(&user->host->ip), net_ip2addr(&host->ip)); + ret = TRUE; + } else { /* non-weak user received a non-weak update with conflicting host. this shouldn't happen. */ string_t *str = t_str_new(128); @@ -709,11 +726,11 @@ ourself. */ *host_r = NULL; } else { + *host_r = host; if (seq <= host->last_seq) { /* already seen this */ return 1; } - *host_r = host; host->last_seq = seq; } return 0; @@ -729,7 +746,7 @@ struct mail_host *host; struct user *user; struct director_host *src_host = conn->host; - bool weak = TRUE; + bool weak = TRUE, weak_forward = FALSE; int ret; /* note that unlike other commands we don't want to just ignore @@ -750,15 +767,28 @@ return TRUE; } - if (ret > 0) { - /* The entire ring has seen this USER-WEAK. - make it non-weak now. */ + if (ret == 0) + ; + else if (dir_host == conn->dir->self_host) { + /* We originated this USER-WEAK request. The entire ring has seen + it and there weren't any conflicts. Make the user non-weak. */ + dir_debug("user refresh: %u Our USER-WEAK seen by the entire ring", + username_hash); + src_host = conn->dir->self_host; weak = FALSE; - src_host = conn->dir->self_host; + } else { + /* The original USER-WEAK sender will send a new non-weak USER + update saying what really happened. We'll still need to forward + this around the ring to the origin so it also knows it has + travelled through the ring. */ + dir_debug("user refresh: %u Remote USER-WEAK from %s seen by the entire ring, ignoring", + username_hash, net_ip2addr(&dir_host->ip)); + weak_forward = TRUE; } if (director_user_refresh(conn, username_hash, - host, ioloop_time, weak, &user)) { + host, ioloop_time, weak, &user) || + weak_forward) { if (!user->weak) director_update_user(conn->dir, src_host, user); else { From dovecot at dovecot.org Mon Jul 29 22:19:13 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:19:13 +0300 Subject: dovecot-2.1: director: Log director disconnection errno correctly. Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/a34c9e40d473 changeset: 14984:a34c9e40d473 user: Timo Sirainen date: Mon Jul 29 22:10:01 2013 +0300 description: director: Log director disconnection errno correctly. diffstat: src/director/director-connection.c | 13 ++++++++----- 1 files changed, 8 insertions(+), 5 deletions(-) diffs (52 lines): diff -r df041f75b6da -r a34c9e40d473 src/director/director-connection.c --- a/src/director/director-connection.c Mon Jul 29 22:08:26 2013 +0300 +++ b/src/director/director-connection.c Mon Jul 29 22:10:01 2013 +0300 @@ -124,7 +124,8 @@ static void director_connection_disconnected(struct director_connection **conn); static void director_connection_reconnect(struct director_connection **conn, const char *reason); -static void director_connection_log_disconnect(struct director_connection *conn); +static void +director_connection_log_disconnect(struct director_connection *conn, int err); static void ATTR_FORMAT(2, 3) director_cmd_error(struct director_connection *conn, const char *fmt, ...) @@ -175,7 +176,7 @@ static void director_connection_wait_timeout(struct director_connection *conn) { - director_connection_log_disconnect(conn); + director_connection_log_disconnect(conn, ETIMEDOUT); director_connection_deinit(&conn, "Timeout waiting for disconnect after CONNECT"); } @@ -1271,7 +1272,7 @@ } static void -director_connection_log_disconnect(struct director_connection *conn) +director_connection_log_disconnect(struct director_connection *conn, int err) { unsigned int secs = ioloop_time - conn->created; string_t *str = t_str_new(128); @@ -1287,8 +1288,10 @@ str_printfa(str, "Director %s disconnected: ", conn->name); str_append(str, "Connection closed"); - if (errno != 0 && errno != EPIPE) + if (err != 0 && err != EPIPE) { + errno = err; str_printfa(str, ": %m"); + } str_printfa(str, " (connected %u secs, " "in=%"PRIuUOFF_T" out=%"PRIuUOFF_T, @@ -1312,7 +1315,7 @@ return; case -1: /* disconnected */ - director_connection_log_disconnect(conn); + director_connection_log_disconnect(conn, conn->input->stream_errno); director_connection_disconnected(&conn); return; case -2: From dovecot at dovecot.org Mon Jul 29 22:19:13 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:19:13 +0300 Subject: dovecot-2.1: director: Fixes to director-test. Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/15537f882a0b changeset: 14986:15537f882a0b user: Timo Sirainen date: Mon Jul 29 22:18:34 2013 +0300 description: director: Fixes to director-test. diffstat: src/director/director-test.c | 18 +++++++++++------- src/director/director-test.sh | 3 ++- 2 files changed, 13 insertions(+), 8 deletions(-) diffs (92 lines): diff -r 966a9ee85d58 -r 15537f882a0b src/director/director-test.c --- a/src/director/director-test.c Mon Jul 29 22:10:49 2013 +0300 +++ b/src/director/director-test.c Mon Jul 29 22:18:34 2013 +0300 @@ -33,7 +33,7 @@ #define IMAP_PORT 14300 #define DIRECTOR_IN_PORT 9091 #define DIRECTOR_OUT_PORT 9090 -#define USER_TIMEOUT_MSECS (1000*60) +#define USER_TIMEOUT_MSECS (1000*10) /* FIXME: this should be based on director_user_expire */ #define ADMIN_RANDOM_TIMEOUT_MSECS 500 #define DIRECTOR_CONN_MAX_DELAY_MSECS 100 #define DIRECTOR_DISCONNECT_TIMEOUT_SECS 10 @@ -126,7 +126,7 @@ i_error("User logging into unknown host %s", net_ip2addr(&local_ip)); host = i_new(struct host, 1); - host->refcount++; + host->refcount = 1; host->ip = local_ip; host->vhost_count = 100; hash_table_insert(hosts, &host->ip, host); @@ -330,12 +330,13 @@ } static void -director_connection_create(int in_fd, const struct ip_addr *local_ip) +director_connection_create(int in_fd, const struct ip_addr *local_ip, + const struct ip_addr *remote_ip) { struct director_connection *conn; int out_fd; - out_fd = net_connect_ip(local_ip, DIRECTOR_OUT_PORT, NULL); + out_fd = net_connect_ip(local_ip, DIRECTOR_OUT_PORT, remote_ip); if (out_fd == -1) { (void)close(in_fd); return; @@ -383,16 +384,18 @@ static void client_connected(struct master_service_connection *conn) { - struct ip_addr local_ip; + struct ip_addr local_ip, remote_ip; unsigned int local_port; if (net_getsockname(conn->fd, &local_ip, &local_port) < 0) i_fatal("net_getsockname() failed: %m"); + if (net_getpeername(conn->fd, &remote_ip, NULL) < 0) + i_fatal("net_getsockname() failed: %m"); if (local_port == IMAP_PORT) imap_client_create(conn->fd); else if (local_port == DIRECTOR_IN_PORT) - director_connection_create(conn->fd, &local_ip); + director_connection_create(conn->fd, &local_ip, &remote_ip); else { i_error("Connection to unknown port %u", local_port); return; @@ -496,6 +499,7 @@ struct host *host; host = i_new(struct host, 1); + host->refcount = 1; if (net_addr2ip(args[0], &host->ip) < 0 || str_to_uint(args[1], &host->vhost_count) < 0) i_fatal("host list broken"); @@ -541,7 +545,7 @@ admin_read_hosts(admin); to_disconnect = - timeout_add(1000*(1 + rand()%DIRECTOR_DISCONNECT_TIMEOUT_SECS), + timeout_add(1000*(5 + rand()%DIRECTOR_DISCONNECT_TIMEOUT_SECS), director_connection_disconnect_timeout, NULL); } diff -r 966a9ee85d58 -r 15537f882a0b src/director/director-test.sh --- a/src/director/director-test.sh Mon Jul 29 22:10:49 2013 +0300 +++ b/src/director/director-test.sh Mon Jul 29 22:18:34 2013 +0300 @@ -26,6 +26,7 @@ info_log_path = /var/log/dovecot-access.log director_servers =$dirs director_mail_servers = 127.0.0.1-127.0.0.255 +director_user_expire = 15s disable_plaintext_auth = no ssl = no @@ -90,4 +91,4 @@ echo echo "Start testing:" echo -echo "imaptest host=director user=test%d.%d - select=0 no_tracking" +echo "imaptest host=director user=test%d.%d - select=0 no_tracking clients=1000" From dovecot at dovecot.org Mon Jul 29 22:19:17 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:19:17 +0300 Subject: dovecot-2.2: director: Don't allow director_user_expire to be le... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/d95ca476098c changeset: 16616:d95ca476098c user: Timo Sirainen date: Mon Jul 29 22:06:13 2013 +0300 description: director: Don't allow director_user_expire to be less than 10 seconds. diffstat: src/director/director-settings.c | 20 +++++++++++++++++++- 1 files changed, 19 insertions(+), 1 deletions(-) diffs (35 lines): diff -r 50117c22151e -r d95ca476098c src/director/director-settings.c --- a/src/director/director-settings.c Wed Jul 17 10:23:21 2013 +0300 +++ b/src/director/director-settings.c Mon Jul 29 22:06:13 2013 +0300 @@ -7,6 +7,8 @@ #include "director-settings.h" /* */ +static bool director_settings_verify(void *_set, pool_t pool, const char **error_r); + static struct file_listener_settings director_unix_listeners_array[] = { { "login/director", 0, "", "" }, { "director-admin", 0600, "", "" } @@ -93,5 +95,21 @@ .type_offset = (size_t)-1, .struct_size = sizeof(struct director_settings), - .parent_offset = (size_t)-1 + .parent_offset = (size_t)-1, + + .check_func = director_settings_verify }; + +/* */ +static bool +director_settings_verify(void *_set, pool_t pool ATTR_UNUSED, const char **error_r) +{ + struct director_settings *set = _set; + + if (set->director_user_expire < 10) { + *error_r = "director_user_expire is too low"; + return FALSE; + } + return TRUE; +} +/* */ From dovecot at dovecot.org Mon Jul 29 22:19:17 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:19:17 +0300 Subject: dovecot-2.2: director: Expire users a bit more correctly. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0e0eb964685a changeset: 16617:0e0eb964685a user: Timo Sirainen date: Mon Jul 29 22:08:26 2013 +0300 description: director: Expire users a bit more correctly. Also make sure that the "user near expiring" interval is at least 3 seconds in case director_user_expire is very low. diffstat: src/director/user-directory.c | 20 +++++++++++++++----- 1 files changed, 15 insertions(+), 5 deletions(-) diffs (59 lines): diff -r d95ca476098c -r 0e0eb964685a src/director/user-directory.c --- a/src/director/user-directory.c Mon Jul 29 22:06:13 2013 +0300 +++ b/src/director/user-directory.c Mon Jul 29 22:08:26 2013 +0300 @@ -11,7 +11,8 @@ /* n% of timeout_secs */ #define USER_NEAR_EXPIRING_PERCENTAGE 10 -/* but max. of this many secs */ +/* but min/max. of this many secs */ +#define USER_NEAR_EXPIRING_MIN 3 #define USER_NEAR_EXPIRING_MAX 30 struct user_directory_iter { @@ -65,7 +66,7 @@ { time_t expire_timestamp = user->timestamp + dir->timeout_secs; - if (expire_timestamp >= ioloop_time) + if (expire_timestamp > ioloop_time) return TRUE; if (user->kill_state != USER_KILL_STATE_NONE) { @@ -93,9 +94,15 @@ struct user *user_directory_lookup(struct user_directory *dir, unsigned int username_hash) { + struct user *user; + user_directory_drop_expired(dir); - - return hash_table_lookup(dir->hash, POINTER_CAST(username_hash)); + user = hash_table_lookup(dir->hash, POINTER_CAST(username_hash)); + if (user != NULL && !user_directory_user_has_connections(dir, user)) { + user_free(dir, user); + user = NULL; + } + return user; } static void @@ -230,6 +237,8 @@ { struct user_directory *dir; + i_assert(timeout_secs > USER_NEAR_EXPIRING_MIN); + dir = i_new(struct user_directory, 1); dir->timeout_secs = timeout_secs; dir->user_near_expiring_secs = @@ -237,7 +246,8 @@ dir->user_near_expiring_secs = I_MIN(dir->user_near_expiring_secs, USER_NEAR_EXPIRING_MAX); dir->user_near_expiring_secs = - I_MAX(dir->user_near_expiring_secs, 1); + I_MAX(dir->user_near_expiring_secs, USER_NEAR_EXPIRING_MIN); + i_assert(dir->timeout_secs/2 > dir->user_near_expiring_secs); dir->username_hash_fmt = i_strdup(username_hash_fmt); hash_table_create_direct(&dir->hash, default_pool, 0); From dovecot at dovecot.org Mon Jul 29 22:19:17 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:19:17 +0300 Subject: dovecot-2.2: director: Log director disconnection errno correctly. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0ad8da149774 changeset: 16618:0ad8da149774 user: Timo Sirainen date: Mon Jul 29 22:10:01 2013 +0300 description: director: Log director disconnection errno correctly. diffstat: src/director/director-connection.c | 13 ++++++++----- 1 files changed, 8 insertions(+), 5 deletions(-) diffs (52 lines): diff -r 0e0eb964685a -r 0ad8da149774 src/director/director-connection.c --- a/src/director/director-connection.c Mon Jul 29 22:08:26 2013 +0300 +++ b/src/director/director-connection.c Mon Jul 29 22:10:01 2013 +0300 @@ -124,7 +124,8 @@ static void director_connection_disconnected(struct director_connection **conn); static void director_connection_reconnect(struct director_connection **conn, const char *reason); -static void director_connection_log_disconnect(struct director_connection *conn); +static void +director_connection_log_disconnect(struct director_connection *conn, int err); static void ATTR_FORMAT(2, 3) director_cmd_error(struct director_connection *conn, const char *fmt, ...) @@ -175,7 +176,7 @@ static void director_connection_wait_timeout(struct director_connection *conn) { - director_connection_log_disconnect(conn); + director_connection_log_disconnect(conn, ETIMEDOUT); director_connection_deinit(&conn, "Timeout waiting for disconnect after CONNECT"); } @@ -1271,7 +1272,7 @@ } static void -director_connection_log_disconnect(struct director_connection *conn) +director_connection_log_disconnect(struct director_connection *conn, int err) { unsigned int secs = ioloop_time - conn->created; string_t *str = t_str_new(128); @@ -1287,8 +1288,10 @@ str_printfa(str, "Director %s disconnected: ", conn->name); str_append(str, "Connection closed"); - if (errno != 0 && errno != EPIPE) + if (err != 0 && err != EPIPE) { + errno = err; str_printfa(str, ": %m"); + } str_printfa(str, " (connected %u secs, " "in=%"PRIuUOFF_T" out=%"PRIuUOFF_T, @@ -1312,7 +1315,7 @@ return; case -1: /* disconnected */ - director_connection_log_disconnect(conn); + director_connection_log_disconnect(conn, conn->input->stream_errno); director_connection_disconnected(&conn); return; case -2: From dovecot at dovecot.org Mon Jul 29 22:19:17 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:19:17 +0300 Subject: dovecot-2.2: director: Fixes to handling users near expiration. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/9b8b5ce6ecb0 changeset: 16619:9b8b5ce6ecb0 user: Timo Sirainen date: Mon Jul 29 22:11:46 2013 +0300 description: director: Fixes to handling users near expiration. diffstat: src/director/director-connection.c | 44 +++++++++++++++++++++++++++++++------ 1 files changed, 37 insertions(+), 7 deletions(-) diffs (72 lines): diff -r 0ad8da149774 -r 9b8b5ce6ecb0 src/director/director-connection.c --- a/src/director/director-connection.c Mon Jul 29 22:10:01 2013 +0300 +++ b/src/director/director-connection.c Mon Jul 29 22:11:46 2013 +0300 @@ -475,7 +475,24 @@ dir_debug("user refresh: %u set weak", username_hash); user->weak = TRUE; ret = TRUE; - } else if (user->host != host) { + } else if (weak) { + dir_debug("user refresh: %u weak update to %s ignored, " + "we recently changed it to %s", + username_hash, net_ip2addr(&host->ip), + net_ip2addr(&user->host->ip)); + host = user->host; + ret = TRUE; + } else if (user->host == host) { + /* update to the same host */ + } else if (user_directory_user_is_near_expiring(dir->users, user)) { + /* host conflict for a user that is already near expiring. we can + assume that the other director had already dropped this user + and we should have as well. use the new host. */ + dir_debug("user refresh: %u is nearly expired, " + "replacing host %s with %s", username_hash, + net_ip2addr(&user->host->ip), net_ip2addr(&host->ip)); + ret = TRUE; + } else { /* non-weak user received a non-weak update with conflicting host. this shouldn't happen. */ string_t *str = t_str_new(128); @@ -729,7 +746,7 @@ struct mail_host *host; struct user *user; struct director_host *src_host = conn->host; - bool weak = TRUE; + bool weak = TRUE, weak_forward = FALSE; int ret; /* note that unlike other commands we don't want to just ignore @@ -750,15 +767,28 @@ return TRUE; } - if (ret > 0) { - /* The entire ring has seen this USER-WEAK. - make it non-weak now. */ + if (ret == 0) + ; + else if (dir_host == conn->dir->self_host) { + /* We originated this USER-WEAK request. The entire ring has seen + it and there weren't any conflicts. Make the user non-weak. */ + dir_debug("user refresh: %u Our USER-WEAK seen by the entire ring", + username_hash); + src_host = conn->dir->self_host; weak = FALSE; - src_host = conn->dir->self_host; + } else { + /* The original USER-WEAK sender will send a new non-weak USER + update saying what really happened. We'll still need to forward + this around the ring to the origin so it also knows it has + travelled through the ring. */ + dir_debug("user refresh: %u Remote USER-WEAK from %s seen by the entire ring, ignoring", + username_hash, net_ip2addr(&dir_host->ip)); + weak_forward = TRUE; } if (director_user_refresh(conn, username_hash, - host, ioloop_time, weak, &user)) { + host, ioloop_time, weak, &user) || + weak_forward) { if (!user->weak) director_update_user(conn->dir, src_host, user); else { From dovecot at dovecot.org Mon Jul 29 22:19:17 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:19:17 +0300 Subject: dovecot-2.2: director: Fixes to director-test. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/366024103c09 changeset: 16620:366024103c09 user: Timo Sirainen date: Mon Jul 29 22:19:03 2013 +0300 description: director: Fixes to director-test. diffstat: src/director/director-test.c | 18 +++++++++++------- src/director/director-test.sh | 3 ++- 2 files changed, 13 insertions(+), 8 deletions(-) diffs (92 lines): diff -r 9b8b5ce6ecb0 -r 366024103c09 src/director/director-test.c --- a/src/director/director-test.c Mon Jul 29 22:11:46 2013 +0300 +++ b/src/director/director-test.c Mon Jul 29 22:19:03 2013 +0300 @@ -33,7 +33,7 @@ #define IMAP_PORT 14300 #define DIRECTOR_IN_PORT 9091 #define DIRECTOR_OUT_PORT 9090 -#define USER_TIMEOUT_MSECS (1000*60) +#define USER_TIMEOUT_MSECS (1000*10) /* FIXME: this should be based on director_user_expire */ #define ADMIN_RANDOM_TIMEOUT_MSECS 500 #define DIRECTOR_CONN_MAX_DELAY_MSECS 100 #define DIRECTOR_DISCONNECT_TIMEOUT_SECS 10 @@ -126,7 +126,7 @@ i_error("User logging into unknown host %s", net_ip2addr(&local_ip)); host = i_new(struct host, 1); - host->refcount++; + host->refcount = 1; host->ip = local_ip; host->vhost_count = 100; hash_table_insert(hosts, &host->ip, host); @@ -331,12 +331,13 @@ } static void -director_connection_create(int in_fd, const struct ip_addr *local_ip) +director_connection_create(int in_fd, const struct ip_addr *local_ip, + const struct ip_addr *remote_ip) { struct director_connection *conn; int out_fd; - out_fd = net_connect_ip(local_ip, DIRECTOR_OUT_PORT, NULL); + out_fd = net_connect_ip(local_ip, DIRECTOR_OUT_PORT, remote_ip); if (out_fd == -1) { i_close_fd(&in_fd); return; @@ -386,16 +387,18 @@ static void client_connected(struct master_service_connection *conn) { - struct ip_addr local_ip; + struct ip_addr local_ip, remote_ip; unsigned int local_port; if (net_getsockname(conn->fd, &local_ip, &local_port) < 0) i_fatal("net_getsockname() failed: %m"); + if (net_getpeername(conn->fd, &remote_ip, NULL) < 0) + i_fatal("net_getsockname() failed: %m"); if (local_port == IMAP_PORT) imap_client_create(conn->fd); else if (local_port == DIRECTOR_IN_PORT) - director_connection_create(conn->fd, &local_ip); + director_connection_create(conn->fd, &local_ip, &remote_ip); else { i_error("Connection to unknown port %u", local_port); return; @@ -499,6 +502,7 @@ struct host *host; host = i_new(struct host, 1); + host->refcount = 1; if (net_addr2ip(args[0], &host->ip) < 0 || str_to_uint(args[1], &host->vhost_count) < 0) i_fatal("host list broken"); @@ -542,7 +546,7 @@ admin_read_hosts(admin); to_disconnect = - timeout_add(1000*(1 + rand()%DIRECTOR_DISCONNECT_TIMEOUT_SECS), + timeout_add(1000*(5 + rand()%DIRECTOR_DISCONNECT_TIMEOUT_SECS), director_connection_disconnect_timeout, (void *)NULL); } diff -r 9b8b5ce6ecb0 -r 366024103c09 src/director/director-test.sh --- a/src/director/director-test.sh Mon Jul 29 22:11:46 2013 +0300 +++ b/src/director/director-test.sh Mon Jul 29 22:19:03 2013 +0300 @@ -26,6 +26,7 @@ info_log_path = /var/log/dovecot-access.log director_servers =$dirs director_mail_servers = 127.0.0.1-127.0.0.255 +director_user_expire = 15s disable_plaintext_auth = no ssl = no @@ -90,4 +91,4 @@ echo echo "Start testing:" echo -echo "imaptest host=director user=test%d.%d - select=0 no_tracking" +echo "imaptest host=director user=test%d.%d - select=0 no_tracking clients=1000" From dovecot at dovecot.org Mon Jul 29 22:48:49 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:48:49 +0300 Subject: dovecot-2.1: director: Update existing users' timestamps during ... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/77213cdc8d7f changeset: 14987:77213cdc8d7f user: Timo Sirainen date: Mon Jul 29 22:48:36 2013 +0300 description: director: Update existing users' timestamps during handshake. diffstat: src/director/director-connection.c | 17 +++++++++++++++++ src/director/user-directory.c | 37 +++++++++++++++++++++++++++++++++++++ src/director/user-directory.h | 3 +++ 3 files changed, 57 insertions(+), 0 deletions(-) diffs (108 lines): diff -r 15537f882a0b -r 77213cdc8d7f src/director/director-connection.c --- a/src/director/director-connection.c Mon Jul 29 22:18:34 2013 +0300 +++ b/src/director/director-connection.c Mon Jul 29 22:48:36 2013 +0300 @@ -119,6 +119,7 @@ unsigned int synced:1; unsigned int wrong_host:1; unsigned int verifying_left:1; + unsigned int users_unsorted:1; }; static void director_connection_disconnected(struct director_connection **conn); @@ -578,6 +579,10 @@ } director_user_refresh(conn, username_hash, host, timestamp, weak, &user); + if (user->timestamp < timestamp) { + conn->users_unsorted = TRUE; + user->timestamp = timestamp; + } return TRUE; } @@ -975,6 +980,12 @@ unsigned int handshake_secs = time(NULL) - conn->created; string_t *str; + if (conn->users_unsorted && conn->user_iter == NULL) { + /* we sent our user list before receiving remote's */ + conn->users_unsorted = FALSE; + user_directory_sort(conn->dir->users); + } + if (handshake_secs >= DIRECTOR_HANDSHAKE_WARN_SECS || director_debug) { str = t_str_new(128); str_printfa(str, "director(%s): Handshake took %u secs, " @@ -1440,6 +1451,12 @@ user_directory_iter_deinit(&conn->user_iter); director_connection_send(conn, "DONE\n"); + if (conn->users_unsorted && conn->handshake_received) { + /* we received remote's list of users before sending ours */ + conn->users_unsorted = FALSE; + user_directory_sort(conn->dir->users); + } + ret = o_stream_flush(conn->output); timeout_reset(conn->to_ping); return ret; diff -r 15537f882a0b -r 77213cdc8d7f src/director/user-directory.c --- a/src/director/user-directory.c Mon Jul 29 22:18:34 2013 +0300 +++ b/src/director/user-directory.c Mon Jul 29 22:48:36 2013 +0300 @@ -210,6 +210,43 @@ } } +static int user_timestamp_cmp(struct user *const *user1, + struct user *const *user2) +{ + if ((*user1)->timestamp < (*user2)->timestamp) + return -1; + if ((*user1)->timestamp > (*user2)->timestamp) + return 1; + return 0; +} + +void user_directory_sort(struct user_directory *dir) +{ + ARRAY_DEFINE(users, struct user *); + struct user *user, *const *userp; + unsigned int i, users_count = hash_table_count(dir->hash); + + if (users_count == 0) { + i_assert(dir->head == NULL); + return; + } + + /* place all users into array and sort it */ + i_array_init(&users, users_count); + user = dir->head; + for (i = 0; i < users_count; i++, user = user->next) + array_append(&users, &user, 1); + i_assert(user == NULL); + array_sort(&users, user_timestamp_cmp); + + /* recreate the linked list */ + dir->head = dir->tail = NULL; + array_foreach(&users, userp) + DLLIST2_APPEND(&dir->head, &dir->tail, *userp); + i_assert(dir->head->timestamp <= dir->tail->timestamp); + array_free(&users); +} + unsigned int user_directory_get_username_hash(struct user_directory *dir, const char *username) { diff -r 15537f882a0b -r 77213cdc8d7f src/director/user-directory.h --- a/src/director/user-directory.h Mon Jul 29 22:18:34 2013 +0300 +++ b/src/director/user-directory.h Mon Jul 29 22:48:36 2013 +0300 @@ -64,6 +64,9 @@ /* Remove all users that have pointers to given host */ void user_directory_remove_host(struct user_directory *dir, struct mail_host *host); +/* Sort users based on the timestamp. This is called only after updating + timestamps based on remote director's user list after handshake. */ +void user_directory_sort(struct user_directory *dir); unsigned int user_directory_get_username_hash(struct user_directory *dir, const char *username); From dovecot at dovecot.org Mon Jul 29 22:49:22 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 22:49:22 +0300 Subject: dovecot-2.2: director: Update existing users' timestamps during ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/5ef62caea61b changeset: 16621:5ef62caea61b user: Timo Sirainen date: Mon Jul 29 22:49:17 2013 +0300 description: director: Update existing users' timestamps during handshake. diffstat: src/director/director-connection.c | 17 +++++++++++++++++ src/director/user-directory.c | 37 +++++++++++++++++++++++++++++++++++++ src/director/user-directory.h | 3 +++ 3 files changed, 57 insertions(+), 0 deletions(-) diffs (108 lines): diff -r 366024103c09 -r 5ef62caea61b src/director/director-connection.c --- a/src/director/director-connection.c Mon Jul 29 22:19:03 2013 +0300 +++ b/src/director/director-connection.c Mon Jul 29 22:49:17 2013 +0300 @@ -119,6 +119,7 @@ unsigned int synced:1; unsigned int wrong_host:1; unsigned int verifying_left:1; + unsigned int users_unsorted:1; }; static void director_connection_disconnected(struct director_connection **conn); @@ -578,6 +579,10 @@ (void)director_user_refresh(conn, username_hash, host, timestamp, weak, &user); + if (user->timestamp < timestamp) { + conn->users_unsorted = TRUE; + user->timestamp = timestamp; + } return TRUE; } @@ -975,6 +980,12 @@ unsigned int handshake_secs = time(NULL) - conn->created; string_t *str; + if (conn->users_unsorted && conn->user_iter == NULL) { + /* we sent our user list before receiving remote's */ + conn->users_unsorted = FALSE; + user_directory_sort(conn->dir->users); + } + if (handshake_secs >= DIRECTOR_HANDSHAKE_WARN_SECS || director_debug) { str = t_str_new(128); str_printfa(str, "director(%s): Handshake took %u secs, " @@ -1437,6 +1448,12 @@ user_directory_iter_deinit(&conn->user_iter); director_connection_send(conn, "DONE\n"); + if (conn->users_unsorted && conn->handshake_received) { + /* we received remote's list of users before sending ours */ + conn->users_unsorted = FALSE; + user_directory_sort(conn->dir->users); + } + ret = o_stream_flush(conn->output); timeout_reset(conn->to_ping); return ret; diff -r 366024103c09 -r 5ef62caea61b src/director/user-directory.c --- a/src/director/user-directory.c Mon Jul 29 22:19:03 2013 +0300 +++ b/src/director/user-directory.c Mon Jul 29 22:49:17 2013 +0300 @@ -210,6 +210,43 @@ } } +static int user_timestamp_cmp(struct user *const *user1, + struct user *const *user2) +{ + if ((*user1)->timestamp < (*user2)->timestamp) + return -1; + if ((*user1)->timestamp > (*user2)->timestamp) + return 1; + return 0; +} + +void user_directory_sort(struct user_directory *dir) +{ + ARRAY(struct user *) users; + struct user *user, *const *userp; + unsigned int i, users_count = hash_table_count(dir->hash); + + if (users_count == 0) { + i_assert(dir->head == NULL); + return; + } + + /* place all users into array and sort it */ + i_array_init(&users, users_count); + user = dir->head; + for (i = 0; i < users_count; i++, user = user->next) + array_append(&users, &user, 1); + i_assert(user == NULL); + array_sort(&users, user_timestamp_cmp); + + /* recreate the linked list */ + dir->head = dir->tail = NULL; + array_foreach(&users, userp) + DLLIST2_APPEND(&dir->head, &dir->tail, *userp); + i_assert(dir->head->timestamp <= dir->tail->timestamp); + array_free(&users); +} + unsigned int user_directory_get_username_hash(struct user_directory *dir, const char *username) { diff -r 366024103c09 -r 5ef62caea61b src/director/user-directory.h --- a/src/director/user-directory.h Mon Jul 29 22:19:03 2013 +0300 +++ b/src/director/user-directory.h Mon Jul 29 22:49:17 2013 +0300 @@ -64,6 +64,9 @@ /* Remove all users that have pointers to given host */ void user_directory_remove_host(struct user_directory *dir, struct mail_host *host); +/* Sort users based on the timestamp. This is called only after updating + timestamps based on remote director's user list after handshake. */ +void user_directory_sort(struct user_directory *dir); unsigned int user_directory_get_username_hash(struct user_directory *dir, const char *username); From dovecot at dovecot.org Mon Jul 29 23:21:50 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 23:21:50 +0300 Subject: dovecot-2.1: director: Avoid infinite SYNC loops if the originat... Message-ID: details: http://hg.dovecot.org/dovecot-2.1/rev/c9d7586d01ae changeset: 14988:c9d7586d01ae user: Timo Sirainen date: Mon Jul 29 23:21:38 2013 +0300 description: director: Avoid infinite SYNC loops if the originating director goes away for a long time. diffstat: src/director/director-connection.c | 51 +++++++++++++++++++++++++++++-------- src/director/director-host.h | 3 ++ src/director/director.c | 12 +++++--- src/director/director.h | 3 +- 4 files changed, 51 insertions(+), 18 deletions(-) diffs (184 lines): diff -r 77213cdc8d7f -r c9d7586d01ae src/director/director-connection.c --- a/src/director/director-connection.c Mon Jul 29 22:48:36 2013 +0300 +++ b/src/director/director-connection.c Mon Jul 29 23:21:38 2013 +0300 @@ -73,6 +73,7 @@ #define DIRECTOR_WAIT_DISCONNECT_SECS 10 #define DIRECTOR_HANDSHAKE_WARN_SECS 29 #define DIRECTOR_HANDSHAKE_BYTES_LOG_MIN_SECS (60*30) +#define DIRECTOR_MAX_SYNC_SEQ_DUPLICATES 4 #if DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS <= DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS # error DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS is too low @@ -217,7 +218,7 @@ dir->sync_seq++; director_set_ring_unsynced(dir); director_sync_send(dir, dir->self_host, dir->sync_seq, - DIRECTOR_VERSION_MINOR); + DIRECTOR_VERSION_MINOR, ioloop_time); } director_connection_set_ping_timeout(conn); } @@ -1084,10 +1085,11 @@ return 0; } -static void +static bool director_connection_sync_host(struct director_connection *conn, struct director_host *host, - uint32_t seq, unsigned int minor_version) + uint32_t seq, unsigned int minor_version, + unsigned int timestamp) { struct director *dir = conn->dir; @@ -1099,7 +1101,7 @@ if (host->self) { if (dir->sync_seq != seq) { /* stale SYNC event */ - return; + return FALSE; } /* sync_seq increases when we get disconnected, so we must be successfully connected to both directions */ @@ -1117,10 +1119,28 @@ conn->name, seq); director_set_ring_synced(dir); } - } else if (dir->right != NULL) { - /* forward it to the connection on right */ - director_sync_send(dir, host, seq, minor_version); + } else { + if (seq < host->last_sync_seq) { + /* stale SYNC event */ + return FALSE; + } else if (host->last_sync_seq != seq || + timestamp < host->last_sync_timestamp) { + host->last_sync_seq = seq; + host->last_sync_timestamp = timestamp; + host->last_sync_seq_counter = 1; + } else if (++host->last_sync_seq_counter > + DIRECTOR_MAX_SYNC_SEQ_DUPLICATES) { + /* we've received this too many times already */ + return FALSE; + } + + if (dir->right != NULL) { + /* forward it to the connection on right */ + director_sync_send(dir, host, seq, minor_version, + timestamp); + } } + return TRUE; } static bool director_connection_sync(struct director_connection *conn, @@ -1129,7 +1149,7 @@ struct director *dir = conn->dir; struct director_host *host; struct ip_addr ip; - unsigned int port, seq, minor_version = 0; + unsigned int port, seq, minor_version = 0, timestamp = ioloop_time; if (str_array_length(args) < 3 || !director_args_parse_ip_port(conn, args, &ip, &port) || @@ -1137,18 +1157,25 @@ director_cmd_error(conn, "Invalid parameters"); return FALSE; } - if (args[3] != NULL) + if (args[3] != NULL) { minor_version = atoi(args[3]); + if (args[4] != NULL && str_to_uint(args[4], ×tamp) < 0) { + director_cmd_error(conn, "Invalid parameters"); + return FALSE; + } + } /* find the originating director. if we don't see it, it was already removed and we can ignore this sync. */ host = director_host_lookup(dir, &ip, port); if (host != NULL) { - director_connection_sync_host(conn, host, seq, - minor_version); + if (!director_connection_sync_host(conn, host, seq, + minor_version, timestamp)) + return TRUE; } - if (host == NULL || !host->self) + if ((host == NULL || !host->self) && + dir->self_host->last_sync_timestamp != ioloop_time) director_resend_sync(dir); return TRUE; } diff -r 77213cdc8d7f -r c9d7586d01ae src/director/director-host.h --- a/src/director/director-host.h Mon Jul 29 22:48:36 2013 +0300 +++ b/src/director/director-host.h Mon Jul 29 23:21:38 2013 +0300 @@ -20,6 +20,9 @@ it can be ignored (or: it must be ignored to avoid potential command loops) */ unsigned int last_seq; + /* use these to avoid infinitely sending SYNCs for directors that + aren't connected in the ring. */ + unsigned int last_sync_seq, last_sync_seq_counter, last_sync_timestamp; /* Last time host was detected to be down */ time_t last_network_failure; time_t last_protocol_failure; diff -r 77213cdc8d7f -r c9d7586d01ae src/director/director.c --- a/src/director/director.c Mon Jul 29 22:48:36 2013 +0300 +++ b/src/director/director.c Mon Jul 29 23:21:38 2013 +0300 @@ -302,7 +302,8 @@ } void director_sync_send(struct director *dir, struct director_host *host, - uint32_t seq, unsigned int minor_version) + uint32_t seq, unsigned int minor_version, + unsigned int timestamp) { string_t *str; @@ -311,8 +312,8 @@ net_ip2addr(&host->ip), host->port, seq); if (minor_version > 0 && director_connection_get_minor_version(dir->right) > 0) { - /* only minor_version>0 supports this parameter */ - str_printfa(str, "\t%u", minor_version); + /* only minor_version>0 supports extra parameters */ + str_printfa(str, "\t%u\t%u", minor_version, timestamp); } str_append_c(str, '\n'); director_connection_send(dir->right, str_c(str)); @@ -329,8 +330,9 @@ { if (!dir->ring_synced && dir->left != NULL && dir->right != NULL) { /* send a new SYNC in case the previous one got dropped */ + dir->self_host->last_sync_timestamp = ioloop_time; director_sync_send(dir, dir->self_host, dir->sync_seq, - DIRECTOR_VERSION_MINOR); + DIRECTOR_VERSION_MINOR, ioloop_time); if (dir->to_sync != NULL) timeout_reset(dir->to_sync); return TRUE; @@ -393,7 +395,7 @@ director_connection_set_synced(dir->left, FALSE); director_connection_set_synced(dir->right, FALSE); director_sync_send(dir, dir->self_host, dir->sync_seq, - DIRECTOR_VERSION_MINOR); + DIRECTOR_VERSION_MINOR, ioloop_time); } void director_sync_freeze(struct director *dir) diff -r 77213cdc8d7f -r c9d7586d01ae src/director/director.h --- a/src/director/director.h Mon Jul 29 22:48:36 2013 +0300 +++ b/src/director/director.h Mon Jul 29 23:21:38 2013 +0300 @@ -103,7 +103,8 @@ void director_set_ring_unsynced(struct director *dir); void director_set_state_changed(struct director *dir); void director_sync_send(struct director *dir, struct director_host *host, - uint32_t seq, unsigned int minor_version); + uint32_t seq, unsigned int minor_version, + unsigned int timestamp); bool director_resend_sync(struct director *dir); void director_notify_ring_added(struct director_host *added_host, From dovecot at dovecot.org Mon Jul 29 23:22:52 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Mon, 29 Jul 2013 23:22:52 +0300 Subject: dovecot-2.2: director: Avoid infinite SYNC loops if the originat... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/a6736e42301e changeset: 16622:a6736e42301e user: Timo Sirainen date: Mon Jul 29 23:22:46 2013 +0300 description: director: Avoid infinite SYNC loops if the originating director goes away for a long time. diffstat: src/director/director-connection.c | 51 +++++++++++++++++++++++++++++-------- src/director/director-host.h | 3 ++ src/director/director.c | 12 +++++--- src/director/director.h | 3 +- 4 files changed, 51 insertions(+), 18 deletions(-) diffs (184 lines): diff -r 5ef62caea61b -r a6736e42301e src/director/director-connection.c --- a/src/director/director-connection.c Mon Jul 29 22:49:17 2013 +0300 +++ b/src/director/director-connection.c Mon Jul 29 23:22:46 2013 +0300 @@ -73,6 +73,7 @@ #define DIRECTOR_WAIT_DISCONNECT_SECS 10 #define DIRECTOR_HANDSHAKE_WARN_SECS 29 #define DIRECTOR_HANDSHAKE_BYTES_LOG_MIN_SECS (60*30) +#define DIRECTOR_MAX_SYNC_SEQ_DUPLICATES 4 #if DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS <= DIRECTOR_CONNECTION_PING_TIMEOUT_MSECS # error DIRECTOR_CONNECTION_DONE_TIMEOUT_MSECS is too low @@ -216,7 +217,7 @@ dir->sync_seq++; director_set_ring_unsynced(dir); director_sync_send(dir, dir->self_host, dir->sync_seq, - DIRECTOR_VERSION_MINOR); + DIRECTOR_VERSION_MINOR, ioloop_time); } director_connection_set_ping_timeout(conn); } @@ -1084,10 +1085,11 @@ return 0; } -static void +static bool director_connection_sync_host(struct director_connection *conn, struct director_host *host, - uint32_t seq, unsigned int minor_version) + uint32_t seq, unsigned int minor_version, + unsigned int timestamp) { struct director *dir = conn->dir; @@ -1099,7 +1101,7 @@ if (host->self) { if (dir->sync_seq != seq) { /* stale SYNC event */ - return; + return FALSE; } /* sync_seq increases when we get disconnected, so we must be successfully connected to both directions */ @@ -1117,10 +1119,28 @@ conn->name, seq); director_set_ring_synced(dir); } - } else if (dir->right != NULL) { - /* forward it to the connection on right */ - director_sync_send(dir, host, seq, minor_version); + } else { + if (seq < host->last_sync_seq) { + /* stale SYNC event */ + return FALSE; + } else if (host->last_sync_seq != seq || + timestamp < host->last_sync_timestamp) { + host->last_sync_seq = seq; + host->last_sync_timestamp = timestamp; + host->last_sync_seq_counter = 1; + } else if (++host->last_sync_seq_counter > + DIRECTOR_MAX_SYNC_SEQ_DUPLICATES) { + /* we've received this too many times already */ + return FALSE; + } + + if (dir->right != NULL) { + /* forward it to the connection on right */ + director_sync_send(dir, host, seq, minor_version, + timestamp); + } } + return TRUE; } static bool director_connection_sync(struct director_connection *conn, @@ -1129,7 +1149,7 @@ struct director *dir = conn->dir; struct director_host *host; struct ip_addr ip; - unsigned int port, seq, minor_version = 0; + unsigned int port, seq, minor_version = 0, timestamp = ioloop_time; if (str_array_length(args) < 3 || !director_args_parse_ip_port(conn, args, &ip, &port) || @@ -1137,18 +1157,25 @@ director_cmd_error(conn, "Invalid parameters"); return FALSE; } - if (args[3] != NULL) + if (args[3] != NULL) { minor_version = atoi(args[3]); + if (args[4] != NULL && str_to_uint(args[4], ×tamp) < 0) { + director_cmd_error(conn, "Invalid parameters"); + return FALSE; + } + } /* find the originating director. if we don't see it, it was already removed and we can ignore this sync. */ host = director_host_lookup(dir, &ip, port); if (host != NULL) { - director_connection_sync_host(conn, host, seq, - minor_version); + if (!director_connection_sync_host(conn, host, seq, + minor_version, timestamp)) + return TRUE; } - if (host == NULL || !host->self) + if ((host == NULL || !host->self) && + dir->self_host->last_sync_timestamp != ioloop_time) (void)director_resend_sync(dir); return TRUE; } diff -r 5ef62caea61b -r a6736e42301e src/director/director-host.h --- a/src/director/director-host.h Mon Jul 29 22:49:17 2013 +0300 +++ b/src/director/director-host.h Mon Jul 29 23:22:46 2013 +0300 @@ -20,6 +20,9 @@ it can be ignored (or: it must be ignored to avoid potential command loops) */ unsigned int last_seq; + /* use these to avoid infinitely sending SYNCs for directors that + aren't connected in the ring. */ + unsigned int last_sync_seq, last_sync_seq_counter, last_sync_timestamp; /* Last time host was detected to be down */ time_t last_network_failure; time_t last_protocol_failure; diff -r 5ef62caea61b -r a6736e42301e src/director/director.c --- a/src/director/director.c Mon Jul 29 22:49:17 2013 +0300 +++ b/src/director/director.c Mon Jul 29 23:22:46 2013 +0300 @@ -302,7 +302,8 @@ } void director_sync_send(struct director *dir, struct director_host *host, - uint32_t seq, unsigned int minor_version) + uint32_t seq, unsigned int minor_version, + unsigned int timestamp) { string_t *str; @@ -311,8 +312,8 @@ net_ip2addr(&host->ip), host->port, seq); if (minor_version > 0 && director_connection_get_minor_version(dir->right) > 0) { - /* only minor_version>0 supports this parameter */ - str_printfa(str, "\t%u", minor_version); + /* only minor_version>0 supports extra parameters */ + str_printfa(str, "\t%u\t%u", minor_version, timestamp); } str_append_c(str, '\n'); director_connection_send(dir->right, str_c(str)); @@ -329,8 +330,9 @@ { if (!dir->ring_synced && dir->left != NULL && dir->right != NULL) { /* send a new SYNC in case the previous one got dropped */ + dir->self_host->last_sync_timestamp = ioloop_time; director_sync_send(dir, dir->self_host, dir->sync_seq, - DIRECTOR_VERSION_MINOR); + DIRECTOR_VERSION_MINOR, ioloop_time); if (dir->to_sync != NULL) timeout_reset(dir->to_sync); return TRUE; @@ -393,7 +395,7 @@ director_connection_set_synced(dir->left, FALSE); director_connection_set_synced(dir->right, FALSE); director_sync_send(dir, dir->self_host, dir->sync_seq, - DIRECTOR_VERSION_MINOR); + DIRECTOR_VERSION_MINOR, ioloop_time); } void director_sync_freeze(struct director *dir) diff -r 5ef62caea61b -r a6736e42301e src/director/director.h --- a/src/director/director.h Mon Jul 29 22:49:17 2013 +0300 +++ b/src/director/director.h Mon Jul 29 23:22:46 2013 +0300 @@ -103,7 +103,8 @@ void director_set_ring_unsynced(struct director *dir); void director_set_state_changed(struct director *dir); void director_sync_send(struct director *dir, struct director_host *host, - uint32_t seq, unsigned int minor_version); + uint32_t seq, unsigned int minor_version, + unsigned int timestamp); bool director_resend_sync(struct director *dir); void director_notify_ring_added(struct director_host *added_host, From dovecot at dovecot.org Tue Jul 30 13:09:57 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Tue, 30 Jul 2013 13:09:57 +0300 Subject: dovecot-2.2: lib-auth: Don't crash in auth_master_user_list_*() ... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/0cab916b28c6 changeset: 16623:0cab916b28c6 user: Timo Sirainen date: Tue Jul 30 13:09:52 2013 +0300 description: lib-auth: Don't crash in auth_master_user_list_*() if socket can't be connected to. diffstat: src/lib-auth/auth-master.c | 19 ++++++++++++------- 1 files changed, 12 insertions(+), 7 deletions(-) diffs (43 lines): diff -r a6736e42301e -r 0cab916b28c6 src/lib-auth/auth-master.c --- a/src/lib-auth/auth-master.c Mon Jul 29 23:22:46 2013 +0300 +++ b/src/lib-auth/auth-master.c Tue Jul 30 13:09:52 2013 +0300 @@ -352,13 +352,15 @@ io_loop_set_current(conn->prev_ioloop); lib_signals_reset_ioloop(); } - io_loop_set_current(conn->ioloop); + if (conn->ioloop != NULL) { + io_loop_set_current(conn->ioloop); - timeout_remove(&conn->to); - io_remove(&conn->io); - i_stream_unref(&conn->input); - o_stream_unref(&conn->output); - io_loop_destroy(&conn->ioloop); + timeout_remove(&conn->to); + io_remove(&conn->io); + i_stream_unref(&conn->input); + o_stream_unref(&conn->output); + io_loop_destroy(&conn->ioloop); + } if ((conn->flags & AUTH_MASTER_FLAG_NO_IDLE_TIMEOUT) == 0) { if (conn->prev_ioloop == NULL) @@ -685,13 +687,16 @@ { const char *line; + if (ctx->failed) + return NULL; + /* try to read already buffered input */ line = i_stream_next_line(ctx->conn->input); if (line != NULL) { T_BEGIN { auth_handle_line(ctx->conn, line); } T_END; - } else if (!ctx->failed) { + } else { /* wait for more data */ io_loop_run(ctx->conn->ioloop); } From dovecot at dovecot.org Tue Jul 30 13:26:51 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Tue, 30 Jul 2013 13:26:51 +0300 Subject: dovecot-2.2: lib-storage: With temporary privilege dropping, swi... Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/2470bb9106b0 changeset: 16624:2470bb9106b0 user: Timo Sirainen date: Tue Jul 30 13:26:07 2013 +0300 description: lib-storage: With temporary privilege dropping, switch to root before doing config lookup. This fixes the problem of not having enough permissions to connect to the config socket. diffstat: src/lib-storage/mail-storage-service.c | 11 ++++++++++- 1 files changed, 10 insertions(+), 1 deletions(-) diffs (28 lines): diff -r 0cab916b28c6 -r 2470bb9106b0 src/lib-storage/mail-storage-service.c --- a/src/lib-storage/mail-storage-service.c Tue Jul 30 13:09:52 2013 +0300 +++ b/src/lib-storage/mail-storage-service.c Tue Jul 30 13:26:07 2013 +0300 @@ -967,6 +967,16 @@ int ret = 1; user_pool = pool_alloconly_create("mail storage service user", 1024*6); + flags = mail_storage_service_input_get_flags(ctx, input); + + if ((flags & MAIL_STORAGE_SERVICE_FLAG_TEMP_PRIV_DROP) != 0 && + geteuid() != 0) { + /* we dropped privileges only temporarily. switch back to root + before reading settings, so we'll definitely have enough + permissions to connect to the config socket. */ + if (seteuid(0) < 0) + i_fatal("seteuid(0) failed: %m"); + } if (mail_storage_service_read_settings(ctx, input, user_pool, &user_info, &set_parser, @@ -977,7 +987,6 @@ return -1; } - flags = mail_storage_service_input_get_flags(ctx, input); if ((flags & MAIL_STORAGE_SERVICE_FLAG_NO_LOG_INIT) == 0 && !ctx->log_initialized) { /* initialize logging again, in case we only read the From dovecot at dovecot.org Tue Jul 30 13:26:51 2013 From: dovecot at dovecot.org (dovecot at dovecot.org) Date: Tue, 30 Jul 2013 13:26:51 +0300 Subject: dovecot-2.2: quota-status: Keep the config connection open. Message-ID: details: http://hg.dovecot.org/dovecot-2.2/rev/51b8020b29f6 changeset: 16625:51b8020b29f6 user: Timo Sirainen date: Tue Jul 30 13:26:40 2013 +0300 description: quota-status: Keep the config connection open. diffstat: src/plugins/quota/quota-status.c | 4 +++- 1 files changed, 3 insertions(+), 1 deletions(-) diffs (17 lines): diff -r 2470bb9106b0 -r 51b8020b29f6 src/plugins/quota/quota-status.c --- a/src/plugins/quota/quota-status.c Tue Jul 30 13:26:07 2013 +0300 +++ b/src/plugins/quota/quota-status.c Tue Jul 30 13:26:40 2013 +0300 @@ -215,10 +215,12 @@ int main(int argc, char *argv[]) { + enum master_service_flags service_flags = + MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN; int c; protocol = QUOTA_PROTOCOL_UNKNOWN; - master_service = master_service_init("quota-status", 0, + master_service = master_service_init("quota-status", service_flags, &argc, &argv, "p:"); while ((c = master_getopt(master_service)) > 0) { switch (c) {