Re: proxy-dict with tcp connection

Hi Aki, Am 03.08.17 um 19:49 schrieb Aki Tuomi:

...

On August 3, 2017 at 2:10 PM Ralf Becker wrote:

I try to create a patch to allow (proxy-)dict to use tcp connections instead of a unix domain socket.

I'm replacing connection_init_client_unix with connection_init_client_ip:

--- ./src/lib-dict/dict-client.c.orig +++ ./src/lib-dict/dict-client.c @@ -721,6 +721,10 @@ client_dict_init(struct dict *driver, const char *uri, struct ioloop *old_ioloop = current_ioloop; struct client_dict *dict; const char *p, *dest_uri, *path; + const char *const *args; + unsigned int argc; + struct ip_addr ip; + in_port_t port=0; unsigned int idle_msecs = DICT_CLIENT_DEFAULT_TIMEOUT_MSECS; unsigned int warn_slow_msecs = DICT_CLIENT_DEFAULT_WARN_SLOW_MSECS;

@@ -772,7 +776,21 @@ client_dict_init(struct dict *driver, const char *uri, dict->warn_slow_msecs = warn_slow_msecs; i_array_init(&dict->cmds, 32);

- if (uri[0] == ':') { + args = t_strsplit(uri, ":"); + for(argc=0; args[argc] != NULL; argc++); + + if (argc == 3) { /* host:ip:somewhere --> argc == 3 */ + if (net_addr2ip(args[0], &ip) < 0) { + *error_r = t_strdup_printf("Invalid IP: %s in URI: %s", args[0], uri); + return -1; + } + if (net_str2port(args[1], &port) < 0) { + *error_r = t_strdup_printf("Invalid port: %s in URI: %s", args[1], uri); + return -1; + } + dest_uri = strrchr(uri, ':'); + } else if (uri[0] == ':') { /* default path */ path = t_strconcat(set->base_dir, "/"DEFAULT_DICT_SERVER_SOCKET_FNAME, NULL); @@ -784,7 +802,13 @@ client_dict_init(struct dict *driver, const char *uri, path = t_strconcat(set->base_dir, "/", t_strdup_until(uri, dest_uri), NULL); } - connection_init_client_unix(dict_connections, &dict->conn.conn, path); + if (port > 0) { + connection_init_client_ip(dict_connections, &dict->conn.conn, &ip, port); + } else { + connection_init_client_unix(dict_connections, &dict->conn.conn, path); + } dict->uri = i_strdup(dest_uri + 1);

dict->ioloop = io_loop_create();

But unfortunately this crashes:

Jul 28 13:20:04 auth: Error: auth worker: Aborted PASSL request for info@outdoor-training.de: Worker process died unexpectedly Jul 28 13:20:04 auth-worker(705): Fatal: master: service(auth-worker): child 705 killed with signal 11 (core dumped) Jul 28 13:20:04 doveadm(10.44.88.1,info@outdoor-training.de): Error: user info@outdoor-training.de: Auth PASS lookup failed

It looks like the tcp connection gets opened non-blocking and the first write / dict lookup happens to early:

4303041 13:44:25.120398220 0 auth (29884) < connect res=-115(EINPROGRESS) tuple=172.18.0.2:47552->10.44.99.180:2001

Looking at dict-memcached-ascii.c I probably need to do something like:

i_array_init(&dict->input_states, 4); i_array_init(&dict->replies, 4);

dict->ioloop = io_loop_create(); io_loop_set_current(old_ioloop); *dict_r = &dict->dict;

to wait until the socket is ready ...

Any idea / tips?

Ralf It's probably cleaner to make a "proxy-tcp" driver so parsing all the funny things gets easier. Also it will require some restructing in the client_dict_connect code.

Thanks for the advice. I will try to find a way to share protocol stuff, but keep connection details separate. Can you elaborate a bit about how eg. dict-memcached-ascii is waiting between opening the socket and writing to it: https://github.com/dovecot/core/blob/master/src/lib-dict/dict-memcached-asci... Btw. do you know if your company would be available to develop such small enhancements like dict using tcp instead of unix-domain sockets? Ralf -- Ralf Becker EGroupware GmbH [www.egroupware.org] Handelsregister HRB Kaiserslautern 3587 Geschäftsführer Birgit und Ralf Becker Leibnizstr. 17, 67663 Kaiserslautern, Germany Telefon +49 631 31657-0