doveadm index crash/assert
Larry Rosenman
larryrtx at gmail.com
Thu Jan 3 01:55:28 EET 2019
Thanks, Stephan. I've added the workaround patch to the FreeBSD port. I
went through the mailbox trees that I was seeing errors on, and no crashes
with the patches applied.
Let me know if there is anything I can do to help or provide access to.
On Wed, Jan 2, 2019 at 4:05 PM Stephan Bosch <stephan at rename-it.nl> wrote:
>
> Op 02/01/2019 om 22:51 schreef Larry Rosenman:
> > https://www.lerctr.org/~ler/dovecot/doveadm-index-fts-debug.txt
> >
> > https://www.lerctr.org/~ler/dovecot/doveadm-index-fts-bt.txt
> >
> > I wish there was a way to set plugins {fts_solr = <blah>} from the
> > command line :(
> >
> > but I turned it on globally for that run.
>
> Yeah, it is caused by the interaction between the solr and tika HTTP
> clients in this case. The bug itself will require some care and
> attention, but a quick fix should be provided by:
>
> diff --git a/src/plugins/fts-solr/solr-connection.c
> b/src/plugins/fts-solr/solr-connection.c
> index 947ea76af..3389729d1 100644
> --- a/src/plugins/fts-solr/solr-connection.c
> +++ b/src/plugins/fts-solr/solr-connection.c
> @@ -156,7 +156,7 @@ int solr_connection_init(const char *url,
> http_set.request_timeout_msecs = 60*1000;
> http_set.ssl = ssl_client_set;
> http_set.debug = debug;
> - solr_http_client = http_client_init(&http_set);
> + solr_http_client = http_client_init_private(&http_set);
> }
>
> conn->xml_parser = XML_ParserCreate("UTF-8");
> diff --git a/src/plugins/fts/fts-parser-tika.c
> b/src/plugins/fts/fts-parser-tika.c
> index a4b8b5c30..b8b57a350 100644
> --- a/src/plugins/fts/fts-parser-tika.c
> +++ b/src/plugins/fts/fts-parser-tika.c
> @@ -77,7 +77,7 @@ tika_get_http_client_url(struct mail_user *user,
> struct http_url **http_url_r)
> http_set.request_timeout_msecs = 60*1000;
> http_set.ssl = &ssl_set;
> http_set.debug = user->mail_debug;
> - tika_http_client = http_client_init(&http_set);
> + tika_http_client = http_client_init_private(&http_set);
> }
> *http_url_r = tuser->http_url;
> return 0;
>
> This will bluntly prevent sharing any state between Solr and Tika HTTP
> clients, also across users (if there is more than one per mail process).
>
> Regards,
>
>
> Stephan.
>
> >
> > On Wed, Jan 2, 2019 at 3:40 PM Stephan Bosch <stephan at rename-it.nl
> > <mailto:stephan at rename-it.nl>> wrote:
> >
> > Oh, d'oh. I was looking for some solr debug logging in your
> > output, but
> > I didn't see much of it. Then I noticed that for some reason fts-solr
> > has its own debug setting , while fts-tika uses mail_debug. Can you
> > enable that as well?
> >
> > Regards,
> >
> > Stephan.
> >
> > Op 02/01/2019 om 14:41 schreef Larry Rosenman:
> > >
> > > created with: doveadm -D -o mail_debug=yes index 2>&1 | tee
> > > https://www.lerctr.org/~ler/dovecot/doveadm.index.debug.out
> > >
> > > bt full for the above:
> > > https://www.lerctr.org/~ler/dovecot/doveadm.index.crash.bt.txt
> > >
> > > On Wed, Jan 2, 2019 at 2:11 AM Stephan Bosch
> > <stephan at rename-it.nl <mailto:stephan at rename-it.nl>
> > > <mailto:stephan at rename-it.nl <mailto:stephan at rename-it.nl>>>
> wrote:
> > >
> > >
> > > Op 01/01/2019 om 02:12 schreef Larry Rosenman:
> > > > I got the following crash whilst looking at my logs and
> > seeing some
> > > > indexer-worker crashes.
> > > >
> > > > doveadm(ler): Panic: file http-client-request.c: line 1108
> > > > (http_client_request_send_more): assertion failed:
> > > (req->payload_input
> > > > != NULL)
> > > > Abort trap (core dumped)
> > > >
> > > > bt full at:
> > > > https://www.lerctr.org/~ler/doveadm.index.crash.txt
> > > >
> > > > doveconf -n:
> > > > https://www.lerctr.org/~ler/doveconf.n.txt
> > > >
> > > > 2.3.4 from FreeBSD ports.
> > > >
> > > > What else do you need?
> > >
> > > This may have something to do with the fact that you're
> > using both
> > > Tika
> > > and Solr. Is this something you can reproduce? It would be
> > helpful to
> > > have a debug log for the Tika and Solr HTTP exchange before the
> > > panic.
> > > This can be obtained by having mail_debug=yes.
> > >
> > > Regards,
> > >
> > >
> > > Stephan.
> > >
> > >
> > >
> > > >
> > > >
> > > > --
> > > > Larry Rosenman http://www.lerctr.org/~ler
> > > > Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com
> > <mailto:larryrtx at gmail.com>
> > > <mailto:larryrtx at gmail.com <mailto:larryrtx at gmail.com>>
> > > > <mailto:larryrtx at gmail.com <mailto:larryrtx at gmail.com>
> > <mailto:larryrtx at gmail.com <mailto:larryrtx at gmail.com>>>
> > > > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
> > >
> > >
> > >
> > > --
> > > Larry Rosenman http://www.lerctr.org/~ler
> > > Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com
> > <mailto:larryrtx at gmail.com>
> > > <mailto:larryrtx at gmail.com <mailto:larryrtx at gmail.com>>
> > > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
> >
> >
> >
> > --
> > Larry Rosenman http://www.lerctr.org/~ler
> > Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com
> > <mailto:larryrtx at gmail.com>
> > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
>
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20190102/da3cd641/attachment.html>
More information about the dovecot
mailing list