Thanks, Stephan.  I've added the workaround patch to the FreeBSD port.  I went through the mailbox trees that I was seeing errors on, and no crashes with the patches applied. 

Let me know if there is anything I can do to help or provide access to. 



On Wed, Jan 2, 2019 at 4:05 PM Stephan Bosch <stephan@rename-it.nl> wrote:

Op 02/01/2019 om 22:51 schreef Larry Rosenman:
> https://www.lerctr.org/~ler/dovecot/doveadm-index-fts-debug.txt
>
> https://www.lerctr.org/~ler/dovecot/doveadm-index-fts-bt.txt
>
> I wish there was a way to set plugins {fts_solr = <blah>} from the
> command line :(
>
> but I turned it on globally for that run.

Yeah, it is caused by the interaction between the solr and tika HTTP
clients in this case. The bug itself will require some care and
attention, but a quick fix should be provided by:

diff --git a/src/plugins/fts-solr/solr-connection.c
b/src/plugins/fts-solr/solr-connection.c
index 947ea76af..3389729d1 100644
--- a/src/plugins/fts-solr/solr-connection.c
+++ b/src/plugins/fts-solr/solr-connection.c
@@ -156,7 +156,7 @@ int solr_connection_init(const char *url,
                 http_set.request_timeout_msecs = 60*1000;
                 http_set.ssl = ssl_client_set;
                 http_set.debug = debug;
-               solr_http_client = http_client_init(&http_set);
+               solr_http_client = http_client_init_private(&http_set);
         }

         conn->xml_parser = XML_ParserCreate("UTF-8");
diff --git a/src/plugins/fts/fts-parser-tika.c
b/src/plugins/fts/fts-parser-tika.c
index a4b8b5c30..b8b57a350 100644
--- a/src/plugins/fts/fts-parser-tika.c
+++ b/src/plugins/fts/fts-parser-tika.c
@@ -77,7 +77,7 @@ tika_get_http_client_url(struct mail_user *user,
struct http_url **http_url_r)
                 http_set.request_timeout_msecs = 60*1000;
                 http_set.ssl = &ssl_set;
                 http_set.debug = user->mail_debug;
-               tika_http_client = http_client_init(&http_set);
+               tika_http_client = http_client_init_private(&http_set);
         }
         *http_url_r = tuser->http_url;
         return 0;

This will bluntly prevent sharing any state between Solr and Tika HTTP
clients, also across users (if there is more than one per mail process).

Regards,


Stephan.

>
> On Wed, Jan 2, 2019 at 3:40 PM Stephan Bosch <stephan@rename-it.nl
> <mailto:stephan@rename-it.nl>> wrote:
>
>     Oh, d'oh. I was looking for some solr debug logging in your
>     output, but
>     I didn't see much of it. Then I noticed that for some reason fts-solr
>     has its own debug setting , while fts-tika uses mail_debug. Can you
>     enable that as well?
>
>     Regards,
>
>     Stephan.
>
>     Op 02/01/2019 om 14:41 schreef Larry Rosenman:
>     >
>     > created with: doveadm -D -o mail_debug=yes index 2>&1 | tee
>     > https://www.lerctr.org/~ler/dovecot/doveadm.index.debug.out
>     >
>     > bt full for the above:
>     > https://www.lerctr.org/~ler/dovecot/doveadm.index.crash.bt.txt
>     >
>     > On Wed, Jan 2, 2019 at 2:11 AM Stephan Bosch
>     <stephan@rename-it.nl <mailto:stephan@rename-it.nl>
>     > <mailto:stephan@rename-it.nl <mailto:stephan@rename-it.nl>>> wrote:
>     >
>     >
>     >     Op 01/01/2019 om 02:12 schreef Larry Rosenman:
>     >     > I got the following crash whilst looking at my logs and
>     seeing some
>     >     > indexer-worker crashes.
>     >     >
>     >     > doveadm(ler): Panic: file http-client-request.c: line 1108
>     >     > (http_client_request_send_more): assertion failed:
>     >     (req->payload_input
>     >     > != NULL)
>     >     > Abort trap (core dumped)
>     >     >
>     >     > bt full at:
>     >     > https://www.lerctr.org/~ler/doveadm.index.crash.txt
>     >     >
>     >     > doveconf -n:
>     >     > https://www.lerctr.org/~ler/doveconf.n.txt
>     >     >
>     >     > 2.3.4 from FreeBSD ports.
>     >     >
>     >     > What else do you need?
>     >
>     >     This may have something to do with the fact that you're
>     using both
>     >     Tika
>     >     and Solr. Is this something you can reproduce? It would be
>     helpful to
>     >     have a debug log for the Tika and Solr HTTP exchange before the
>     >     panic.
>     >     This can be obtained by having mail_debug=yes.
>     >
>     >     Regards,
>     >
>     >
>     >     Stephan.
>     >
>     >
>     >
>     >     >
>     >     >
>     >     > --
>     >     > Larry Rosenman http://www.lerctr.org/~ler
>     >     > Phone: +1 214-642-9640 (c)     E-Mail: larryrtx@gmail.com
>     <mailto:larryrtx@gmail.com>
>     >     <mailto:larryrtx@gmail.com <mailto:larryrtx@gmail.com>>
>     >     > <mailto:larryrtx@gmail.com <mailto:larryrtx@gmail.com>
>     <mailto:larryrtx@gmail.com <mailto:larryrtx@gmail.com>>>
>     >     > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
>     >
>     >
>     >
>     > --
>     > Larry Rosenman http://www.lerctr.org/~ler
>     > Phone: +1 214-642-9640 (c) E-Mail: larryrtx@gmail.com
>     <mailto:larryrtx@gmail.com>
>     > <mailto:larryrtx@gmail.com <mailto:larryrtx@gmail.com>>
>     > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
>
>
>
> --
> Larry Rosenman http://www.lerctr.org/~ler
> Phone: +1 214-642-9640 (c) E-Mail: larryrtx@gmail.com
> <mailto:larryrtx@gmail.com>
> US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106


--
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c)     E-Mail: larryrtx@gmail.com
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106