Problem with tika

[Arjen Heidinga]

Hello all,

For some time now we've bin using solr as a search engine (working
great). I have added tika for searching inside documents, however it
keeps crashing when indexing. When just indexing mails that arrive works
good, but a reindex on all mail constantly crashes with below stacktrace.

When I observe the packets with wireshark I see a HTTP-flow going to
tika and suddenly, midway a document dovecot (or the server) sends a
RST. Sometimes after thousands of documents, sometimes after a few.
Usually after a few hundred.

It appears there are less errors using http than https.



Relevant config:

OS: CentOS6, fully updated

plugin {
  fts = solr
  batch_size = 1
  fts_solr =
url=https://username:password@solr-01.vevida.net:443/solr/dovecot/
  #fts_tika = https://solr-01.vevida.net:443/tika/
  batch_size = 1000
  fts_autoindex=yes
  soft_commit=no
}

# dovecot --version
2.3.9.3 (9f41b88fa)

# Configure options:

    --docdir=%{_docdir}/dovecot  \
    --disable-static             \
    --with-nss                   \
    --with-shadow                \
    --with-pam                   \
    --with-gssapi=plugin         \
    --with-ldap=plugin           \
    --with-sql=plugin            \
    --with-pgsql                 \
    --with-sqlite                \
    --with-zlib                  \
    --with-bzlib                 \
    --with-lzma                  \
    --with-libcap                \
    --with-ssl=openssl           \
    --with-ssldir=%{ssldir}      \
    --with-solr                  \
    --with-docs

# It is compiled agains the latest openssl

# Tika and Solr: Both latest versions.

#Stacktrace:

doveadm(info at samenmetrenske.nl): Info: Sent: Caching mails seq=1..161

doveadm(info at samenmetrenske.nl): Panic: file http-client-request.c: line
1173 (http_client_request_send_more): assertion failed:
(req->payload_input != NULL)

doveadm(info at xxxxxxxxxxxxxxxxxxx.x): Error: Raw backtrace:
/usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x2f)
[0x7f95d805acbf] ->
/usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x26) [0x7f95d805add6]
-> /usr/lib64/dovecot/libdovecot.so.0(+0xe90ba) [0x7f95d80660ba] ->
/usr/lib64/dovecot/libdovecot.so.0(+0xe9161) [0x7f95d8066161] ->
/usr/lib64/dovecot/libdovecot.so.0(+0x41158) [0x7f95d7fbe158] ->
/usr/lib64/dovecot/libdovecot.so.0(http_client_request_send_more+0x424)
[0x7f95d8005094] ->
/usr/lib64/dovecot/libdovecot.so.0(http_client_connection_output+0x11a)
[0x7f95d800a24a] ->
/usr/lib64/dovecot/libssl_iostream_openssl.so(+0x8f6a) [0x7f95d57a2f6a]
-> /usr/lib64/dovecot/libdovecot.so.0(+0x114483) [0x7f95d8091483] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x61)
[0x7f95d807e581] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xdc)
[0x7f95d808076c] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x5c)
[0x7f95d807e67c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38)
[0x7f95d807e8c8] -> /usr/lib64/dovecot/libdovecot.so.0(+0x89105)
[0x7f95d8006105] ->
/usr/lib64/dovecot/libdovecot.so.0(http_client_request_send_payload+0x1f)
[0x7f95d80063cf] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xd31d)
[0x7f95d6ad931d] ->
/usr/lib64/dovecot/lib20_fts_plugin.so(fts_parser_more+0x1a)
[0x7f95d6ad83ca] ->
/usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x761)
[0x7f95d6ad6401] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x114ca)
[0x7f95d6add4ca] ->
/usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x2a)
[0x7f95d835ab4a] -> doveadm(+0x31e75) [0x55e7c1052e75] ->
doveadm(+0x321fb) [0x55e7c10531fb] -> doveadm(+0x2c321) [0x55e7c104d321]
-> doveadm(+0x2c577) [0x55e7c104d577] ->
doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x1e8) [0x55e7c104ec38] ->
doveadm(doveadm_cmd_run_ver2+0x52e) [0x55e7c105fafe] ->
doveadm(doveadm_cmd_try_run_ver2+0x37) [0x55e7c105fb97] ->
doveadm(main+0x21a) [0x55e7c1062aca] ->
/lib64/libc.so.6(__libc_start_main+0x100) [0x7f95d7983d20] ->
doveadm(+0x1c479) [0x55e7c103d479]

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://dovecot.org/pipermail/dovecot/attachments/20200306/a38a8b71/attachment.sig>