Problem with tika

[Arjen Heidinga]

Greetings,

Is the anyone who could please help me shed light on this?

Arjen

On 06-03-2020 09:52, Arjen Heidinga wrote:
> Hello all,
>
> For some time now we've bin using solr as a search engine (working
> great). I have added tika for searching inside documents, however it
> keeps crashing when indexing. When just indexing mails that arrive works
> good, but a reindex on all mail constantly crashes with below stacktrace.
>
> When I observe the packets with wireshark I see a HTTP-flow going to
> tika and suddenly, midway a document dovecot (or the server) sends a
> RST. Sometimes after thousands of documents, sometimes after a few.
> Usually after a few hundred.
>
> It appears there are less errors using http than https.
>
>
>
> Relevant config:
>
> OS: CentOS6, fully updated
>
> plugin {
>   fts = solr
>   batch_size = 1
>   fts_solr =
> url=https://username:password@solr-01.vevida.net:443/solr/dovecot/
>   #fts_tika = https://solr-01.vevida.net:443/tika/
>   batch_size = 1000
>   fts_autoindex=yes
>   soft_commit=no
> }
>
> # dovecot --version
> 2.3.9.3 (9f41b88fa)
>
> # Configure options:
>
>     --docdir=%{_docdir}/dovecot  \
>     --disable-static             \
>     --with-nss                   \
>     --with-shadow                \
>     --with-pam                   \
>     --with-gssapi=plugin         \
>     --with-ldap=plugin           \
>     --with-sql=plugin            \
>     --with-pgsql                 \
>     --with-sqlite                \
>     --with-zlib                  \
>     --with-bzlib                 \
>     --with-lzma                  \
>     --with-libcap                \
>     --with-ssl=openssl           \
>     --with-ssldir=%{ssldir}      \
>     --with-solr                  \
>     --with-docs
>
> # It is compiled agains the latest openssl
>
> # Tika and Solr: Both latest versions.
>
> #Stacktrace:
>
> doveadm(info at samenmetrenske.nl): Info: Sent: Caching mails seq=1..161
>
> doveadm(info at samenmetrenske.nl): Panic: file http-client-request.c: line
> 1173 (http_client_request_send_more): assertion failed:
> (req->payload_input != NULL)
>
> doveadm(info at xxxxxxxxxxxxxxxxxxx.x): Error: Raw backtrace:
> /usr/lib64/dovecot/libdovecot.so.0(backtrace_append+0x2f)
> [0x7f95d805acbf] ->
> /usr/lib64/dovecot/libdovecot.so.0(backtrace_get+0x26) [0x7f95d805add6]
> -> /usr/lib64/dovecot/libdovecot.so.0(+0xe90ba) [0x7f95d80660ba] ->
> /usr/lib64/dovecot/libdovecot.so.0(+0xe9161) [0x7f95d8066161] ->
> /usr/lib64/dovecot/libdovecot.so.0(+0x41158) [0x7f95d7fbe158] ->
> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_send_more+0x424)
> [0x7f95d8005094] ->
> /usr/lib64/dovecot/libdovecot.so.0(http_client_connection_output+0x11a)
> [0x7f95d800a24a] ->
> /usr/lib64/dovecot/libssl_iostream_openssl.so(+0x8f6a) [0x7f95d57a2f6a]
> -> /usr/lib64/dovecot/libdovecot.so.0(+0x114483) [0x7f95d8091483] ->
> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x61)
> [0x7f95d807e581] ->
> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xdc)
> [0x7f95d808076c] ->
> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x5c)
> [0x7f95d807e67c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38)
> [0x7f95d807e8c8] -> /usr/lib64/dovecot/libdovecot.so.0(+0x89105)
> [0x7f95d8006105] ->
> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_send_payload+0x1f)
> [0x7f95d80063cf] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xd31d)
> [0x7f95d6ad931d] ->
> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_parser_more+0x1a)
> [0x7f95d6ad83ca] ->
> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x761)
> [0x7f95d6ad6401] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x114ca)
> [0x7f95d6add4ca] ->
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x2a)
> [0x7f95d835ab4a] -> doveadm(+0x31e75) [0x55e7c1052e75] ->
> doveadm(+0x321fb) [0x55e7c10531fb] -> doveadm(+0x2c321) [0x55e7c104d321]
> -> doveadm(+0x2c577) [0x55e7c104d577] ->
> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x1e8) [0x55e7c104ec38] ->
> doveadm(doveadm_cmd_run_ver2+0x52e) [0x55e7c105fafe] ->
> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x55e7c105fb97] ->
> doveadm(main+0x21a) [0x55e7c1062aca] ->
> /lib64/libc.so.6(__libc_start_main+0x100) [0x7f95d7983d20] ->
> doveadm(+0x1c479) [0x55e7c103d479]
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://dovecot.org/pipermail/dovecot/attachments/20200519/addea463/attachment.sig>