<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">On 05/02/2021 15:04, Dean Carpenter
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:258395056c628017e242b95e312919a7@areyes.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<p>Is there anything I can do here ? This makes Tika unusable :(
That really sucks because we have a *lot* of attachments.</p>
<p>Thanks -</p>
<p>On 2021-02-03 2:07 pm, Dean Carpenter wrote:</p>
<blockquote type="cite" style="padding-left:5px;
border-left:#1010ff 2px solid; margin-left:5px"><!-- html ignored --><!-- head ignored --><!-- meta ignored -->
<p>Just noticed this in the logs</p>
<blockquote style="padding-left: 5px; border-left: #1010ff 2px
solid; margin-left: 5px;">
<p>doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client[2]: request
[Req2: PUT <a class="moz-txt-link-freetext" href="http://localhost/tika/">http://localhost/tika/</a>]: Submitted (requests
left=1)<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client[2]: request
[Req2: PUT <a class="moz-txt-link-freetext" href="http://localhost/tika/">http://localhost/tika/</a>]: Waiting for request to
finish</p>
</blockquote>
<p>It doesn't have the port on the URL ...</p>
<p> </p>
<p>On 2021-02-03 1:59 pm, Dean Carpenter wrote:</p>
<blockquote style="padding-left: 5px; border-left: #1010ff 2px
solid; margin-left: 5px;"><!-- node type 8 --><!-- node type 8 --><!-- node type 8 -->
<p>Getting panic in http-client-request.c: line 1240 during
indexing</p>
<p>I'm testing the install/setup of dovecot on an Ubuntu Focal
20.04 system, using the dovecot repo. The packages that are
installed are</p>
<p>harryb@dove1:~$ dpkg -l dovecot\* solr\* jetty\* tesser\* |
fgrep ii<br>
ii dovecot-core 2:2.3.13-2+ubuntu20.04 amd64 secure
POP3/IMAP server - core files<br>
ii dovecot-imapd 2:2.3.13-2+ubuntu20.04 amd64 secure
POP3/IMAP server - IMAP daemon<br>
ii dovecot-lmtpd 2:2.3.13-2+ubuntu20.04 amd64 secure
POP3/IMAP server - LMTP server<br>
ii dovecot-managesieved 2:2.3.13-2+ubuntu20.04 amd64 secure
POP3/IMAP server - ManageSieve server<br>
ii dovecot-mysql 2:2.3.13-2+ubuntu20.04 amd64 secure
POP3/IMAP server - MySQL support<br>
ii dovecot-sieve 2:2.3.13-2+ubuntu20.04 amd64 secure
POP3/IMAP server - Sieve filters support<br>
ii dovecot-solr 2:2.3.13-2+ubuntu20.04 amd64 secure
POP3/IMAP server - Solr support<br>
ii jetty9 9.4.26-1 all Java servlet engine and webserver<br>
ii solr-common 3.6.2+dfsg-22 all Enterprise search server
based on Lucene3 - common files<br>
ii solr-jetty 3.6.2+dfsg-22 all Enterprise search server
based on Lucene3 - Jetty integration<br>
ii tesseract-ocr 4.1.1-2build2 amd64 Tesseract command line
OCR tool<br>
ii tesseract-ocr-eng 1:4.00~git30-7274cfa-1 all
tesseract-ocr language files for English<br>
ii tesseract-ocr-osd 1:4.00~git30-7274cfa-1 all
tesseract-ocr language files for script and orientation</p>
<p>Apache Tika v1.25 is also installed</p>
<p>harryb@dove1:~$ l /usr/local/bin/tika*<br>
-rw-r--r-- 1 root root 79337717 Feb 3 12:40
/usr/local/bin/tika-server-1.25.jar</p>
<p>and running pretty standard-ish</p>
<p>/usr/bin/java -jar /usr/local/bin/tika-server-1.25.jar
--port 9998 --host localhost -enableUnsecureFeatures
-enableFileUrl --log info</p>
<p>I use mbsync to pull in sample mailboxes for testing</p>
<p>The solr conf in /etc/dovecot/conf.d/ is as follows<br>
plugin {<br>
fts = solr<br>
fts_solr = url=<a class="moz-txt-link-freetext" href="http://localhost:8693/solr/">http://localhost:8693/solr/</a><br>
fts_autoindex = yes<br>
fts_filters = lowercase snowball stopwords<br>
fts_tokenizers = generic email-address<br>
fts_tokenizer_generic = algorithm=simple<br>
fts_autoindex_exclude = \Junk<br>
fts_autoindex_exclude2 = \Trash<br>
fts_autoindex_exclude3 = \Spam<br>
}<br>
plugin {<br>
fts_tika = <a class="moz-txt-link-freetext" href="http://localhost:9998/tika/">http://localhost:9998/tika/</a><br>
}</p>
<p>Unfortunately indexing is panic'ing. Whether I index a test
folder manually, or just looking in /var/log/dovecot.log</p>
<p>I'll see the following very consistently :</p>
<p>harryb@dove1:~$ sudo doveadm -D index -u <a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>
Receipts</p>
<p>Debug: Loading modules from directory:
/usr/lib/dovecot/modules<br>
Debug: Module loaded:
/usr/lib/dovecot/modules/lib10_mail_crypt_plugin.so<br>
Debug: Module loaded:
/usr/lib/dovecot/modules/lib15_notify_plugin.so<br>
Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_fts_plugin.so<br>
Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_replication_plugin.so<br>
Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_zlib_plugin.so<br>
Debug: Module loaded:
/usr/lib/dovecot/modules/lib21_fts_solr_plugin.so<br>
Debug: Loading modules from directory:
/usr/lib/dovecot/modules/doveadm<br>
Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so<br>
Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: auth-master: passdb
lookup(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Started passdb lookup<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: auth-master: conn
unix:/var/run/dovecot/auth-userdb: Connecting<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: auth-master: conn
unix:/var/run/dovecot/auth-userdb (pid=165148,uid=0): Client
connected (fd=9)<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: auth-master: passdb
lookup(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): auth PASS input:
<a class="moz-txt-link-abbreviated" href="mailto:user=harryb@example.com">user=harryb@example.com</a><br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: auth-master: passdb
lookup(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Finished passdb lookup
(<a class="moz-txt-link-abbreviated" href="mailto:user=harryb@example.com">user=harryb@example.com</a> )<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>)<180749><>: Debug:
auth-master: userdb lookup(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Started
userdb lookup<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>)<180749><>: Debug:
auth-master: userdb lookup(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): auth USER
input: <a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a> home=/var/mail/example.com/harryb
uid=1001 gid=1001<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>)<180749><>: Debug:
auth-master: userdb lookup(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Finished
userdb lookup (<a class="moz-txt-link-abbreviated" href="mailto:username=harryb@example.com">username=harryb@example.com</a>
home=/var/mail/example.com/harryb uid=1001 gid=1001)<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: Effective uid=1001,
gid=1001, home=/var/mail/example.com/harryb<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: Namespace inbox:
type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes,
subscriptions=yes
location=maildir:~/Maildir:LAYOUT=fs:INBOX=~/Maildir/INBOX<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: fs:
root=/var/mail/example.com/harryb/Maildir, index=,
indexpvt=, control=,
inbox=/var/mail/example.com/harryb/Maildir/INBOX, alt=<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: Mailbox Receipts:
Mailbox opened because: index<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Info: Receipts: Caching mails
seq=1..72<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: Mailbox Receipts: UID 1:
Opened mail because: prefetch<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: Mailbox Receipts: UID 1:
Opened mail because: fts indexing<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client: peer
[::1]:9998 (shared): Peer created<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client: peer
[::1]:9998: Peer pool created<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client[2]: peer
[::1]:9998: Peer created<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client[2]: queue
<a class="moz-txt-link-freetext" href="http://localhost:9998">http://localhost:9998</a>: Setting up connection to [::1]:9998
(1 requests pending)<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client[2]: peer
[::1]:9998: Linked queue <a class="moz-txt-link-freetext" href="http://localhost:9998">http://localhost:9998</a> (1 queues
linked)<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client[2]: queue
<a class="moz-txt-link-freetext" href="http://localhost:9998">http://localhost:9998</a>: Started new connection to [::1]:9998<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client[2]: request
[Req2: PUT <a class="moz-txt-link-freetext" href="http://localhost/tika/">http://localhost/tika/</a>]: Submitted (requests
left=1)<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client[2]: request
[Req2: PUT <a class="moz-txt-link-freetext" href="http://localhost/tika/">http://localhost/tika/</a>]: Waiting for request to
finish<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client[2]: peer
[::1]:9998: Creating 1 new connections to handle requests
(already 0 usable, connecting to 0, closing 0)<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client[2]: peer
[::1]:9998: Making new connection 1 of 1 (0 connections
exist, 0 pending)<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client: conn
[::1]:9998 [2]: Connecting<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client: conn
[::1]:9998 [2]: Waiting for connect (fd=20) to finish for
max 0 msecs<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Debug: http-client: conn
[::1]:9998 [2]: HTTP connection created (1 parallel
connections exist)<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Panic: file
http-client-request.c: line 1240
(http_client_request_send_more): assertion failed:
(req->payload_input != NULL)<br>
doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x41)
[0x7ffb60d697a1] ->
/usr/lib/dovecot/libdovecot.so.0(backtrace_get+0x22)
[0x7ffb60d698c2] ->
/usr/lib/dovecot/libdovecot.so.0(+0x104b3b) [0x7ffb60d75b3b]
-> /usr/lib/dovecot/libdovecot.so.0(+0x104b77)
[0x7ffb60d75b77] ->
/usr/lib/dovecot/libdovecot.so.0(+0x5b4d5) [0x7ffb60ccc4d5]
-> /usr/lib/dovecot/libdovecot.so.0(+0x5291e)
[0x7ffb60cc391e] ->
/usr/lib/dovecot/libdovecot.so.0(http_client_connection_output+0xf2)
[0x7ffb60d19f82] ->
/usr/lib/dovecot/libdovecot.so.0(+0x12b1c5) [0x7ffb60d9c1c5]
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x6d)
[0x7ffb60d8bd4d] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x145)
[0x7ffb60d8d395] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x54)
[0x7ffb60d8bdf4] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x48)
[0x7ffb60d8bf68] ->
/usr/lib/dovecot/libdovecot.so.0(+0xa482d) [0x7ffb60d1582d]
->
/usr/lib/dovecot/libdovecot.so.0(http_client_request_send_payload+0x34)
[0x7ffb60d159c4] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x1068d)
[0x7ffb604ef68d] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_parser_more+0x2b)
[0x7ffb604ee4ab] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xd77f)
[0x7ffb604ec77f] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x52)
[0x7ffb604ecec2] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x13573)
[0x7ffb604f2573] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x32)
[0x7ffb60e92ed2] -> doveadm(+0x3982f) [0x55d3519ed82f]
-> doveadm(+0x33e35) [0x55d3519e7e35] ->
doveadm(+0x34a16) [0x55d3519e8a16] ->
doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x225)
[0x55d3519e97b5] -> doveadm(doveadm_cmd_run_ver2+0x500)
[0x55d3519f9e50] ->
doveadm(doveadm_cmd_try_run_ver2+0x3e) [0x55d3519f9eae]
-> doveadm(main+0x1d4) [0x55d3519d8dc4] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)
[0x7ffb608fd0b3] -> doveadm(_start+0x2e) [0x55d3519d92ae]<br>
Aborted</p>
<p>The key line above is</p>
<p>doveadm(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>): Panic: file
http-client-request.c: line 1240
(http_client_request_send_more): assertion failed:
(req->payload_input != NULL)</p>
<p>It's the same each time. In /var/log/syslog there are
usually a couple of extra lines</p>
<p>Feb 03 13:25:17 indexer: Error: Indexer worker
disconnected, discarding 1 requests for <a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a> <br>
Feb 03 13:25:17
indexer-worker(<a class="moz-txt-link-abbreviated" href="mailto:harryb@example.com">harryb@example.com</a>)<202106><p+xgBonqGmAIFQMATj4HXg:IvE7Eo3qGmB6FQMATj4HXg>:
Fatal: master: service(indexer-worker): child 202106 killed
with signal 6 (core dumps disabled -
<a class="moz-txt-link-freetext" href="https://dovecot.org/bugreport.html#coredumps">https://dovecot.org/bugreport.html#coredumps</a>)</p>
<div>
<pre>--
Dean Carpenter
deano is at areyes dot com
203 six oh four 6644</pre>
</div>
</blockquote>
</blockquote>
</blockquote>
<p>Hi Dean</p>
<p>I guess you need the patch that was posted on this list by Jeff
Sipek on 19.08.20 17:37</p>
<p>John<br>
</p>
</body>
</html>