Re: Re: Re: Dovecot+Solr+Tika segmentation fault
A fix was posted on this list which seems to avoid segfaults when using solr+tika. I reposted it not long ago in response to another question. Before exploring the other segfaults it would be worthwhile getting that fix. If other segfaults continue then I suggest posting the logging from Dovecot
Scheduled for 2.3.13.
I applied the fix and will observe it for the next couple of days.
Unfortunately, the issue still persists even with this patch. The log message is slightly different, though, regarding disconnection:
Dec 03 17:10:49 server.name tika[988]: INFO tika/ (application/pdf) Dec 03 17:10:49 server.name tika[988]: INFO tika/ (image/png) Dec 03 17:10:49 server.name kernel: indexer-worker[15805]: segfault at 0 ip 00007f140483db64 sp 00007ffc04fa7960 error 4 in libdovecot.so.0.0.0[7f14047f3000+db000] Dec 03 17:10:49 server.name kernel: Code: 89 f5 48 8d b8 f0 01 00 00 48 8b 52 08 4c 8b 62 48 e8 90 89 fb ff 48 8b 44 24 08 be 00 20 00 00 4c 89 ef 48 8b 80 10 01 00 00 <4c> 8b 38 e8 64 b2 fb ff 48 8b 44 24 0 Dec 03 17:10:49 server.name dovecot[988]: Dec 03 16:10:49 indexer: Error: Indexer worker disconnected, discarding 1 requests for username_stripped Dec 03 17:10:49 server.name dovecot[988]: Dec 03 16:10:49 indexer-worker(username_stripped)<1952>Ed3ek5G1BMQgARa4EL8DAKGMINRlZjjS:c+gACAkOyV+gBwAAhFs1yA: Fatal: master: service(indexer-worker): child 1952 killed
It seems that there is something else going wrong in this version.
Best, Manuel
On 4. Dec 2020, at 9.29, Manuel Josupeit-Walter dj_delinquent@web.de wrote:
A fix was posted on this list which seems to avoid segfaults when using solr+tika. I reposted it not long ago in response to another question. Before exploring the other segfaults it would be worthwhile getting that fix. If other segfaults continue then I suggest posting the logging from Dovecot
Scheduled for 2.3.13.
I applied the fix and will observe it for the next couple of days.
Unfortunately, the issue still persists even with this patch. The log message is slightly different, though, regarding disconnection:
Dec 03 17:10:49 server.name tika[988]: INFO tika/ (application/pdf) Dec 03 17:10:49 server.name tika[988]: INFO tika/ (image/png) Dec 03 17:10:49 server.name kernel: indexer-worker[15805]: segfault at 0 ip 00007f140483db64 sp 00007ffc04fa7960 error 4 in libdovecot.so.0.0.0[7f14047f3000+db000] Dec 03 17:10:49 server.name kernel: Code: 89 f5 48 8d b8 f0 01 00 00 48 8b 52 08 4c 8b 62 48 e8 90 89 fb ff 48 8b 44 24 08 be 00 20 00 00 4c 89 ef 48 8b 80 10 01 00 00 <4c> 8b 38 e8 64 b2 fb ff 48 8b 44 24 0 Dec 03 17:10:49 server.name dovecot[988]: Dec 03 16:10:49 indexer: Error: Indexer worker disconnected, discarding 1 requests for username_stripped Dec 03 17:10:49 server.name dovecot[988]: Dec 03 16:10:49 indexer-worker(username_stripped)<1952>Ed3ek5G1BMQgARa4EL8DAKGMINRlZjjS:c+gACAkOyV+gBwAAhFs1yA: Fatal: master: service(indexer-worker): child 1952 killed
It seems that there is something else going wrong in this version.
This is a segfault, while the fix was for an assert crash, so it's a different issue. Can you get a gdb backtrace of the crash? Otherwise we can't debug it further. See https://dovecot.org/bugreport-mail#coredumps
Scheduled for 2.3.13.
I applied the fix and will observe it for the next couple of days.
Unfortunately, the issue still persists even with this patch. The log message is slightly different, though, regarding disconnection.
Dec 03 17:10:49 server.name kernel: indexer-worker[15805]: segfault at 0 ip 00007f140483db64 sp 00007ffc04fa7960 error 4 in libdovecot.so.0.0.0[7f14047f3000+db000]
It seems that there is something else going wrong in this version.
This is a segfault, while the fix was for an assert crash, so it's a different issue. Can you get a gdb backtrace of the crash? Otherwise we can't debug it further. See https://dovecot.org/bugreport-mail#coredumps[https://dovecot.org/bugreport-mail#coredumps]
I will try to. However, it will take some time, though, because I need to make it work in a Docker container (sysctl and read-only filesystem). I will get back to you will details as soon as possible.
On 04/12/2020 09:29, Manuel Josupeit-Walter wrote:
A fix was posted on this list which seems to avoid segfaults when using solr+tika. I reposted it not long ago in response to another question. Before exploring the other segfaults it would be worthwhile getting that fix. If other segfaults continue then I suggest posting the logging from Dovecot https://github.com/dovecot/core/commit/043d111cea78533cf22d3bfe0872c8376d842f56[https://github.com/dovecot/core/commit/043d111cea78533cf22d3bfe0872c8376d842f56] (DOV-4090)
Scheduled for 2.3.13. I applied the fix and will observe it for the next couple of days. Unfortunately, the issue still persists even with this patch. The log message is slightly different, though, regarding disconnection:
Dec 03 17:10:49 server.name tika[988]: INFO tika/ (application/pdf) Dec 03 17:10:49 server.name tika[988]: INFO tika/ (image/png) Dec 03 17:10:49 server.name kernel: indexer-worker[15805]: segfault at 0 ip 00007f140483db64 sp 00007ffc04fa7960 error 4 in libdovecot.so.0.0.0[7f14047f3000+db000] Dec 03 17:10:49 server.name kernel: Code: 89 f5 48 8d b8 f0 01 00 00 48 8b 52 08 4c 8b 62 48 e8 90 89 fb ff 48 8b 44 24 08 be 00 20 00 00 4c 89 ef 48 8b 80 10 01 00 00 <4c> 8b 38 e8 64 b2 fb ff 48 8b 44 24 0 Dec 03 17:10:49 server.name dovecot[988]: Dec 03 16:10:49 indexer: Error: Indexer worker disconnected, discarding 1 requests for username_stripped Dec 03 17:10:49 server.name dovecot[988]: Dec 03 16:10:49 indexer-worker(username_stripped)<1952>Ed3ek5G1BMQgARa4EL8DAKGMINRlZjjS:c+gACAkOyV+gBwAAhFs1yA: Fatal: master: service(indexer-worker): child 1952 killed
It seems that there is something else going wrong in this version.
Best, Manuel
Also just to clarify the above mentioned patch solves an assert which happens when using solr. There is another patch which solves an assert when using solr+tika together: that one is not mine but I'll cite it here to make it easier to identify it. When running solr+tika together I found I needed both patches. I first saw this in an email to this list from Josef 'Jeff' Sipek on 19/08/2020. diff --git a/src/plugins/fts-solr/solr-connection.c b/src/plugins/fts-solr/solr-connection.c index ae720b5e2870a852c1b6c440939e3c7c0fa72b5c..9d364f93e2cd1b716b9ab61bd39656a6c5b1ea04 100644 --- a/src/plugins/fts-solr/solr-connection.c +++ b/src/plugins/fts-solr/solr-connection.c @@ -103,7 +103,7 @@ int solr_connection_init(const struct fts_solr_settings *solr_set, http_set.ssl = ssl_client_set; http_set.debug = solr_set->debug; http_set.rawlog_dir = solr_set->rawlog_dir; - solr_http_client = http_client_init(&http_set); + solr_http_client = http_client_init_private(&http_set); } *conn_r = conn; diff --git a/src/plugins/fts/fts-parser-tika.c b/src/plugins/fts/fts-parser-tika.c index a4b8b5c3034f57e22e77caa759c090da6b62f8ba..b8b57a350b9a710d101ac7ccbcc14560d415d905 100644 --- a/src/plugins/fts/fts-parser-tika.c +++ b/src/plugins/fts/fts-parser-tika.c @@ -77,7 +77,7 @@ tika_get_http_client_url(struct mail_user *user, struct http_url **http_url_r) http_set.request_timeout_msecs = 60*1000; http_set.ssl = &ssl_set; http_set.debug = user->mail_debug; - tika_http_client = http_client_init(&http_set); + tika_http_client = http_client_init_private(&http_set); } *http_url_r = tuser->http_url; return 0; John
participants (3)
-
John Fawcett
-
Manuel Josupeit-Walter
-
Timo Sirainen