fts_solr crashs

[M. Koehler]

Hi Aki,

Am 27.06.2016 um 11:14 schrieb Aki Tuomi:
> On 27.06.2016 12:11, M. Koehler wrote:
>> I´ve set up in dovecot 2.2.24-1~auto+49 (from dovecot repo) fts_solr
>> and fts_tika - jetti8 (from Debian Jessie) and latest tika-server
>> running on a seperate machine. But if I want to rescan all messages
>> for reindexing for instance all attachments with "doveadm -v index -u
>> user at domain.tld INBOX" with 3137 mail in the INBOX  it counts and then
>> by 2900 mails the doveadm crashes with the following error message:
>>
>> doveadm(user at domain.tld): Info: INBOX: Caching mails seq=1..3137
>> 2900/3137doveadm(user at domain.tld): Error: fts_solr: Indexing failed:
>> 500 Java heap space  java.lang.OutOfMemoryError: Java heap space
>> doveadm(user at domain.tld): Panic: file http-client-request.c: line 769
>> (http_client_request_send_payload): assertion failed: (ret == 0)
>> doveadm(user at domain.tld): Error: Raw backtrace:
>> /usr/lib/dovecot/libdovecot.so.0(+0x8dc1e) [0x7f269375ec1e] ->
>> /usr/lib/dovecot/libdovecot.so.0(+0x8dc98) [0x7f269375ec98] ->
>> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f26936fd02e] ->
>> /usr/lib/dovecot/libdovecot.so.0(http_client_request_send_payload+0xc8) [0x7f2693716b88]
>> ->
>> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(solr_connection_post_more+0x49)
>> [0x7f2691c538b9] ->
>> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(+0x361e)
>> [0x7f2691c4f61e] ->
>> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x9a17) [0x7f2692ad5a17]
>> -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x9e20)
>> [0x7f2692ad5e20] ->
>> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x999)
>> [0x7f2692ad6809] ->
>> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0x1056e)
>> [0x7f2692adc56e] ->
>> /usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x29)
>> [0x7f2693a1f7b9] -> doveadm(+0x2e947) [0x7f26943be947] ->
>> doveadm(+0x28daf) [0x7f26943b8daf] -> doveadm(+0x2989d)
>> [0x7f26943b989d] ->
>> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x278) [0x7f26943ba788]
>> -> doveadm(doveadm_cmd_run_ver2+0x560) [0x7f26943c8c70] ->
>> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x7f26943c8cc7] ->
>> doveadm(main+0x1df) [0x7f26943aa33f] ->
>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)
>> [0x7f2693347b45] -> doveadm(+0x1a716) [0x7f26943aa716]
>>
>> I don´t see any shortages on the seperate solr machine. What could I
>> do? How do I get more details to locate the problem?!
> Well. Your server reports:
>
> 2900/3137doveadm(user at domain.tld): Error: fts_solr: Indexing failed: 500
> Java heap space  java.lang.OutOfMemoryError: Java heap space
>
> So I guess this is your problem?
>
> The dovecot crash is due to assertion failure, which happens because the
> request failed.
Well, that´s right - I´ve checked it with the solr server but I can´t 
imagine what the problem is. I increase the java heap memory from 256MB 
to 2GB but it crashes already by 2900 messages. Will Dovecot forward 
message by message to solr server  or "as a block"? And how could I find 
out which message maybe caused the problem? I´ve checked the INBOX and 
the biggest mail got 48MB as an attachment.

Best regards,
Michael