indexer-worker panics with latest mercurial
Hi.
I am seeing the following in my logs. Happy to downgrade to an earlier version if you think this might be the problem, but dovecot.org is extremely slow (and has been for months) so I found it easier just to pull the latest from mercurial.
Thanks.
Laeeth
Mar 12 20:48:39 indexer: Error: Indexer worker disconnected, discarding 1 reques ts for laeeth@laeeth.com Mar 12 20:48:39 indexer-worker(laeeth@laeeth.com): Fatal: master: service(indexe r-worker): child 24003 killed with signal 6 (core dumps disabled) Mar 12 20:49:01 dsync-local(laeeth@laeeth.com): Error: Couldn't lock /home/mail/ laeeth_laeeth_com/.dovecot-sync.lock: Timed out after 30 seconds Mar 12 20:49:16 indexer-worker(laeeth@laeeth.com): Error: fts_tika: PUT http://l ocalhost:9997/tika/ failed: 500 Server Error Mar 12 20:49:17 indexer-worker(rosie@kaleidicassociates.com): Warning: I/O leak: 0x7fc47d60fcf0 (line 127, fd 25) Mar 12 20:49:17 indexer-worker(rosie@kaleidicassociates.com): Panic: file ioloop .c: line 39 (io_add_file): assertion failed: (callback != NULL) Mar 12 20:49:17 indexer-worker(rosie@kaleidicassociates.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7fc47d63fc40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7fc47d63fd1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7fc47d5f02e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x87bd0) [0x7fc47d650bd0] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7fc47d65183e] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7fc47d63db14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7fc47d614448] -> /usr/lib64/dovecot/libdovecot.so.0(+0x43876) [0x7fc47d60c876] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x29) [0x7fc47c733159] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x3637) [0x7fc47c72f637] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x6d83) [0x7fc47cf5dd83] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x614) [0x7fc47cf5e3c4] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xc14e) [0x7fc47cf6314e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x19) [0x7fc47d8ef429] -> dovecot/indexer-worker() [0x40237c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fc47d65155c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbb) [0x7fc47d65249b] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7fc47d6515c9] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fc47d651648] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fc47d5f5703] -> dovecot/indexer-worker(main+0xcb) [0x401e7b] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x32c6021d65] -> dovecot/indexer-worker() [0x401f25] Mar 12 20:49:17 indexer: Error: Indexer worker disconnected, discarding 15 reque sts for rosie@kaleidicassociates.com Mar 12 20:49:17 indexer-worker(rosie@kaleidicassociates.com): Fatal: master: service(indexer-worker): child 24022 killed with signal 6 (core dumps disabled) Mar 12 20:49:17 indexer-worker(laeeth@kaleidicassociates.com): Warning: I/O leak: 0x7f22c4d4dcf0 (line 127, fd 23) Mar 12 20:49:17 indexer-worker(laeeth@kaleidicassociates.com): Panic: file ioloop.c: line 829 (io_loop_move_io): assertion failed: ((old_io->condition & IO_NOTIFY) == 0) Mar 12 20:49:17 indexer-worker(laeeth@kaleidicassociates.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7f22c4d7dc40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7f22c4d7dd1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f22c4d2e2e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x888ba) [0x7f22c4d8f8ba] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7f22c4d7bb14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7f22c4d52448] -> /usr/lib64/dovecot/libdovecot.so.0(+0x43876) [0x7f22c4d4a876] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(solr_connection_post_more+0x29) [0x7f22c3e71159] -> /usr/lib64/dovecot/lib21_fts_solr_plugin.so(+0x3637) [0x7f22c3e6d637] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0x6d83) [0x7f22c469bd83] -> /usr/lib64/dovecot/lib20_fts_plugin.so(fts_build_mail+0x123) [0x7f22c469bed3] -> /usr/lib64/dovecot/lib20_fts_plugin.so(+0xc14e) [0x7f22c46a114e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_precache+0x19) [0x7f22c502d429] -> dovecot/indexer-worker() [0x40237c] -> /usr/lib64/dovecot/libdovecot.so.0(io _loop_call_io+0x4c) [0x7f22c4d8f55c] -> /usr/lib64/dovecot/libdovecot.s o.0(io_loop_handler_run_internal+0xbb) [0x7f22c4d9049b] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7f22c4d8f5c9] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f22c4d8f648] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f22c4d33703] -> dovecot/indexer-worker(main+0xcb) [0x401e7b] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x32c6021d65] -> dovecot/indexer-worker() [0x401f25] Mar 12 20:49:17 indexer: Error: Indexer worker disconnected, discarding 6 requests for laeeth@kaleidicassociates.com Mar 12 20:49:17 indexer-worker(laeeth@kaleidicassociates.com): Fatal: master: service(indexer-worker): child 24020 killed with signal 6 (core dumps disabled) Mar 12 20:49:26 indexer-worker(laeeth@laeeth.com): Warning: I/O leak: 0x7f0d5aa00cf0 (line 127, fd 23) Mar 12 20:49:26 indexer-worker(laeeth@laeeth.com): Panic: file ioloop-iolist.c: line 22: unreached Mar 12 20:49:26 indexer-worker(laeeth@laeeth.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x76c40) [0x7f0d5aa30c40] -> /usr/lib64/dovecot/libdovecot.so.0(+0x76d1e) [0x7f0d5aa30d1e] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0d5a9e12e5] -> /usr/lib64/dovecot/libdovecot.so.0(+0x88a0d) [0x7f0d5aa42a0d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x3b) [0x7f0d5aa4319b] -> /usr/lib64/dovecot/libdovecot.so.0(+0x87b61) [0x7f0d5aa41b61] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_move_io+0x3e) [0x7f0d5aa4283e] -> /usr/lib64/dovecot/libdovecot.so.0(connection_switch_ioloop+0x14) [0x7f0d5aa2eb14] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_switch_ioloop+0x28) [0x7f0d5aa0
On 12 Mar 2015, at 22:55, Laeeth Isharc laeeth@laeeth.com wrote:
Hi.
I am seeing the following in my logs. Happy to downgrade to an earlier version if you think this might be the problem, but dovecot.org is extremely slow (and has been for months) so I found it easier just to pull the latest from mercurial.
dovecot.org isn't slow as far as I know, although I've noticed some network connectivity problems to some places .. It's in Azure, so I guess somehow related to that..
Mar 12 20:48:39 indexer: Error: Indexer worker disconnected, discarding 1 reques ts for laeeth@laeeth.com Mar 12 20:48:39 indexer-worker(laeeth@laeeth.com): Fatal: master: service(indexe r-worker): child 24003 killed with signal 6 (core dumps disabled) Mar 12 20:49:01 dsync-local(laeeth@laeeth.com): Error: Couldn't lock /home/mail/ laeeth_laeeth_com/.dovecot-sync.lock: Timed out after 30 seconds Mar 12 20:49:16 indexer-worker(laeeth@laeeth.com): Error: fts_tika: PUT http://l ocalhost:9997/tika/ failed: 500 Server Error Mar 12 20:49:17 indexer-worker(rosie@kaleidicassociates.com): Warning: I/O leak: 0x7fc47d60fcf0 (line 127, fd 25) Mar 12 20:49:17 indexer-worker(rosie@kaleidicassociates.com): Panic: file ioloop .c: line 39 (io_add_file): assertion failed: (callback != NULL)
I tried this a few times, but I wasn't able to cause the crash. Since you're already pulling from mercurial, can you find which mercurial commit caused it to start breaking?
participants (2)
-
Laeeth Isharc
-
Timo Sirainen