Ok so that timeval makes no sense. We'll look into it.
Aki
On October 24, 2016 at 12:22 AM Larry Rosenman larryrtx@gmail.com wrote:
doveadm(mrm): Debug: http-client: conn 127.0.0.1:9998 [1]: Got 200 response for request [Req38: PUT http://localhost:9998/tika/] (took 296 ms + 8 ms in queue) doveadm(mrm): Panic: kevent(): Invalid argument
Program received signal SIGABRT, Aborted. 0x00000008014e6f7a in thr_kill () from /lib/libc.so.7 (gdb) fr 6 #6 0x00000008011a3e49 in io_loop_handler_run_internal (ioloop=0x801c214e0) at ioloop-kqueue.c:131 131 i_panic("kevent(): %m"); (gdb) p ts $1 = {tv_sec = 34389923520, tv_nsec = 140737488345872000} (gdb) p errno $2 = 22 (gdb) p ret $3 = -1 (gdb) p *ioloop $4 = {prev = 0x801c21080, cur_ctx = 0x0, io_files = 0x801c4f980, next_io_file = 0x0, timeouts = 0x801d17540, timeouts_new = {arr = { buffer = 0x801cd9700, element_size = 8}, v = 0x801cd9700, v_modifiable = 0x801cd9700}, handler_context = 0x801d17560, notify_handler_context = 0x0, max_fd_count = 0, time_moved_callback = 0x800d53bb0
, next_max_time = 1477257580, ioloop_wait_usecs = 27148, io_pending_count = 1, running = 1, iolooping = 1} (gdb) p* ctx $5 = {kq = 21, deleted_count = 0, events = {arr = {buffer = 0x801cd9740, element_size = 32}, v = 0x801cd9740, v_modifiable = 0x801cd9740}} (gdb) p *events $6 = {ident = 22, filter = -1, flags = 0, fflags = 0, data = 8, udata = 0x801c4f980} (gdb)thebighonker.lerctr.org ~ $ ps auxw|grep doveadm mrm 46965 0.0 0.2 108516 55264 0 I+ 4:19PM 0:02.28 gdb /usr/local/bin/doveadm (gdb7111) mrm 46985 0.0 0.0 81236 15432 0 TX 4:19PM 0:03.51 /usr/local/bin/doveadm -D -vvvvvvv index * ler 47221 0.0 0.0 18856 2360 1 S+ 4:21PM 0:00.00 grep doveadm thebighonker.lerctr.org ~ $ sudo lsof -p 46985 Password: COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME doveadm 46985 mrm cwd VDIR 22,2669215774 152 4 /home/mrm doveadm 46985 mrm rtd VDIR 19,766509061 28 4 / doveadm 46985 mrm txt VREG 119,3584295129 1714125 182952 /usr/local/bin/doveadm doveadm 46985 mrm txt VREG 19,766509061 132272 14382 /libexec/ld-elf.so.1 doveadm 46985 mrm txt VREG 22,2669215774 6920 10680 /home/mrm/mail/TRAVEL/.imap/hawaiian.airlines/dovecot.index.log doveadm 46985 mrm txt VREG 22,2669215774 7224 10716 /home/mrm/mail/TRAVEL/.imap/priceline/dovecot.index.log doveadm 46985 mrm txt VREG 22,2669215774 11080 10650 /home/mrm/mail/TRAVEL/.imap/alamo/dovecot.index.log doveadm 46985 mrm txt VREG 22,2669215774 2968 10679 /home/mrm/mail/TRAVEL/.imap/hawaiian.airlines/dovecot.index.cache doveadm 46985 mrm txt VREG 22,2669215774 3108 10715 /home/mrm/mail/TRAVEL/.imap/priceline/dovecot.index.cache doveadm 46985 mrm txt VREG 22,2669215774 6520 139902 /home/mrm/mail/.imap/Sent/dovecot.index.log doveadm 46985 mrm txt VREG 22,2669215774 9236 10648 /home/mrm/mail/TRAVEL/.imap/alamo/dovecot.index.cache doveadm 46985 mrm txt VREG 22,2669215774 174892 143343 /home/mrm/mail/.imap/Sent/dovecot.index.cache doveadm 46985 mrm txt VREG 22,2669215774 32656 143058 /home/mrm/mail/.imap/INBOX/dovecot.index.log doveadm 46985 mrm txt VREG 19,766509061 720 30627 /usr/share/i18n/csmapper/CP/CP1251%UCS.mps doveadm 46985 mrm txt VREG 19,766509061 720 30630 /usr/share/i18n/csmapper/CP/CP1252%UCS.mps doveadm 46985 mrm txt VREG 19,766509061 89576 6846 /lib/libz.so.6 doveadm 46985 mrm txt VREG 19,766509061 62008 5994 /lib/libcrypt.so.5 doveadm 46985 mrm txt VREG 119,3584295129 6725689 183611 /usr/local/lib/dovecot/libdovecot-storage.so.0.0.0 doveadm 46985 mrm txt VREG 119,3584295129 3162259 183615 /usr/local/lib/dovecot/libdovecot.so.0.0.0 doveadm 46985 mrm txt VREG 19,766509061 1649944 4782 /lib/libc.so.7 doveadm 46985 mrm txt VREG 119,3584295129 80142 183550 /usr/local/lib/dovecot/lib15_notify_plugin.so doveadm 46985 mrm txt VREG 119,3584295129 652615 183556 /usr/local/lib/dovecot/lib20_fts_plugin.so doveadm 46985 mrm txt VREG 119,3584295129 2730888 268825 /usr/local/lib/libicui18n.so.57.1 doveadm 46985 mrm txt VREG 119,3584295129 1753976 268849 /usr/local/lib/libicuuc.so.57.1 doveadm 46985 mrm txt VREG 119,3584295129 1704 268821 /usr/local/lib/libicudata.so.57.1 doveadm 46985 mrm txt VREG 19,766509061 102560 6745 /lib/libthr.so.3 doveadm 46985 mrm txt VREG 19,766509061 184712 5795 /lib/libm.so.5 doveadm 46985 mrm txt VREG 19,766509061 774000 5642 /usr/lib/libc++.so.1 doveadm 46985 mrm txt VREG 19,766509061 103304 5742 /lib/libcxxrt.so.1 doveadm 46985 mrm txt VREG 19,766509061 56344 7436 /lib/libgcc_s.so.1 doveadm 46985 mrm txt VREG 119,3584295129 349981 183782 /usr/local/lib/dovecot/lib21_fts_lucene_plugin.so doveadm 46985 mrm txt VREG 119,3584295129 1969384 113258 /usr/local/lib/libclucene-core.so.2.3.3.4 doveadm 46985 mrm txt VREG 119,3584295129 128992 113261 /usr/local/lib/libclucene-shared.so.2.3.3.4 doveadm 46985 mrm txt VREG 119,3584295129 143141 183578 /usr/local/lib/dovecot/lib90_stats_plugin.so doveadm 46985 mrm txt VREG 119,3584295129 37368 151926 /usr/local/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so doveadm 46985 mrm txt VREG 119,3584295129 693808 151924 /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0.0.0 doveadm 46985 mrm txt VREG 119,3584295129 146477 183599 /usr/local/lib/dovecot/libdovecot-lda.so.0.0.0 doveadm 46985 mrm txt VREG 119,3584295129 13823 183780 /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so doveadm 46985 mrm txt VREG 119,3584295129 88081 183527 /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so doveadm 46985 mrm txt VREG 19,766509061 8304 6330 /usr/lib/i18n/libiconv_std.so.4 doveadm 46985 mrm txt VREG 19,766509061 6744 6318 /usr/lib/i18n/libUTF8.so.4 doveadm 46985 mrm txt VREG 19,766509061 4384 6336 /usr/lib/i18n/libmapper_none.so.4 doveadm 46985 mrm txt VREG 19,766509061 7584 6345 /usr/lib/i18n/libmapper_std.so.4 doveadm 46985 mrm 0u VCHR 0,188 0t390889 188 /dev/pts/0 doveadm 46985 mrm 1u VCHR 0,188 0t390889 188 /dev/pts/0 doveadm 46985 mrm 2u VCHR 0,188 0t390889 188 /dev/pts/0 doveadm 46985 mrm 3u PIPE 0xfffff806fdf505d0 16384 ->0xfffff806fdf50730 doveadm 46985 mrm 4u PIPE 0xfffff806fdf50730 0 ->0xfffff806fdf505d0 doveadm 46985 mrm 5u KQUEUE 0xfffff806350b0600 count=0, state=0 doveadm 46985 mrm 6w FIFO 163,709754999 0t0 29707 /var/run/dovecot/stats-mail doveadm 46985 mrm 7u VREG 22,2669215774 11080 10650 /home/mrm/mail/TRAVEL/.imap/alamo/dovecot.index.log doveadm 46985 mrm 8u VREG 22,2669215774 536 137895 /home/mrm/mail/TRAVEL/.imap/alamo/dovecot.index doveadm 46985 mrm 9u VREG 22,2669215774 6920 10680 /home/mrm/mail/TRAVEL/.imap/hawaiian.airlines/dovecot.index.log doveadm 46985 mrm 10u VREG 22,2669215774 2968 10679 /home/mrm/mail/TRAVEL/.imap/hawaiian.airlines/dovecot.index.cache doveadm 46985 mrm 11u VREG 22,2669215774 6520 139902 /home/mrm/mail/.imap/Sent/dovecot.index.log doveadm 46985 mrm 12u VREG 22,2669215774 9288 139905 /home/mrm/mail/.imap/Sent/dovecot.index doveadm 46985 mrm 13u VREG 22,2669215774 7224 10716 /home/mrm/mail/TRAVEL/.imap/priceline/dovecot.index.log doveadm 46985 mrm 14u VREG 22,2669215774 3108 10715 /home/mrm/mail/TRAVEL/.imap/priceline/dovecot.index.cache doveadm 46985 mrm 15u VREG 22,2669215774 9236 10648 /home/mrm/mail/TRAVEL/.imap/alamo/dovecot.index.cache doveadm 46985 mrm 16u VREG 22,2669215774 174892 143343 /home/mrm/mail/.imap/Sent/dovecot.index.cache doveadm 46985 mrm 17u VREG 22,2669215774 32656 143058 /home/mrm/mail/.imap/INBOX/dovecot.index.log doveadm 46985 mrm 18u VREG 22,2669215774 0 135848 /home/mrm (zroot/home/mrm) doveadm 46985 mrm 19u VREG 22,2669215774 35656 135336 /home/mrm/mail/.imap/INBOX/dovecot.index doveadm 46985 mrm 20u VREG 22,2669215774 0 135849 /home/mrm (zroot/home/mrm) doveadm 46985 mrm 21u KQUEUE 0xfffff80163b1ba00 count=1, state=0 doveadm 46985 mrm 22u IPv4 0xfffff805ea69a000 0t0 TCP localhost:44730->localhost:9998 (ESTABLISHED) doveadm 46985 mrm 25uR VREG 22,2669215774 32997612 4151 /home/mrm/mail/Sent thebighonker.lerctr.org
On Sun, Oct 23, 2016 at 12:20 PM, Aki Tuomi aki.tuomi@dovecot.fi wrote:
According to man page, the only way it can return EINVAL (22) is either bad filter, or bad timeout. I can't see how the filter would be bad, so I'm guessing ts must be bad. Unfortunately I forgot to ask for it, so I am going to have to ask you run it again and run
p ts
if that's valid, then the only thing that can be bad if the file descriptor 23.
Aki
On October 23, 2016 at 7:42 PM Larry Rosenman larryrtx@gmail.com wrote:
ok, gdb7 works: (gdb) fr 6 #6 0x00000008011a3e49 in io_loop_handler_run_internal (ioloop=0x801c214e0) at ioloop-kqueue.c:131 131 i_panic("kevent(): %m"); (gdb) p errno $1 = 22 (gdb) p ret $2 = -1 (gdb) p *ioloop $3 = {prev = 0x801c21080, cur_ctx = 0x0, io_files = 0x801c4f980, next_io_file = 0x0, timeouts = 0x801c19e60, timeouts_new = {arr = {buffer = 0x801c5ac80, element_size = 8}, v = 0x801c5ac80, v_modifiable = 0x801c5ac80}, handler_context = 0x801c19e80, notify_handler_context = 0x0, max_fd_count = 0, time_moved_callback = 0x800d53bb0
, next_max_time = 1477240784, ioloop_wait_usecs = 29863, io_pending_count = 1, running = 1, iolooping = 1} (gdb) p *ctx $4 = {kq = 22, deleted_count = 0, events = {arr = {buffer = 0x801c5acc0, element_size = 32}, v = 0x801c5acc0, v_modifiable = 0x801c5acc0}} (gdb) p *events $5 = {ident = 23, filter = -1, flags = 0, fflags = 0, data = 8, udata = 0x801c4f980} (gdb) On Sun, Oct 23, 2016 at 11:27 AM, Larry Rosenman larryrtx@gmail.com wrote:
grrr.
/home/mrm $ gdb /usr/local/bin/doveadm GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... (gdb) run -D -vvvvvv index * Starting program: /usr/local/bin/doveadm -D -vvvvvv index *
Program received signal SIGTRAP, Trace/breakpoint trap. Cannot remove breakpoints because program is no longer writable. It might be running in another process. Further execution is probably impossible. 0x0000000800624490 in ?? () (gdb)
Ideas?
On Sun, Oct 23, 2016 at 11:14 AM, Aki Tuomi aki.tuomi@dovecot.fi wrote:
Hi,
can you run doveadm in gdb, wait for it to crash, and then go to frame 6 ( io_loop_handler_run_internal) and run
p errno p ret p *ioloop p *ctx p *events
Sorry but the crash doesn't make enough sense yet to me, we need to determine what the invalid parameter is.
Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx@gmail.com US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
-- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx@gmail.com US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
-- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx@gmail.com US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
-- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx@gmail.com US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281