keent() from Tika - with doveadm

Aki Tuomi aki.tuomi at dovecot.fi
Mon Oct 24 09:34:38 UTC 2016


Hi!

We found some problems with those patches, and ended up doing slightly
different fix:

https://github.com/dovecot/core/compare/3e41b3d%5E...cca98b.patch

Aki

On 24.10.2016 10:17, Aki Tuomi wrote:
> Hi!
>
> Can you try these two patches?
>
> Aki
>
>
> On 24.10.2016 08:48, Aki Tuomi wrote:
>> Ok so that timeval makes no sense. We'll look into it. 
>>
>> Aki
>>
>>> On October 24, 2016 at 12:22 AM Larry Rosenman <larryrtx at gmail.com> wrote:
>>>
>>>
>>> doveadm(mrm): Debug: http-client: conn 127.0.0.1:9998 [1]: Got 200 response
>>> for request [Req38: PUT http://localhost:9998/tika/] (took 296 ms + 8 ms in
>>> queue)
>>> doveadm(mrm): Panic: kevent(): Invalid argument
>>>
>>> Program received signal SIGABRT, Aborted.
>>> 0x00000008014e6f7a in thr_kill () from /lib/libc.so.7
>>> (gdb) fr 6
>>> #6  0x00000008011a3e49 in io_loop_handler_run_internal (ioloop=0x801c214e0)
>>>     at ioloop-kqueue.c:131
>>> 131 i_panic("kevent(): %m");
>>> (gdb) p ts
>>> $1 = {tv_sec = 34389923520, tv_nsec = 140737488345872000}
>>> (gdb) p errno
>>> $2 = 22
>>> (gdb) p ret
>>> $3 = -1
>>> (gdb) p *ioloop
>>> $4 = {prev = 0x801c21080, cur_ctx = 0x0, io_files = 0x801c4f980,
>>>   next_io_file = 0x0, timeouts = 0x801d17540, timeouts_new = {arr = {
>>>       buffer = 0x801cd9700, element_size = 8}, v = 0x801cd9700,
>>>     v_modifiable = 0x801cd9700}, handler_context = 0x801d17560,
>>>   notify_handler_context = 0x0, max_fd_count = 0,
>>>   time_moved_callback = 0x800d53bb0 <mail_storage_service_time_moved>,
>>>   next_max_time = 1477257580, ioloop_wait_usecs = 27148, io_pending_count =
>>> 1,
>>>   running = 1, iolooping = 1}
>>> (gdb) p* ctx
>>> $5 = {kq = 21, deleted_count = 0, events = {arr = {buffer = 0x801cd9740,
>>>       element_size = 32}, v = 0x801cd9740, v_modifiable = 0x801cd9740}}
>>> (gdb) p *events
>>> $6 = {ident = 22, filter = -1, flags = 0, fflags = 0, data = 8,
>>>   udata = 0x801c4f980}
>>> (gdb)
>>>
>>> thebighonker.lerctr.org ~ $ ps auxw|grep doveadm
>>> mrm      46965   0.0  0.2   108516  55264  0  I+    4:19PM    0:02.28 gdb
>>> /usr/local/bin/doveadm (gdb7111)
>>> mrm      46985   0.0  0.0    81236  15432  0  TX    4:19PM    0:03.51
>>> /usr/local/bin/doveadm -D -vvvvvvv index *
>>> ler      47221   0.0  0.0    18856   2360  1  S+    4:21PM    0:00.00 grep
>>> doveadm
>>> thebighonker.lerctr.org ~ $ sudo lsof -p 46985
>>> Password:
>>> COMMAND   PID USER   FD     TYPE             DEVICE SIZE/OFF   NODE NAME
>>> doveadm 46985  mrm  cwd     VDIR      22,2669215774      152      4
>>> /home/mrm
>>> doveadm 46985  mrm  rtd     VDIR       19,766509061       28      4 /
>>> doveadm 46985  mrm  txt     VREG     119,3584295129  1714125 182952
>>> /usr/local/bin/doveadm
>>> doveadm 46985  mrm  txt     VREG       19,766509061   132272  14382
>>> /libexec/ld-elf.so.1
>>> doveadm 46985  mrm  txt     VREG      22,2669215774     6920  10680
>>> /home/mrm/mail/TRAVEL/.imap/hawaiian.airlines/dovecot.index.log
>>> doveadm 46985  mrm  txt     VREG      22,2669215774     7224  10716
>>> /home/mrm/mail/TRAVEL/.imap/priceline/dovecot.index.log
>>> doveadm 46985  mrm  txt     VREG      22,2669215774    11080  10650
>>> /home/mrm/mail/TRAVEL/.imap/alamo/dovecot.index.log
>>> doveadm 46985  mrm  txt     VREG      22,2669215774     2968  10679
>>> /home/mrm/mail/TRAVEL/.imap/hawaiian.airlines/dovecot.index.cache
>>> doveadm 46985  mrm  txt     VREG      22,2669215774     3108  10715
>>> /home/mrm/mail/TRAVEL/.imap/priceline/dovecot.index.cache
>>> doveadm 46985  mrm  txt     VREG      22,2669215774     6520 139902
>>> /home/mrm/mail/.imap/Sent/dovecot.index.log
>>> doveadm 46985  mrm  txt     VREG      22,2669215774     9236  10648
>>> /home/mrm/mail/TRAVEL/.imap/alamo/dovecot.index.cache
>>> doveadm 46985  mrm  txt     VREG      22,2669215774   174892 143343
>>> /home/mrm/mail/.imap/Sent/dovecot.index.cache
>>> doveadm 46985  mrm  txt     VREG      22,2669215774    32656 143058
>>> /home/mrm/mail/.imap/INBOX/dovecot.index.log
>>> doveadm 46985  mrm  txt     VREG       19,766509061      720  30627
>>> /usr/share/i18n/csmapper/CP/CP1251%UCS.mps
>>> doveadm 46985  mrm  txt     VREG       19,766509061      720  30630
>>> /usr/share/i18n/csmapper/CP/CP1252%UCS.mps
>>> doveadm 46985  mrm  txt     VREG       19,766509061    89576   6846
>>> /lib/libz.so.6
>>> doveadm 46985  mrm  txt     VREG       19,766509061    62008   5994
>>> /lib/libcrypt.so.5
>>> doveadm 46985  mrm  txt     VREG     119,3584295129  6725689 183611
>>> /usr/local/lib/dovecot/libdovecot-storage.so.0.0.0
>>> doveadm 46985  mrm  txt     VREG     119,3584295129  3162259 183615
>>> /usr/local/lib/dovecot/libdovecot.so.0.0.0
>>> doveadm 46985  mrm  txt     VREG       19,766509061  1649944   4782
>>> /lib/libc.so.7
>>> doveadm 46985  mrm  txt     VREG     119,3584295129    80142 183550
>>> /usr/local/lib/dovecot/lib15_notify_plugin.so
>>> doveadm 46985  mrm  txt     VREG     119,3584295129   652615 183556
>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>> doveadm 46985  mrm  txt     VREG     119,3584295129  2730888 268825
>>> /usr/local/lib/libicui18n.so.57.1
>>> doveadm 46985  mrm  txt     VREG     119,3584295129  1753976 268849
>>> /usr/local/lib/libicuuc.so.57.1
>>> doveadm 46985  mrm  txt     VREG     119,3584295129     1704 268821
>>> /usr/local/lib/libicudata.so.57.1
>>> doveadm 46985  mrm  txt     VREG       19,766509061   102560   6745
>>> /lib/libthr.so.3
>>> doveadm 46985  mrm  txt     VREG       19,766509061   184712   5795
>>> /lib/libm.so.5
>>> doveadm 46985  mrm  txt     VREG       19,766509061   774000   5642
>>> /usr/lib/libc++.so.1
>>> doveadm 46985  mrm  txt     VREG       19,766509061   103304   5742
>>> /lib/libcxxrt.so.1
>>> doveadm 46985  mrm  txt     VREG       19,766509061    56344   7436
>>> /lib/libgcc_s.so.1
>>> doveadm 46985  mrm  txt     VREG     119,3584295129   349981 183782
>>> /usr/local/lib/dovecot/lib21_fts_lucene_plugin.so
>>> doveadm 46985  mrm  txt     VREG     119,3584295129  1969384 113258
>>> /usr/local/lib/libclucene-core.so.2.3.3.4
>>> doveadm 46985  mrm  txt     VREG     119,3584295129   128992 113261
>>> /usr/local/lib/libclucene-shared.so.2.3.3.4
>>> doveadm 46985  mrm  txt     VREG     119,3584295129   143141 183578
>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>> doveadm 46985  mrm  txt     VREG     119,3584295129    37368 151926
>>> /usr/local/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
>>> doveadm 46985  mrm  txt     VREG     119,3584295129   693808 151924
>>> /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0.0.0
>>> doveadm 46985  mrm  txt     VREG     119,3584295129   146477 183599
>>> /usr/local/lib/dovecot/libdovecot-lda.so.0.0.0
>>> doveadm 46985  mrm  txt     VREG     119,3584295129    13823 183780
>>> /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so
>>> doveadm 46985  mrm  txt     VREG     119,3584295129    88081 183527
>>> /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so
>>> doveadm 46985  mrm  txt     VREG       19,766509061     8304   6330
>>> /usr/lib/i18n/libiconv_std.so.4
>>> doveadm 46985  mrm  txt     VREG       19,766509061     6744   6318
>>> /usr/lib/i18n/libUTF8.so.4
>>> doveadm 46985  mrm  txt     VREG       19,766509061     4384   6336
>>> /usr/lib/i18n/libmapper_none.so.4
>>> doveadm 46985  mrm  txt     VREG       19,766509061     7584   6345
>>> /usr/lib/i18n/libmapper_std.so.4
>>> doveadm 46985  mrm    0u    VCHR              0,188 0t390889    188
>>> /dev/pts/0
>>> doveadm 46985  mrm    1u    VCHR              0,188 0t390889    188
>>> /dev/pts/0
>>> doveadm 46985  mrm    2u    VCHR              0,188 0t390889    188
>>> /dev/pts/0
>>> doveadm 46985  mrm    3u    PIPE 0xfffff806fdf505d0    16384
>>>  ->0xfffff806fdf50730
>>> doveadm 46985  mrm    4u    PIPE 0xfffff806fdf50730        0
>>>  ->0xfffff806fdf505d0
>>> doveadm 46985  mrm    5u  KQUEUE 0xfffff806350b0600
>>> count=0, state=0
>>> doveadm 46985  mrm    6w    FIFO      163,709754999      0t0  29707
>>> /var/run/dovecot/stats-mail
>>> doveadm 46985  mrm    7u    VREG      22,2669215774    11080  10650
>>> /home/mrm/mail/TRAVEL/.imap/alamo/dovecot.index.log
>>> doveadm 46985  mrm    8u    VREG      22,2669215774      536 137895
>>> /home/mrm/mail/TRAVEL/.imap/alamo/dovecot.index
>>> doveadm 46985  mrm    9u    VREG      22,2669215774     6920  10680
>>> /home/mrm/mail/TRAVEL/.imap/hawaiian.airlines/dovecot.index.log
>>> doveadm 46985  mrm   10u    VREG      22,2669215774     2968  10679
>>> /home/mrm/mail/TRAVEL/.imap/hawaiian.airlines/dovecot.index.cache
>>> doveadm 46985  mrm   11u    VREG      22,2669215774     6520 139902
>>> /home/mrm/mail/.imap/Sent/dovecot.index.log
>>> doveadm 46985  mrm   12u    VREG      22,2669215774     9288 139905
>>> /home/mrm/mail/.imap/Sent/dovecot.index
>>> doveadm 46985  mrm   13u    VREG      22,2669215774     7224  10716
>>> /home/mrm/mail/TRAVEL/.imap/priceline/dovecot.index.log
>>> doveadm 46985  mrm   14u    VREG      22,2669215774     3108  10715
>>> /home/mrm/mail/TRAVEL/.imap/priceline/dovecot.index.cache
>>> doveadm 46985  mrm   15u    VREG      22,2669215774     9236  10648
>>> /home/mrm/mail/TRAVEL/.imap/alamo/dovecot.index.cache
>>> doveadm 46985  mrm   16u    VREG      22,2669215774   174892 143343
>>> /home/mrm/mail/.imap/Sent/dovecot.index.cache
>>> doveadm 46985  mrm   17u    VREG      22,2669215774    32656 143058
>>> /home/mrm/mail/.imap/INBOX/dovecot.index.log
>>> doveadm 46985  mrm   18u    VREG      22,2669215774        0 135848
>>> /home/mrm (zroot/home/mrm)
>>> doveadm 46985  mrm   19u    VREG      22,2669215774    35656 135336
>>> /home/mrm/mail/.imap/INBOX/dovecot.index
>>> doveadm 46985  mrm   20u    VREG      22,2669215774        0 135849
>>> /home/mrm (zroot/home/mrm)
>>> doveadm 46985  mrm   21u  KQUEUE 0xfffff80163b1ba00
>>> count=1, state=0
>>> doveadm 46985  mrm   22u    IPv4 0xfffff805ea69a000      0t0    TCP
>>> localhost:44730->localhost:9998 (ESTABLISHED)
>>> doveadm 46985  mrm   25uR   VREG      22,2669215774 32997612   4151
>>> /home/mrm/mail/Sent
>>> thebighonker.lerctr.org
>>>
>>>
>>>
>>> On Sun, Oct 23, 2016 at 12:20 PM, Aki Tuomi <aki.tuomi at dovecot.fi> wrote:
>>>
>>>> According to man page, the only way it can return EINVAL (22) is either
>>>> bad filter, or bad timeout. I can't see how the filter would be bad, so I'm
>>>> guessing ts must be bad. Unfortunately I forgot to ask for it, so I am
>>>> going to have to ask you run it again and run
>>>>
>>>> p ts
>>>>
>>>> if that's valid, then the only thing that can be bad if the file
>>>> descriptor 23.
>>>>
>>>> Aki
>>>>
>>>>> On October 23, 2016 at 7:42 PM Larry Rosenman <larryrtx at gmail.com>
>>>> wrote:
>>>>> ok, gdb7 works:
>>>>> (gdb) fr 6
>>>>> #6  0x00000008011a3e49 in io_loop_handler_run_internal
>>>> (ioloop=0x801c214e0)
>>>>> at ioloop-kqueue.c:131
>>>>> 131 i_panic("kevent(): %m");
>>>>> (gdb) p errno
>>>>> $1 = 22
>>>>> (gdb) p ret
>>>>> $2 = -1
>>>>> (gdb) p *ioloop
>>>>> $3 = {prev = 0x801c21080, cur_ctx = 0x0, io_files = 0x801c4f980,
>>>>> next_io_file = 0x0, timeouts = 0x801c19e60, timeouts_new = {arr =
>>>> {buffer =
>>>>> 0x801c5ac80, element_size = 8}, v = 0x801c5ac80,
>>>>>     v_modifiable = 0x801c5ac80}, handler_context = 0x801c19e80,
>>>>> notify_handler_context = 0x0, max_fd_count = 0, time_moved_callback =
>>>>> 0x800d53bb0 <mail_storage_service_time_moved>,
>>>>>   next_max_time = 1477240784, ioloop_wait_usecs = 29863,
>>>> io_pending_count =
>>>>> 1, running = 1, iolooping = 1}
>>>>> (gdb) p *ctx
>>>>> $4 = {kq = 22, deleted_count = 0, events = {arr = {buffer = 0x801c5acc0,
>>>>> element_size = 32}, v = 0x801c5acc0, v_modifiable = 0x801c5acc0}}
>>>>> (gdb) p *events
>>>>> $5 = {ident = 23, filter = -1, flags = 0, fflags = 0, data = 8, udata =
>>>>> 0x801c4f980}
>>>>> (gdb)
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Oct 23, 2016 at 11:27 AM, Larry Rosenman <larryrtx at gmail.com>
>>>> wrote:
>>>>>> grrr.
>>>>>>
>>>>>> /home/mrm $ gdb /usr/local/bin/doveadm
>>>>>> GNU gdb 6.1.1 [FreeBSD]
>>>>>> Copyright 2004 Free Software Foundation, Inc.
>>>>>> GDB is free software, covered by the GNU General Public License, and
>>>> you
>>>>>> are
>>>>>> welcome to change it and/or distribute copies of it under certain
>>>>>> conditions.
>>>>>> Type "show copying" to see the conditions.
>>>>>> There is absolutely no warranty for GDB.  Type "show warranty" for
>>>> details.
>>>>>> This GDB was configured as "amd64-marcel-freebsd"...
>>>>>> (gdb) run -D -vvvvvv index *
>>>>>> Starting program: /usr/local/bin/doveadm -D -vvvvvv index *
>>>>>>
>>>>>> Program received signal SIGTRAP, Trace/breakpoint trap.
>>>>>> Cannot remove breakpoints because program is no longer writable.
>>>>>> It might be running in another process.
>>>>>> Further execution is probably impossible.
>>>>>> 0x0000000800624490 in ?? ()
>>>>>> (gdb)
>>>>>>
>>>>>> Ideas?
>>>>>>
>>>>>>
>>>>>> On Sun, Oct 23, 2016 at 11:14 AM, Aki Tuomi <aki.tuomi at dovecot.fi>
>>>> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> can you run doveadm in gdb, wait for it to crash, and then go to
>>>> frame 6
>>>>>>> ( io_loop_handler_run_internal) and run
>>>>>>>
>>>>>>> p errno
>>>>>>> p ret
>>>>>>> p *ioloop
>>>>>>> p *ctx
>>>>>>> p *events
>>>>>>>
>>>>>>> Sorry but the crash doesn't make enough sense yet to me, we need to
>>>>>>> determine what the invalid parameter is.
>>>>>>>
>>>>>>>> Larry Rosenman                     http://www.lerctr.org/~ler
>>>>>>>> Phone: +1 214-642-9640 (c)     E-Mail: larryrtx at gmail.com
>>>>>>>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
>>>>>>
>>>>>> --
>>>>>> Larry Rosenman                     http://www.lerctr.org/~ler
>>>>>> Phone: +1 214-642-9640 (c)     E-Mail: larryrtx at gmail.com
>>>>>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
>>>>>>
>>>>>
>>>>> --
>>>>> Larry Rosenman                     http://www.lerctr.org/~ler
>>>>> Phone: +1 214-642-9640 (c)     E-Mail: larryrtx at gmail.com
>>>>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
>>>
>>> -- 
>>> Larry Rosenman                     http://www.lerctr.org/~ler
>>> Phone: +1 214-642-9640 (c)     E-Mail: larryrtx at gmail.com
>>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281




More information about the dovecot mailing list