Op 16-9-2019 om 11:53 schreef Domenico Pastore via dovecot:

Hi,

I have other information.

I analyzed the core dump:

(gdb) core 22534 [New LWP 22534] Reading symbols from /usr/libexec/dovecot/doveadm-server...Reading symbols from /usr/lib/debug/usr/libexec/dovecot/doveadm-server.debug...done. done. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot-devel01/doveadm-server'. Program terminated with signal 11, Segmentation fault. #0  clear (this=0x0) at /usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/util/VoidMap.h:123 123            if ( dk || dv ){ (gdb)

Can you issue GDB command bt full at this point?

Regards,

Stephan.

Br,

Domenico

*Domenico Pastore* Senior Cloud Engineer *T* 06.98269600 | *M* 347.1474270 | *F* 06.98269680 *Par-Tec http://www.par-tec.it* | *beyond the IT domain* Via Cristoforo Colombo 163, 00147 Roma *CONFIDENZIALE*: Questo messaggio ed i suoi allegati sono di carattere confidenziale. È vietato l'inoltro non autorizzato a destinatari diversi da quelli indicati nel messaggio originale. Se ricevuto per errore si prega di informare il mittente e cancellarlo immediatamente.

Il 16/09/19 09:50, Domenico Pastore via dovecot ha scritto:

...

Hi,

do you have the same problem with lucene?

I collected a strace on the dovecot process during indexing:

[pid 31339] execve("/usr/libexec/dovecot/doveadm-server", ["dovecot-devel01/doveadm-server"], [/* 18 vars */] [pid 30711] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=30932, si_uid=0, si_status=SIGSEGV, si_utime=3, si_stime=1} --- [pid 30711] write(62, "\0", 1)          = 1 [pid 30711] read(124, "8x\0\0008\0\0\0\251\3\0\0", 1020) = 12 [pid 30711] read(61, "\0", 64)          = 1 [pid 30711] open("/proc/sys/fs/suid_dumpable", O_RDONLY) = 227 [pid 30711] read(227, "0\n", 9)         = 2 [pid 30711] write(65, "25 30932 FATAL service(doveadm): child 30932 killed with signal 11 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2)\n", 170) = 170 [pid 30711] write(14, "KILL\t30932\n", 11) = 11 [pid 30711] write(65, "25 30932 BYE\n", 13) = 13 [pid 31339] <... execve resumed> )      = 0 [pid 31339] open("/usr/lib64/dovecot/tls/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) [pid 31339] stat("/usr/lib64/dovecot/tls/x86_64", 0x7ffda7e3f1d0) = -1 ENOENT (No such file or directory) [pid 31339] open("/usr/lib64/dovecot/tls/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) [pid 31339] stat("/usr/lib64/dovecot/tls", 0x7ffda7e3f1d0) = -1 ENOENT (No such file or directory) [pid 31339] open("/usr/lib64/dovecot/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) [pid 31339] stat("/usr/lib64/dovecot/x86_64", 0x7ffda7e3f1d0) = -1 ENOENT (No such file or directory) [pid 31339] open("/usr/lib64/dovecot/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) [pid 31339] stat("/usr/lib64/dovecot", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 [pid 31339] open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 4 [pid 31339] open("/lib64/libz.so.1", O_RDONLY|O_CLOEXEC) = 4

All missing libs are in "/usr/lib64" and no to "/usr/lib64/dovecot"

Can you help me?

Br,

Domenico

*Domenico Pastore* Senior Cloud Engineer *T* 06.98269600 | *M* 347.1474270 | *F* 06.98269680 *Par-Tec http://www.par-tec.it* | *beyond the IT domain* Via Cristoforo Colombo 163, 00147 Roma *CONFIDENZIALE*: Questo messaggio ed i suoi allegati sono di carattere confidenziale. È vietato l'inoltro non autorizzato a destinatari diversi da quelli indicati nel messaggio originale. Se ricevuto per errore si prega di informare il mittente e cancellarlo immediatamente.

Il 12/09/19 12:07, Domenico Pastore via dovecot ha scritto:

...

Hi,

i have a problem with libclucene-core.so.2.3.3.4. I See segfault error to messages log, when dovecot store mail to INBOX.

set 12 13:41:21 box02.par-tec.it kernel: doveadm-server[20285]: *segfault at 38 ip 00007f575b68fc42 sp 00007ffc0272e750 error 4 in libclucene-core.so.2.3.3.4*[7f575b569000+188000] set 12 13:41:22 box02.par-tec.it kernel: doveadm-server[18871]: segfault at 38 ip 00007f9bea6d2c42 sp 00007fff47bceae0 error 4 in libclucene-core.so.2.3.3.4[7f9bea5ac000+188000]

In the dovecot logs:

2019-09-12 13:45:09 auth: Debug: ldap(test4444client@newsvilXXXX,10.0.8.128): Finished userdb lookup 2019-09-12 13:45:09 auth: Debug: userdb out: USER    1 test4444client@newsvilXXXX home=/pec_store/pec_mailbox/pec_mailbox_devel01/4/5/6/7/1/45671_18231 quota_rule=*:bytes=1048576000    mail_plugins+= 2019-09-12 13:45:10  SKENO8Uvel0gbAAA5ZKy4w doveadm(test4444client@newsvilXXXX): *Fatal: master: service(doveadm): child 27680 killed with signal 11 (core dumped)* 2019-09-12 13:45:10 auth: Debug: master in: USER    1 test4444client@newsvilXXXX    service=doveadm lip=192.160.10.3    lport=924    rip=10.0.8.128 rport=36760 2019-09-12 13:45:10 auth: Debug: ldap(test4444client@newsvilXXXX 10.0.8.128): Performing userdb lookup

I have attach the coredump of error and dovecot.conf

Thanks for support.

Br,

Domenico

------------------- info: ------------------- # 2.3.7.2 (3c910f64b): /pec/config/dovecot/dovecot-devel01/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 3.10.0-957.27.2.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: box02

Hi,

i have find other information of the coredump.

My mail_plugin is: mail_plugins = quota fts fts_lucene mail_log notify virtual

When I using quota and fts_lucene the coredump is generated. If i use only the quota plugin or fts_lucene plugin, lmtp store work correctly.

Have you an ideas?

Br, Domenico

Domenico Pastore Senior Cloud Engineer T 06.98269600 | M 347.1474270 | F 06.98269680 Par-Techttp://www.par-tec.it | beyond the IT domain Via Cristoforo Colombo 163, 00147 Roma CONFIDENZIALE: Questo messaggio ed i suoi allegati sono di carattere confidenziale. È vietato l'inoltro non autorizzato a destinatari diversi da quelli indicati nel messaggio originale. Se ricevuto per errore si prega di informare il mittente e cancellarlo immediatamente. Il 16/09/19 17:20, Domenico Pastore via dovecot ha scritto:

Hi Stephan,

thanks for your replay.

This is bt full output:

(gdb) bt full #0 clear (this=0x0) at /usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/util/VoidMap.h:123 No locals. #1 lucene::search::FieldSortedHitQueue::_shutdown () at /usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/search/FieldSortedHitQueue.cpp:60 No locals. #2 0x00007f7dc5eabe96 in _lucene_shutdown () at /usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/StdHeader.cpp:33 No locals. #3 0x00007f7dc973bfbb in module_dir_deinit (modules=<optimized out>) at module-dir.c:578 _data_stack_cur_id = 4 module = 0x55ddb26268f0 rev = 0x55ddb25bd050 i = <optimized out> count = 6 #4 0x00007f7dc973c001 in module_dir_unload (modules=modules@entry=0x7f7dc9d2a380 ) at module-dir.c:590 module = <optimized out> next = <optimized out> #5 0x00007f7dc9a26264 in mail_storage_service_deinit (_ctx=_ctx@entry=0x55ddb25d9848) at mail-storage-service.c:1730 ctx = 0x55ddb2612c18 #6 0x000055ddb0e0dd35 in doveadm_mail_cmd_exec (ctx=ctx@entry=0x55ddb25d9808, wildcard_user=wildcard_user@entry=0x0) at doveadm-mail.c:673 cctx = <optimized out> cli = <optimized out> ret = <optimized out> error = 0x7f7dc96762f7 <__x86_return_thunk+5> "\363\220\017\256\350\353\371H\215d$\b\303\350\a" #7 0x000055ddb0e0ebbd in doveadm_cmd_ver2_to_mail_cmd_wrapper (cctx=<optimized out>) at doveadm-mail.c:1144 mctx = 0x55ddb25d9808 wildcard_user = <optimized out> fieldstr = 0x55ddb25b9318 "\315\"N\305}\177" pargv = {arr = {buffer = 0x55ddb25d9a38, element_size = 8}, v = 0x55ddb25d9a38, v_modifiable = 0x55ddb25d9a38} full_args = {arr = {buffer = <optimized out>, element_size = 8}, v = <optimized out>, v_modifiable = <optimized out>} i = <optimized out> cli = <optimized out> tcp_server = <optimized out> mail_cmd = {alloc = 0x7f7dc54e1d00 , name = 0x7f7dc54e22b6 "quota get", usage_args = 0x7f7dc54e2291 ""} dashdash = 0x55ddb0e4a7ca "--" args_pos = 1 #8 0x000055ddb0e1fc98 in doveadm_cmd_run_ver2 (argc=argc@entry=1, argv=argv@entry=0x55ddb25b9158, cctx=cctx@entry=0x7ffff7c52670) at doveadm-cmd.c:574 param = <optimized out> pargv = {arr = {buffer = 0x55ddb25b9418, element_size = 104}, v = 0x55ddb25b9418, v_modifiable = 0x55ddb25b9418} opts = {arr = {buffer = <optimized out>, element_size = 32}, v = <optimized out>, v_modifiable = <optimized out>} pargc = 4 c = -1 ---Type <return> to continue, or q <return> to quit--- li = -252221570 pool = 0x55ddb25b9248 optbuf = 0x55ddb25b9260 __func__ = "doveadm_cmd_run_ver2" #9 0x000055ddb0e24020 in doveadm_cmd_server_run_ver2 (cctx=0x7ffff7c52670, argv=0x55ddb25b9158, argc=1, conn=0x55ddb260dbf8) at client-connection-tcp.c:178 No locals. #10 doveadm_cmd_handle (cctx=0x7ffff7c52670, argv=0x55ddb25b9158, argc=1, cmd_name=<optimized out>, conn=0x55ddb260dbf8) at client-connection-tcp.c:342 mail_cmd = <optimized out> mctx = <optimized out> cmd_ver2 = <optimized out> prev_ioloop = 0x55ddb25c1cc0 cmd = 0x0 #11 client_handle_command (args=<optimized out>, conn=0x55ddb260dbf8) at client-connection-tcp.c:415 cctx = {cmd = 0x55ddb25d73e8, argc = 4, argv = 0x55ddb25b9450, username = 0x55ddb25b9119 "test4444client@newsvilpec.babel.it"mailto:test4444client@newsvilpec.babel.it, local_ip = {family = 2, u = {ip6 = {__in6_u = { __u6_addr8 = "\300\240\n\003", '\000' , __u6_addr16 = {41152, 778, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {51028160, 0, 0, 0}}}, ip4 = {s_addr = 51028160}}}, remote_ip = {family = 2, u = { ip6 = {__in6_u = {__u6_addr8 = "\n\000\b\200", '\000' , __u6_addr16 = {10, 32776, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {2148007946, 0, 0, 0}}}, ip4 = {s_addr = 2148007946}}}, local_port = 924, remote_port = 41491, conn_type = DOVEADM_CONNECTION_TYPE_TCP, input = 0x55ddb25d2dc0, output = 0x55ddb25d8d00} flags = <optimized out> cmd_name = <optimized out> argc = <optimized out> #12 client_connection_tcp_input (conn=0x55ddb260dbf8) at client-connection-tcp.c:561 args = <optimized out> _data_stack_cur_id = 3 line = <optimized out> ok = true minor = 0 #13 0x00007f7dc973155f in io_loop_call_io (io=0x55ddb25e7b20) at ioloop.c:704 ioloop = 0x55ddb25c1cc0 t_id = 2 __func__ = "io_loop_call_io" #14 0x00007f7dc9732fa3 in io_loop_handler_run_internal (ioloop=ioloop@entry=0x55ddb25c1cc0) at ioloop-epoll.c:221 ctx = 0x55ddb25c77b0 events = <optimized out> list = 0x55ddb260be90 io = <optimized out> tv = {tv_sec = 2147483, tv_usec = 0} events_count = <optimized out> msecs = <optimized out> ret = 1 i = 0 ---Type <return> to continue, or q <return> to quit--- call = <optimized out> __func__ = "io_loop_handler_run_internal" #15 0x00007f7dc9731666 in io_loop_handler_run (ioloop=ioloop@entry=0x55ddb25c1cc0) at ioloop.c:756 __func__ = "io_loop_handler_run" #16 0x00007f7dc9731888 in io_loop_run (ioloop=0x55ddb25c1cc0) at ioloop.c:729 __func__ = "io_loop_run" #17 0x00007f7dc96a42d3 in master_service_run (service=0x55ddb25c1b50, callback=callback@entry=0x55ddb0e27260 ) at master-service.c:781 No locals. #18 0x000055ddb0dfd421 in main (argc=1, argv=0x55ddb25c1890) at main.c:151 set_roots = {0x7f7dc99c5d00 , 0x55ddb106c6a0 , 0x0} input = {roots = 0x7ffff7c52850, config_path = 0x0, preserve_environment = false, preserve_user = false, preserve_home = false, never_exec = false, use_sysexits = false, parse_full_config = false, module = 0x55ddb0e46d3a "doveadm", service = 0x55ddb0e46d3a "doveadm", username = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = {__u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = { 0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_name = 0x0} output = {specific_services = 0x0, service_uses_local = false, service_uses_remote = false, used_local = false, used_remote = false, permission_denied = false} error = 0x7f7dc90ecca7 "__vdso_getcpu" c = <optimized out> (gdb)

Br,

Domenico

Domenico Pastore Senior Cloud Engineer T 06.98269600 | M 347.1474270 | F 06.98269680 Par-Techttp://www.par-tec.it | beyond the IT domain Via Cristoforo Colombo 163, 00147 Roma CONFIDENZIALE: Questo messaggio ed i suoi allegati sono di carattere confidenziale. È vietato l'inoltro non autorizzato a destinatari diversi da quelli indicati nel messaggio originale. Se ricevuto per errore si prega di informare il mittente e cancellarlo immediatamente. Il 16/09/19 17:05, Stephan Bosch ha scritto:

Op 16-9-2019 om 11:53 schreef Domenico Pastore via dovecot:

Hi,

I have other information.

I analyzed the core dump:

(gdb) core 22534 [New LWP 22534] Reading symbols from /usr/libexec/dovecot/doveadm-server...Reading symbols from /usr/lib/debug/usr/libexec/dovecot/doveadm-server.debug...done. done. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot-devel01/doveadm-server'. Program terminated with signal 11, Segmentation fault. #0 clear (this=0x0) at /usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/util/VoidMap.h:123 123 if ( dk || dv ){ (gdb)

Can you issue GDB command bt full at this point?

Regards,

Stephan.

Br,

Domenico

*Domenico Pastore* Senior Cloud Engineer *T* 06.98269600 | *M* 347.1474270 | *F* 06.98269680 *Par-Tec http://www.par-tec.ithttp://www.par-tec.it* | *beyond the IT domain* Via Cristoforo Colombo 163, 00147 Roma *CONFIDENZIALE*: Questo messaggio ed i suoi allegati sono di carattere confidenziale. È vietato l'inoltro non autorizzato a destinatari diversi da quelli indicati nel messaggio originale. Se ricevuto per errore si prega di informare il mittente e cancellarlo immediatamente.

Il 16/09/19 09:50, Domenico Pastore via dovecot ha scritto:

Hi,

do you have the same problem with lucene?

I collected a strace on the dovecot process during indexing:

[pid 31339] execve("/usr/libexec/dovecot/doveadm-server", ["dovecot-devel01/doveadm-server"], [/* 18 vars */] [pid 30711] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=30932, si_uid=0, si_status=SIGSEGV, si_utime=3, si_stime=1} --- [pid 30711] write(62, "\0", 1) = 1 [pid 30711] read(124, "8x\0\0008\0\0\0\251\3\0\0", 1020) = 12 [pid 30711] read(61, "\0", 64) = 1 [pid 30711] open("/proc/sys/fs/suid_dumpable", O_RDONLY) = 227 [pid 30711] read(227, "0\n", 9) = 2 [pid 30711] write(65, "25 30932 FATAL service(doveadm): child 30932 killed with signal 11 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2)\n", 170) = 170 [pid 30711] write(14, "KILL\t30932\n", 11) = 11 [pid 30711] write(65, "25 30932 BYE\n", 13) = 13 [pid 31339] <... execve resumed> ) = 0 [pid 31339] open("/usr/lib64/dovecot/tls/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) [pid 31339] stat("/usr/lib64/dovecot/tls/x86_64", 0x7ffda7e3f1d0) = -1 ENOENT (No such file or directory) [pid 31339] open("/usr/lib64/dovecot/tls/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) [pid 31339] stat("/usr/lib64/dovecot/tls", 0x7ffda7e3f1d0) = -1 ENOENT (No such file or directory) [pid 31339] open("/usr/lib64/dovecot/x86_64/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) [pid 31339] stat("/usr/lib64/dovecot/x86_64", 0x7ffda7e3f1d0) = -1 ENOENT (No such file or directory) [pid 31339] open("/usr/lib64/dovecot/libz.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory) [pid 31339] stat("/usr/lib64/dovecot", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 [pid 31339] open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 4 [pid 31339] open("/lib64/libz.so.1", O_RDONLY|O_CLOEXEC) = 4

All missing libs are in "/usr/lib64" and no to "/usr/lib64/dovecot"

Can you help me?

Br,

Domenico

*Domenico Pastore* Senior Cloud Engineer *T* 06.98269600 | *M* 347.1474270 | *F* 06.98269680 *Par-Tec http://www.par-tec.ithttp://www.par-tec.it* | *beyond the IT domain* Via Cristoforo Colombo 163, 00147 Roma *CONFIDENZIALE*: Questo messaggio ed i suoi allegati sono di carattere confidenziale. È vietato l'inoltro non autorizzato a destinatari diversi da quelli indicati nel messaggio originale. Se ricevuto per errore si prega di informare il mittente e cancellarlo immediatamente.

Il 12/09/19 12:07, Domenico Pastore via dovecot ha scritto:

Hi,

i have a problem with libclucene-core.so.2.3.3.4. I See segfault error to messages log, when dovecot store mail to INBOX.

set 12 13:41:21 box02.par-tec.it kernel: doveadm-server[20285]: *segfault at 38 ip 00007f575b68fc42 sp 00007ffc0272e750 error 4 in libclucene-core.so.2.3.3.4*[7f575b569000+188000] set 12 13:41:22 box02.par-tec.it kernel: doveadm-server[18871]: segfault at 38 ip 00007f9bea6d2c42 sp 00007fff47bceae0 error 4 in libclucene-core.so.2.3.3.4[7f9bea5ac000+188000]

In the dovecot logs:

2019-09-12 13:45:09 auth: Debug: ldap(test4444client@newsvilXXXX,10.0.8.128mailto:test4444client@newsvilXXXX,10.0.8.128): Finished userdb lookup 2019-09-12 13:45:09 auth: Debug: userdb out: USER 1 test4444client@newsvilXXXX home=/pec_store/pec_mailbox/pec_mailbox_devel01/4/5/6/7/1/45671_18231 quota_rule=*:bytes=1048576000 mail_plugins+= 2019-09-12 13:45:10 SKENO8Uvel0gbAAA5ZKy4w doveadm(test4444client@newsvilXXXX): *Fatal: master: service(doveadm): child 27680 killed with signal 11 (core dumped)* 2019-09-12 13:45:10 auth: Debug: master in: USER 1 test4444client@newsvilXXXX service=doveadm lip=192.160.10.3 lport=924 rip=10.0.8.128 rport=36760 2019-09-12 13:45:10 auth: Debug: ldap(test4444client@newsvilXXXX 10.0.8.128): Performing userdb lookup

I have attach the coredump of error and dovecot.conf

Thanks for support.

Br,

Domenico

------------------- info: ------------------- # 2.3.7.2 (3c910f64b): /pec/config/dovecot/dovecot-devel01/dovecot.conf # Pigeonhole version 0.5.7.2 (7372921a) # OS: Linux 3.10.0-957.27.2.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: box02