finally managed to locate the dump


here the output:

# gdb /usr/libexec/dovecot/auth /var/lib/systemd/coredump/core.auth.0.3a33f56105e043de802a7dfcee265a07.28130.1543516118000000
GNU gdb (GDB) 8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/libexec/dovecot/auth...done.
[New LWP 28130]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `dovecot/auth wor'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f739c35cd7f in raise () from /usr/lib/libc.so.6
(gdb) bt full
#0 0x00007f739c35cd7f in raise () from /usr/lib/libc.so.6
No symbol table info available.
#1 0x00007f739c347672 in abort () from /usr/lib/libc.so.6
No symbol table info available.
#2 0x00007f739c39f878 in __libc_message () from /usr/lib/libc.so.6
No symbol table info available.
#3 0x00007f739c3a618a in malloc_printerr () from /usr/lib/libc.so.6
No symbol table info available.
#4 0x00007f739c3a7b27 in _int_free () from /usr/lib/libc.so.6
No symbol table info available.
#5 0x00007f739cc33585 in mysql_close (mysql=0x5636af7abdc0) at /usr/src/mariadb-10.3.11/libmariadb/libmariadb/mariadb_lib.c:1939
mysql = 0x5636af7abdc0
p = <optimized out>
p = <optimized out>
#6 0x00005636aef08f7c in driver_sqlpool_disconnect (_db=0x5636af7aaf30) at driver-sqlpool.c:590
conn__foreach_end = 0x5636af7ab570
db = 0x5636af7aaf30
conn = 0x5636af7ab560
#7 0x00005636aeefed65 in db_sql_unref (_conn=0x5636af7b0020) at db-sql.c:128
conn = 0x5636af7ae388
#8 0x00005636aeef7e15 in userdb_deinit (userdb=0x5636af7afff8) at userdb.c:191
idx = 0
__func__ = "userdb_deinit"
#9 0x00005636aeede209 in auth_deinit (auth=0x5636af7afd58) at auth.c:335
passdb = <optimized out>
userdb = 0x5636af7aff18
passdb = <optimized out>
userdb = <optimized out>
#10 auths_deinit () at auth.c:433
auth__foreach_end = 0x5636af7ae2f8
auth = 0x5636af7ae2f0
#11 0x00005636aeedcf65 in main_deinit () at main.c:270
l = <optimized out>
l = <optimized out>
l_end = <optimized out>
#12 main (argc=<optimized out>, argv=<optimized out>) at main.c:401
c = <optimized out>

 



On 2018-11-29 18:07, Joan Moreau wrote:

Can't find any "core" files (updatedb ; locate "core"). Coredump are usually in /var/liv/systemd/coredump for other programs, but nothing for dovecot.

Looks like issue is in 'auth' and 'indexer-worker'. Where can be the coredump files ?


 


On 2018-11-28 18:13, Timo Sirainen wrote:

See https://dovecot.org/bugreport.html#coredumps
 
Without a backtrace it's not really possible to figure out where it's crashing.

On 28 Nov 2018, at 13.20, Joan Moreau <jom@grosjo.net> wrote:

Where to get that ?

 

On 2018-11-27 08:50, Aki Tuomi wrote:

It's still missing core dump (or bt full from it)

Aki

On 27.11.2018 8.39, Joan Moreau wrote:

Thank you Aki

here the requested data (below)

Please not as well that we have numerous subfolders (>50) and pretty big mailbox sizes (>20G)

Bug appears mostly in auth process and index-worker


dovecot -n :

# 2.4.devel (de42b54aa): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.6.devel (65909cfa)
# OS: Linux 4.19.4-arch1-1-ARCH x86_64 ext4
# Hostname: gjserver
base_dir = /run/dovecot
default_login_user = dovecot
default_vsz_limit = 16 G
disable_plaintext_auth = no
listen = *
log_path = /var/log/mail/dovecot.log
mail_gid = mail
mail_location = mdbox:/data/mail/%d/%n:ALT=/data/mail/archives/%d/%n
mail_plugins = fts fts_squat
mail_uid = mailusers
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext
mdbox_rotate_size = 24 M

(...)

passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
(the rest default values)

 


On 2018-11-25 08:08, Aki Tuomi wrote:

 
On 25 November 2018 at 06:29 Joan Moreau < jom@grosjo.net> wrote:
 
 
Hi
 
THis is the lines I have in my dmesg (see below)
 
In dovecot log , I see:
 
Nov 25 04:26:47 auth-worker: Error: double free or corruption (fasttop)
 
What do to about it ?
 
Using lastest 2.3.4 version
 
Thank you
 
--------
 
[132932.169265] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 aa 36 f5 7a 7f 00
00 <40> 24 3d f5 7a 7f 00 00 21 80 00 00 00 00 00 00 00 94 16 d6 ee 55
[134031.969596] auth[27031]: segfault at 55e509612c30 ip
000055e509612c30 sp 00007ffeb96dee48 error 15
[134031.969603] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 3a d3 e4 ef 7f 00
00 <40> b4 d9 e4 ef 7f 00 00 21 80 00 00 00 00 00 00 00 04 5f 09 e5 55
[134081.497871] doveadm[28930]: segfault at 7ffe4a16efc8 ip
00007f393841013e sp 00007ffe4a16efc0 error 6 in
libdovecot.so.0.0.0[7f3938363000+e2000]
[134081.497876] Code: 7d be e9 68 ff ff ff e8 10 4c f5 ff 41 57 41 89 cf
41 56 49 89 f6 41 55 41 89 fd 31 ff 41 54 55 44 89 c5 53 89 d3 48 83 ec
58 <4c> 89 4c 24 08 64 48 8b 04 25 28 00 00 00 48 89 44 24 48 31 c0 e8
[134084.145731] doveadm[29186]: segfault at 7fff1cfdbff8 ip
00007f4376e32ffb sp 00007fff1cfdc000 error 6 in
libdovecot.so.0.0.0[7f4376d86000+e2000]
[134084.145735] Code: ff 66 0f 1f 44 00 00 e9 9d dc f6 ff 0f 1f 00 48 83
ec 08 48 83 3d 14 16 0b 00 00 0f 85 d2 76 f6 ff 31 f6 48 8d 3d 05 16 0b
00 <e8> 00 54 f5 ff 85 c0 0f 88 e4 76 f6 ff 48 83 c4 08 e9 69 dc f6 ff
[135453.211242] indexer-worker[2539]: segfault at 7ffec3ba4ff8 ip
00007ffec43fdcff sp 00007ffec3ba5000 error 6
[135453.211245] Code: 95 4c 89 f7 48 89 75 d0 e8 5e fc ff ff 48 8b 75 d0
e9 56 ff ff ff 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55 41 54 49 89
f4 <53> 48 83 ec 08 48 85 ff 0f 84 b3 00 00 00 48 89 fb 4c 8d 35 69 c3
[135453.730250] indexer-worker[9236]: segfault at 7fffed921ff8 ip
00007fb7c9c4f5b1 sp 00007fffed922000 error 6 in
libdovecot.so.0.0.0[7fb7c9ba2000+e2000]
[135453.730256] Code: 2e 0f 1f 84 00 00 00 00 00 41 57 4d 89 cf 41 56 41
89 fe 41 55 49 89 f5 41 54 41 89 d4 55 89 cd 53 48 83 ec 08 4c 8b 4c 24
40 <e8> 6a fb ff ff 85 c0 7e 4f 48 8b 05 7f f9 0a 00 be 38 00 00 00 48
[135796.171575] auth[11121]: segfault at 555f8645cc30 ip
0000555f8645cc30 sp 00007ffcbb510868 error 15
[135796.171586] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 ba ed 38 b7 7f 00
00 <40> 34 f4 38 b7 7f 00 00 21 80 00 00 00 00 00 00 00 a4 43 86 5f 55
[136710.562003] auth[17828]: segfault at 563443604c30 ip
0000563443604c30 sp 00007ffc1aa8b498 error 15
[136710.562013] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 fa 48 da d5 7f 00
00 <40> 74 4f da d5 7f 00 00 21 80 00 00 00 00 00 00 00 24 5e 43 34 56
[138331.686718] auth[31046]: segfault at 55b27bc63c30 ip
000055b27bc63c30 sp 00007ffd5d5b9298 error 15
[138331.686721] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 9a 08 17 cd 7f 00
00 <40> 14 0f 17 cd 7f 00 00 21 80 00 00 00 00 00 00 00 14 c4 7b b2 55
[138521.070485] auth[32481]: segfault at 556e05197c30 ip
0000556e05197c30 sp 00007ffe87217c08 error 15
[138521.070487] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 aa a7 20 3c 7f 00
00 <40> 24 ae 20 3c 7f 00 00 21 80 00 00 00 00 00 00 00 54 17 05 6e 55
[138782.544700] auth[2408]: segfault at 5570d3e46c30 ip 00005570d3e46c30
sp 00007ffc9118a5e8 error 15
[138782.544709] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 da fc 71 11 7f 00
00 <40> 54 03 72 11 7f 00 00 21 80 00 00 00 00 00 00 00 44 e2 d3 70 55
[139113.225511] indexer-worker[3785]: segfault at 7ffe49316ff8 ip
00007ffe49ba7cff sp 00007ffe49317000 error 6
[139113.225514] Code: 95 4c 89 f7 48 89 75 d0 e8 5e fc ff ff 48 8b 75 d0
e9 56 ff ff ff 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55 41 54 49 89
f4 <53> 48 83 ec 08 48 85 ff 0f 84 b3 00 00 00 48 89 fb 4c 8d 35 69 c3
[139113.733885] indexer-worker[5483]: segfault at 7ffcdc00efb8 ip
00007fa68292a13e sp 00007ffcdc00efb0 error 6 in
libdovecot.so.0.0.0[7fa68287d000+e2000]
[139113.733890] Code: 7d be e9 68 ff ff ff e8 10 4c f5 ff 41 57 41 89 cf
41 56 49 89 f6 41 55 41 89 fd 31 ff 41 54 55 44 89 c5 53 89 d3 48 83 ec
58 <4c> 89 4c 24 08 64 48 8b 04 25 28 00 00 00 48 89 44 24 48 31 c0 e8
[139120.612228] indexer-worker[5498]: segfault at 7ffc90beaff8 ip
00007f0e09ae259f sp 00007ffc90beb000 error 6 in
libdovecot.so.0.0.0[7f0e09a35000+e2000]
[139120.612232] Code: c2 48 8b 44 24 08 48 8b 30 31 c0 e8 cb 2d f5 ff 66
66 2e 0f 1f 84 00 00 00 00 00 41 57 4d 89 cf 41 56 41 89 fe 41 55 49 89
f5 <41> 54 41 89 d4 55 89 cd 53 48 83 ec 08 4c 8b 4c 24 40 e8 6a fb ff
[139747.003229] auth[9545]: segfault at 558b5e039c30 ip 0000558b5e039c30
sp 00007ffd43633008 error 15
[139747.003239] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 2a ef 0e 75 7f 00
00 <40> a4 f5 0e 75 7f 00 00 21 80 00 00 00 00 00 00 00 74 01 5e 8b 55
[140640.900512] auth[17221]: segfault at 55e952d41c30 ip
000055e952d41c30 sp 00007ffc36fa0908 error 15
[140640.900523] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 5a ec 80 b9 7f 00
00 <40> d4 f2 80 b9 7f 00 00 21 80 00 00 00 00 00 00 00 f4 d1 52 e9 55
[140777.927292] auth[18114]: segfault at 55bd72adec30 ip
000055bd72adec30 sp 00007ffca394ba98 error 15
[140777.927302] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 ba cb 67 d7 7f 00
00 <40> 34 d2 67 d7 7f 00 00 21 80 00 00 00 00 00 00 00 c4 ab 72 bd 55
 
---
Aki