Timo Sirainen <mailto:tss@iki.fi> October 2, 2012 4:12 PM
Awesome!
Will this fix make it into 2.1.11, or should I toy with incorporating your change and compiling a new copy of 2.1.10 by hand? Also, will the seg fault have caused performance issues in breaking IDLE connections? Just wondering what sort of impact this fix might have...
Thanks for working on this, it is most appreciated, I'm a big fan of Dovecot!
Joe Auty <mailto:joe@netmusician.org> September 24, 2012 3:03 PM
Timo Sirainen <mailto:tss@iki.fi> September 24, 2012 10:32 AM
Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again?
Thanks Timo, I have done so. Here is the results of my debugging info now:
gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/lib/dovecot/imap-login...Reading symbols from /usr/lib/debug/usr/lib/dovecot/imap-login...done. (no debugging symbols found)...done.
warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...Reading symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot-login.so.0.0.0...done. (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...Reading symbols from /usr/lib/debug/usr/lib/dovecot/libdovecot.so.0.0.0...done. (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Core was generated by `dovecot/imap-login ?'. Program terminated with signal 11, Segmentation fault. #0 hash_table_destroy (_table=0x28) at hash.c:106 106 hash.c: No such file or directory. in hash.c (gdb) bt full #0 hash_table_destroy (_table=0x28) at hash.c:106 table = <value optimized out> #1 0x00007ff300721054 in settings_parser_deinit (_ctx=<value optimized out>) at settings-parser.c:237 ctx = 0x0 #2 0x00007ff30074633d in master_service_settings_cache_deinit (_cache=<value optimized out>) at master-service-settings-cache.c:86 cache = 0x9f9a60 entry = 0xa016e0 next = 0x0 __FUNCTION__ = "master_service_settings_cache_deinit" #3 0x00007ff3009a5018 in main_deinit (binary=<value optimized out>, argc=2, argv=0x9f8370) at main.c:355 No locals. #4 login_binary_run (binary=<value optimized out>, argc=2, argv=0x9f8370) at main.c:407 set_pool = 0x9f8a30 allow_core_dumps = <value optimized out> login_socket = <value optimized out> c = <value optimized out> #5 0x00007ff3003c0c8d in __libc_start_main () from /lib/libc.so.6 No symbol table info available. #6 0x0000000000402459 in _start () No symbol table info available. (gdb)
Joe Auty <mailto:joe@netmusician.org> September 23, 2012 7:05 AM
Timo Sirainen <mailto:tss@iki.fi> September 23, 2012 5:58 AM
You should have a similar log line about the crash in mail.log (or wherever "doveadm log find" says that errors get logged). Find those lines, then configure login processes to dump core files. This probably should work:
service imap-login { executable = imap-login -D }
Next time it crashes hopefully you'll have /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it:
gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core bt full
I hope I'm doing this correctly!
# gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/lib/dovecot/imap-login...(no debugging symbols found)...done.
warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Core was generated by `dovecot/imap-login ?'. Program terminated with signal 11, Segmentation fault. #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x00007f789cf5e018 in login_binary_run () from /usr/lib/dovecot/libdovecot-login.so.0 No symbol table info available. #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 No symbol table info available. #5 0x0000000000402459 in ?? () No symbol table info available. #6 0x00007fff8a9c65f8 in ?? () No symbol table info available. #7 0x000000000000001c in ?? () No symbol table info available. #8 0x0000000000000002 in ?? () No symbol table info available. #9 0x00007fff8a9c7e6a in ?? () No symbol table info available. #10 0x00007fff8a9c7e7d in ?? () No symbol table info available. #11 0x0000000000000000 in ?? () No symbol table info available.
Joe Auty <mailto:joe@netmusician.org> September 22, 2012 3:57 PM Hello,
I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this?
I'm using the 2.1.10 packages obtained with the following in my sources.list:
deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main
I need to use Dovecot 2.1.x because I need to support handling multiple SSL certs.
Timo Sirainen <mailto:tss@iki.fi> September 23, 2012 5:58 AM
You should have a similar log line about the crash in mail.log (or wherever "doveadm log find" says that errors get logged). Find those lines, then configure login processes to dump core files. This probably should work:
service imap-login { executable = imap-login -D }
Next time it crashes hopefully you'll have /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it:
gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core bt full
Joe Auty <mailto:joe@netmusician.org> September 22, 2012 3:57 PM Hello,
I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this?
I'm using the 2.1.10 packages obtained with the following in my sources.list:
deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main
I need to use Dovecot 2.1.x because I need to support handling multiple SSL certs.
Timo Sirainen <mailto:tss@iki.fi> September 24, 2012 10:32 AM
Well, the good news is that it crashes only after it has already disconnected the client anyway. But I thought I fixed this bug in v2.1.10 and I'm not able to reproduce it myself.. Having debugging information available might show something useful. Try installing dovecot-dbg package and getting the bt full again?
Joe Auty <mailto:joe@netmusician.org> September 23, 2012 7:05 AM
Timo Sirainen <mailto:tss@iki.fi> September 23, 2012 5:58 AM
You should have a similar log line about the crash in mail.log (or wherever "doveadm log find" says that errors get logged). Find those lines, then configure login processes to dump core files. This probably should work:
service imap-login { executable = imap-login -D }
Next time it crashes hopefully you'll have /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it:
gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core bt full
I hope I'm doing this correctly!
# gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/lib/dovecot/imap-login...(no debugging symbols found)...done.
warning: Can't read pathname for load map: Input/output error. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Core was generated by `dovecot/imap-login ?'. Program terminated with signal 11, Segmentation fault. #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00007f789cd08e14 in hash_table_destroy () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00007f789ccda054 in settings_parser_deinit () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x00007f789ccff33d in master_service_settings_cache_deinit () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x00007f789cf5e018 in login_binary_run () from /usr/lib/dovecot/libdovecot-login.so.0 No symbol table info available. #4 0x00007f789c979c8d in __libc_start_main () from /lib/libc.so.6 No symbol table info available. #5 0x0000000000402459 in ?? () No symbol table info available. #6 0x00007fff8a9c65f8 in ?? () No symbol table info available. #7 0x000000000000001c in ?? () No symbol table info available. #8 0x0000000000000002 in ?? () No symbol table info available. #9 0x00007fff8a9c7e6a in ?? () No symbol table info available. #10 0x00007fff8a9c7e7d in ?? () No symbol table info available. #11 0x0000000000000000 in ?? () No symbol table info available.
Joe Auty <mailto:joe@netmusician.org> September 22, 2012 3:57 PM Hello,
I'm seeing a lot of these in my /var/log/messages in Debian Squeeze, I suspect this might be causing performance issues. Any suggestions what I can try to fix this?
I'm using the 2.1.10 packages obtained with the following in my sources.list:
deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.1 main
I need to use Dovecot 2.1.x because I need to support handling multiple SSL certs.
Timo Sirainen <mailto:tss@iki.fi> September 23, 2012 5:58 AM
You should have a similar log line about the crash in mail.log (or wherever "doveadm log find" says that errors get logged). Find those lines, then configure login processes to dump core files. This probably should work:
service imap-login { executable = imap-login -D }
Next time it crashes hopefully you'll have /var/run/dovecot/login/core* file(s). Get a gdb backtrace from it send it:
gdb /usr/lib/dovecot/imap-login /var/run/dovecot/login/core bt full