imap-login segfaulting on 2.3.1
Hello,
I have a new director ring I am setting up on centos 7 with dovecot 2.3.1. I haven't been able to replecate this in testing, but as soon as I start pushing production traffic to the new ring I see dozens of these in the logs: Apr 18 00:34:00 d.director.imapd.sonic.net kernel: imap-login[163107]: segfault at 10 ip 00007ff625698dd5sp 00007ffe4b77bb28 error 4 in libdovecot.so.0.0.0[7ff6255bf000+16e000]
My config: # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-693.21.1.el7.x86_64 x86_64 CentOS Linux release 7.4.1708 (Core) # Hostname: c.director.imapd.sonic.net auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes default_client_limit = 20000 director_mail_servers = 64.142.111.75 64.142.111.43 64.142.111.76 64.142.111.39 64.142.111.81 director_servers = c.director.imapd.sonic.net d.director.imapd.sonic.net disable_plaintext_auth = no doveadm_port = 1842 login_trusted_networks = 64.142.18.0/24 64.142.105.2 64.142.111.67 64.142.111.36 64.142.105.3 64.142.111.35 64.142.111.68 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 40103 } service auth { client_limit = 41704 } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9321 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 20000 process_min_avail = 32 } service imap { process_limit = 20480 } service ipc { unix_listener ipc { user = dovecot } } service pop3-login { executable = pop3-login director process_limit = 20000 process_min_avail = 32 } ssl_cert =
backtrace: Stack trace of thread 56084: #0 0x00007fe529be2dd5 i_stream_get_root_io (libdovecot.so.0) #1 0x00007fe529be2e39 i_stream_set_input_pending (libdovecot.so.0) #2 0x00007fe527415a59 openssl_iostream_bio_sync (libssl_iostream_openssl.so) #3 0x00007fe527415c2a openssl_iostream_more (libssl_iostream_openssl.so) #4 0x00007fe527415c6b openssl_iostream_destroy (libssl_iostream_openssl.so) #5 0x00007fe529e8be72 client_unref (libdovecot-login.so.0) #6 0x00007fe529e90441 login_proxy_free_full (libdovecot-login.so.0) #7 0x00007fe529e90e07 login_proxy_finished (libdovecot-login.so.0) #8 0x00007fe529becfb5 io_loop_call_io (libdovecot.so.0) #9 0x00007fe529bee89f io_loop_handler_run_internal (libdovecot.so.0) #10 0x00007fe529bed0b2 io_loop_handler_run (libdovecot.so.0) #11 0x00007fe529bed2d8 io_loop_run (libdovecot.so.0) #12 0x00007fe529b69673 master_service_run (libdovecot.so.0) #13 0x00007fe529e93532 login_binary_run (libdovecot-login.so.0) #14 0x00007fe529767c05 __libc_start_main (libc.so.6) #15 0x00005567b256050a _start (imap-login)
Grant Keller System Operations grant.keller@sonic.com
On 24.04.2018 00:40, Grant Keller wrote:
Hello,
I have a new director ring I am setting up on centos 7 with dovecot 2.3.1. I haven't been able to replecate this in testing, but as soon as I start pushing production traffic to the new ring I see dozens of these in the logs: Apr 18 00:34:00 d.director.imapd.sonic.net kernel: imap-login[163107]: segfault at 10 ip 00007ff625698dd5sp 00007ffe4b77bb28 error 4 in libdovecot.so.0.0.0[7ff6255bf000+16e000]
My config: # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-693.21.1.el7.x86_64 x86_64 CentOS Linux release 7.4.1708 (Core) # Hostname: c.director.imapd.sonic.net auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes default_client_limit = 20000 director_mail_servers = 64.142.111.75 64.142.111.43 64.142.111.76 64.142.111.39 64.142.111.81 director_servers = c.director.imapd.sonic.net d.director.imapd.sonic.net disable_plaintext_auth = no doveadm_port = 1842 login_trusted_networks = 64.142.18.0/24 64.142.105.2 64.142.111.67 64.142.111.36 64.142.105.3 64.142.111.35 64.142.111.68 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 40103 } service auth { client_limit = 41704 } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9321 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 20000 process_min_avail = 32 } service imap { process_limit = 20480 } service ipc { unix_listener ipc { user = dovecot } } service pop3-login { executable = pop3-login director process_limit = 20000 process_min_avail = 32 } ssl_cert =
backtrace: Stack trace of thread 56084: #0 0x00007fe529be2dd5 i_stream_get_root_io (libdovecot.so.0) #1 0x00007fe529be2e39 i_stream_set_input_pending (libdovecot.so.0) #2 0x00007fe527415a59 openssl_iostream_bio_sync (libssl_iostream_openssl.so) #3 0x00007fe527415c2a openssl_iostream_more (libssl_iostream_openssl.so) #4 0x00007fe527415c6b openssl_iostream_destroy (libssl_iostream_openssl.so) #5 0x00007fe529e8be72 client_unref (libdovecot-login.so.0) #6 0x00007fe529e90441 login_proxy_free_full (libdovecot-login.so.0) #7 0x00007fe529e90e07 login_proxy_finished (libdovecot-login.so.0) #8 0x00007fe529becfb5 io_loop_call_io (libdovecot.so.0) #9 0x00007fe529bee89f io_loop_handler_run_internal (libdovecot.so.0) #10 0x00007fe529bed0b2 io_loop_handler_run (libdovecot.so.0) #11 0x00007fe529bed2d8 io_loop_run (libdovecot.so.0) #12 0x00007fe529b69673 master_service_run (libdovecot.so.0) #13 0x00007fe529e93532 login_binary_run (libdovecot-login.so.0) #14 0x00007fe529767c05 __libc_start_main (libc.so.6) #15 0x00005567b256050a _start (imap-login)
Grant Keller System Operations grant.keller@sonic.com
Hi!
Can you provide bt full output?
Aki
Sorry about that, here it is:
(gdb) bt full
#0 i_stream_get_root_io (stream=0x0) at istream.c:911
No locals.
#1 0x00007f963a47de39 in i_stream_set_input_pending (stream=<optimized out>, pending=pending@entry=true)
at istream.c:923
No locals.
#2 0x00007f9637cb0a59 in openssl_iostream_bio_input (type=OPENSSL_IOSTREAM_SYNC_TYPE_HANDSHAKE,
ssl_io=0x5615d14844d0) at iostream-openssl.c:498
data = 0x7f963a4d00bd
Quoting Aki Tuomi (2018-04-23 23:55:24)
On 24.04.2018 00:40, Grant Keller wrote:
Hello,
I have a new director ring I am setting up on centos 7 with dovecot 2.3.1. I haven't been able to replecate this in testing, but as soon as I start pushing production traffic to the new ring I see dozens of these in the logs: Apr 18 00:34:00 d.director.imapd.sonic.net kernel: imap-login[163107]: segfault at 10 ip 00007ff625698dd5sp 00007ffe4b77bb28 error 4 in libdovecot.so.0.0.0[7ff6255bf000+16e000]
My config: # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-693.21.1.el7.x86_64 x86_64 CentOS Linux release 7.4.1708 (Core) # Hostname: c.director.imapd.sonic.net auth_master_user_separator = * auth_username_format = %Ln auth_verbose = yes default_client_limit = 20000 director_mail_servers = 64.142.111.75 64.142.111.43 64.142.111.76 64.142.111.39 64.142.111.81 director_servers = c.director.imapd.sonic.net d.director.imapd.sonic.net disable_plaintext_auth = no doveadm_port = 1842 login_trusted_networks = 64.142.18.0/24 64.142.105.2 64.142.111.67 64.142.111.36 64.142.105.3 64.142.111.35 64.142.111.68 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = proxy=y nopassword=y driver = static } service anvil { client_limit = 40103 } service auth { client_limit = 41704 } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9321 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director process_limit = 20000 process_min_avail = 32 } service imap { process_limit = 20480 } service ipc { unix_listener ipc { user = dovecot } } service pop3-login { executable = pop3-login director process_limit = 20000 process_min_avail = 32 } ssl_cert =
backtrace: Stack trace of thread 56084: #0 0x00007fe529be2dd5 i_stream_get_root_io (libdovecot.so.0) #1 0x00007fe529be2e39 i_stream_set_input_pending (libdovecot.so.0) #2 0x00007fe527415a59 openssl_iostream_bio_sync (libssl_iostream_openssl.so) #3 0x00007fe527415c2a openssl_iostream_more (libssl_iostream_openssl.so) #4 0x00007fe527415c6b openssl_iostream_destroy (libssl_iostream_openssl.so) #5 0x00007fe529e8be72 client_unref (libdovecot-login.so.0) #6 0x00007fe529e90441 login_proxy_free_full (libdovecot-login.so.0) #7 0x00007fe529e90e07 login_proxy_finished (libdovecot-login.so.0) #8 0x00007fe529becfb5 io_loop_call_io (libdovecot.so.0) #9 0x00007fe529bee89f io_loop_handler_run_internal (libdovecot.so.0) #10 0x00007fe529bed0b2 io_loop_handler_run (libdovecot.so.0) #11 0x00007fe529bed2d8 io_loop_run (libdovecot.so.0) #12 0x00007fe529b69673 master_service_run (libdovecot.so.0) #13 0x00007fe529e93532 login_binary_run (libdovecot-login.so.0) #14 0x00007fe529767c05 __libc_start_main (libc.so.6) #15 0x00005567b256050a _start (imap-login)
Grant Keller System Operations grant.keller@sonic.com
Hi!
Can you provide bt full output?
Aki
Grant Keller System Operations grant.keller@sonic.com
participants (2)
-
Aki Tuomi
-
Grant Keller