[Dovecot] worth a few Euros?
Yo Timo, did you say that login processes "just don't crash"? Found these while doing load/malformed data tests. Got the kernel patch allow_setid_cores working. Don't have time to check if they're exploitable. Probably time to get a gamma version out for people, eh?
Cheers, Jared :)
=========================================================================================== Note: Running dovecot-1.0.beta2 with simple (no funniness) config Note: Running on Intel (in VmWare) with RH9. Note: I'm using shadow authentication, but tried with PAM and it still dumped. Note: For this first core, I see the server responding with: "*OK Waiting for authentication process to respond.." Note: Not sure, but I think these problems are timing related. Be sure to check if a pointer is valid and that will get rid of most of these. Note: I have plenty of sample cores if you need one to fix. -Jared DeMott
[root@server dovecot]# gdb ./dovecot-auth -c core.22753 GNU gdb Red Hat Linux (5.3post-0.20021129.18rh) Copyright 2003 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... Core was generated by `dovecot-auth'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libcrypt.so.1...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /lib/libpam.so.0...done. Loaded symbols for /lib/libpam.so.0 Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/tls/libc.so.6...done. Loaded symbols for /lib/tls/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 #0 auth_request_unref (_request=0x807b0f0) at auth-request.c:96 96 i_assert(request->refcount > 0); (gdb) bt #0 auth_request_unref (_request=0x807b0f0) at auth-request.c:96 #1 0x080523ee in auth_request_handler_unref (_handler=0x807b0f0) at auth-request-handler.c:66 #2 0x08050217 in auth_client_connection_destroy (_conn=0x807b0f0) at auth-client-connection.c:327 #3 0x0804ffd8 in auth_client_input (context=0x807b0f0) at auth-client-connection.c:227 #4 0x080607d0 in io_loop_handler_run (ioloop=0x8073100) at ioloop-poll.c:189 #5 0x0805fe29 in io_loop_run (ioloop=0x8073100) at ioloop.c:235 #6 0x08055329 in main (argc=1, argv=0xbfffe454) at main.c:309 #7 0x42015574 in __libc_start_main () from /lib/tls/libc.so.6 (gdb) p request $1 = (struct auth_request *) 0x1 (gdb) i r eax 0x807b0f0 134721776 ecx 0x1 1 edx 0x6c 108 ebx 0x8084e78 134762104 esp 0xbfffe2f0 0xbfffe2f0 ebp 0xbfffe2f8 0xbfffe2f8 esi 0x80a43c8 134890440 edi 0xbfffe314 -1073749228 eip 0x80512d1 0x80512d1 eflags 0x286 646 cs 0x23 35 ss 0x2b 43 ds 0x2b 43 es 0x2b 43 fs 0x0 0 gs 0x33 51 (gdb)
[root@server 1]# gdb ./dovecot-auth -c core.30100 GNU gdb Red Hat Linux (5.3post-0.20021129.18rh) Copyright 2003 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...
warning: exec file is newer than core file. Core was generated by `dovecot-auth'. Program terminated with signal 6, Aborted. #0 0xffffe002 in ?? () (gdb) bt #0 0xffffe002 in ?? () #1 0x42028a73 in ?? () #2 0x0805d47c in default_info_handler (format=0x8067d00 "\024P?u\024?u\020?u\f?u\b?\t", args=0xbffff264 "z|\006\b!") at failures.c:162 #3 0x0805d04b in t_buffer_alloc (size=1108544020) at data-stack.c:347 #4 0x0804fa54 in reply_line_hide_pass (line=0x1
<Address 0x1 out of bounds>) at auth-client-connection.c:35 #5 0x0804fa71 in reply_line_hide_pass (line=0x0) at auth-client-connection.c:36 #6 0x080523bb in auth_request_handler_unref (_handler=0x0) at auth-request-handler.c:74 #7 0x08052b65 in auth_request_handler_auth_continue (handler=0x0, args=0x80774e0 "HU\n\b") at auth-request-handler.c:350 #8 0x0805f916 in io_add (fd=2, condition=39, callback=0x80778e8, context=0x0) at ioloop.c:23 #9 0x080602c8 in io_loop_notify_remove (ioloop=0x0, io=0x8075b00) at ioloop-notify-dn.c:138 #10 0x0805f985 in io_add (fd=0, condition=1073792608, callback=0x80678bc <pwrite_full+64>, context=0x0) at ioloop.c:43 #11 0x08054f94 in add_extra_listeners () at main.c:148 #12 0x42015574 in ?? () (gdb) i r eax 0x0 0 ecx 0x6 6 edx 0x42130a14 1108544020 ebx 0x7594 30100 esp 0xbffff0e0 0xbffff0e0 ebp 0xbffff0e8 0xbffff0e8 esi 0x806daa8 134666920 edi 0x80e92e8 135172840 eip 0xffffe002 0xffffe002 eflags 0x246 582 cs 0x23 35 ss 0x2b 43 ds 0x2b 43 es 0x2b 43 fs 0x0 0 gs 0x33 51 (gdb) ======================================================================================== [root@server login]# gdb ./imap-login -c core.25116 GNU gdb Red Hat Linux (5.3post-0.20021129.18rh) Copyright 2003 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"... Core was generated by `imap-login'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libssl.so.4...done. Loaded symbols for /lib/libssl.so.4 Reading symbols from /lib/libcrypto.so.4...done. Loaded symbols for /lib/libcrypto.so.4 Reading symbols from /lib/libresolv.so.2...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /usr/kerberos/lib/libgssapi_krb5.so.2...done. Loaded symbols for /usr/kerberos/lib/libgssapi_krb5.so.2 Reading symbols from /usr/kerberos/lib/libkrb5.so.3...done. Loaded symbols for /usr/kerberos/lib/libkrb5.so.3 Reading symbols from /usr/kerberos/lib/libk5crypto.so.3...done. Loaded symbols for /usr/kerberos/lib/libk5crypto.so.3 Reading symbols from /usr/kerberos/lib/libcom_err.so.3...done. Loaded symbols for /usr/kerberos/lib/libcom_err.so.3 Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /usr/lib/libz.so.1...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /lib/tls/libc.so.6...done. Loaded symbols for /lib/tls/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 #0 i_stream_read (stream=0x0) at istream.c:48 48 if (stream->closed) (gdb) bt #0 i_stream_read (stream=0x0) at istream.c:48 #1 0x0804ade2 in client_read (client=0x8070410) at client.c:311 #2 0x0804ae3c in client_input (context=0x8070410) at client.c:333 #3 0x08055558 in io_loop_handler_run (ioloop=0x806cc50) at ioloop-poll.c:189 #4 0x08054bb1 in io_loop_run (ioloop=0x806cc50) at ioloop.c:235 #5 0x0804d3ed in main (argc=1, argv=0xbffff504, envp=0xbffff50c) at main.c:341 #6 0x42015574 in __libc_start_main () from /lib/tls/libc.so.6 (gdb) p stream $1 = (struct istream *) 0x0 (gdb) ====================================================================================== [root@server core]# gdb ./imap -c core.15043 GNU gdb Red Hat Linux (5.3post-0.20021129.18rh) Copyright 2003 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux-gnu"...
warning: exec file is newer than core file. Core was generated by `imap'. Program terminated with signal 6, Aborted. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/tls/libc.so.6...done. Loaded symbols for /lib/tls/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libnss_files.so.2...done. Loaded symbols for /lib/libnss_files.so.2 #0 0xffffe002 in ?? () (gdb) bt #0 0xffffe002 in ?? () #1 0x42028a73 in abort () from /lib/tls/libc.so.6 #2 0x0809f9fc in i_internal_panic_handler (fmt=0x80ac360 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfffdd94 "\0251\v\b?") at failures.c:375 #3 0x0809f5cb in i_panic (format=0x80ac360 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:173 #4 0x08082277 in mail_cache_transaction_reserve_more (ctx=0x80cb4a0, block_size=332, commit=true) at mail-cache-transaction.c:257 #5 0x0808248a in mail_cache_transaction_get_space (ctx=0x80cb4a0, min_size=332, max_size=332, offset_r=0x0, available_space_r=0x0, commit=true) at mail-cache-transaction.c:347 #6 0x08082b90 in mail_cache_header_add_field (ctx=0x80cb4a0, field=0) at mail-cache-transaction.c:646 #7 0x08082e0e in mail_cache_add (ctx=0x80cb4a0, seq=1, field=9, data=0x0, data_size=0) at mail-cache-transaction.c:693 #8 0x0807b314 in index_mail_cache_add (mail=0x80cfca0, field=9, data=0x0, data_size=0) at index-mail.c:349 #9 0x0807c799 in index_mail_parse_header_finish (mail=0x80cfca0) at index-mail-headers.c:75 #10 0x0807ce0c in index_mail_parse_header (part=0x42130a14, hdr=0x0, mail=0x80cfca0) at index-mail-headers.c:277 #11 0x0809b3d1 in message_parse_part_header (parser_ctx=0x80d7be8) at message-parser.c:253 #12 0x0809bb01 in message_parser_parse_header (ctx=0x80d7be8, hdr_size=0x80cfd74, callback=0, context=0x0) at message-parser.c:608 #13 0x0807cf18 in index_mail_parse_headers (mail=0x80cfca0, headers=0x80d4088) at index-mail-headers.c:361 #14 0x0807d42f in index_mail_get_headers (_mail=0x80cfca0, field=0x80b0ff5 "Date") at index-mail-headers.c:535 #15 0x0807d4d9 in index_mail_get_first_header (mail=0x80cfca0, field=0x80b0ff5 "Date") at index-mail-headers.c:576 #16 0x080933a4 in mail_get_first_header (mail=0x80d4088, field=0x80b0ff5 "Date") at mail.c:82 #17 0x0807b054 in index_mail_get_date (_mail=0x80cfca0, timezone=0x0) at index-mail.c:253 #18 0x08093370 in mail_get_date (mail=0x80d4088, timezone=0x0) at mail.c:61 #19 0x0805fe97 in mail_thread_input (ctx=0x80bd580, mail=0x80cfca0) at imap-thread.c:456 #20 0x0805f933 in imap_thread (cmd=0x80c37f4, charset=0x80c3b00 "US-ASCII", args=0x80cc628, type=MAIL_THREAD_REFERENCES) at imap-thread.c:143 #21 0x08059863 in cmd_thread (cmd=0x80c37f4) at cmd-thread.c:66 #22 0x08059949 in cmd_uid (cmd=0x80c37f4) at cmd-uid.c:19 #23 0x0805a1a7 in client_handle_input (cmd=0x80c37f4) at client.c:355 #24 0x0805a2c6 in _client_input (context=0x80c37b0) at client.c:406 #25 0x080a4ba8 in io_loop_handler_run (ioloop=0x80c1740) at ioloop-poll.c:189 #26 0x080a4201 in io_loop_run (ioloop=0x80c1740) at ioloop.c:235 #27 0x08061053 in main (argc=1, argv=0xbfffe2d4, envp=0xbfffe2dc) at main.c:238 #28 0x42015574 in __libc_start_main () from /lib/tls/libc.so.6 (gdb) i r eax 0x0 0 ecx 0x6 6 edx 0x42130a14 1108544020 ebx 0x3ac3 15043 esp 0xbfffdc10 0xbfffdc10 ebp 0xbfffdc18 0xbfffdc18 esi 0x80c9cb8 135044280 edi 0x14c 332 eip 0xffffe002 0xffffe002 eflags 0x246 582 cs 0x23 35 ss 0x2b 43 ds 0x2b 43 es 0x2b 43 fs 0x0 0 gs 0x33 51 (gdb)
On 28.1.2006 21:33, "-- --" demottjar@yahoo.com wrote:
Yo Timo, did you say that login processes "just don't crash"? Found these while doing load/malformed data tests. Got the kernel patch allow_setid_cores working. Don't have time to check if they're exploitable. Probably time to get a gamma version out for people, eh?
Any suggestions how I could reproduce these myself? I have this one simple stress test tool myself, but I couldn't get it to cause any crashes.
related. Be sure to check if a pointer is valid and that will get rid of most of these.
That won't help. If any of these crashes happen, the memory is already corrupted and there's no point in trying to avoid crashing.
Note: I have plenty of sample cores if you need one to fix.
Those could be helpful, but I'll also need the exact same binaries that produced them.
[root@server dovecot]# gdb ./dovecot-auth -c .. #0 auth_request_unref (_request=0x807b0f0) at auth-request.c:96
This happened if client did an "AUTHENTICATE" command and then disconnected. Got broken with the large changes I did for beta1 (void pointers are evil). Not exploitable, always crashes while trying to dereference 0x1 pointer. Fixed in CVS.
[root@server 1]# gdb ./dovecot-auth -c core.30100 .. warning: exec file is newer than core file.
So this didn't help, as the core file must be used against the same binary or everything will just show garbage. But I'd guess it's the same problem as above.
[root@server login]# gdb ./imap-login -c core.25116 .. #0 i_stream_read (stream=0x0) at istream.c:48 48 if (stream->closed) (gdb) bt #0 i_stream_read (stream=0x0) at istream.c:48 #1 0x0804ade2 in client_read (client=0x8070410) at client.c:311 #2 0x0804ae3c in client_input (context=0x8070410) at client.c:333
Hmm. Have to look into this. Looking at the code, it seems that client structure was unreferenced too many times, but client_destroy() was never called.
The reference counting code in authentication is pretty horrible. Guess I'll have to clean it up.
[root@server core]# gdb ./imap -c core.15043 .. warning: exec file is newer than core file.
Again this problem, so I don't know how much I can rely on this backtrace.
#1 0x42028a73 in abort () from /lib/tls/libc.so.6 #2 0x0809f9fc in i_internal_panic_handler (fmt=0x80ac360 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfffdd94 "\0251\v\b?") at failures.c:375 #3 0x0809f5cb in i_panic (format=0x80ac360 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:173 #4 0x08082277 in mail_cache_transaction_reserve_more (ctx=0x80cb4a0, block_size=332, commit=true) at mail-cache-transaction.c:257
There's no assert call in line 257. The closest one is at line 252, but I couldn't see how it could happen. I did some cleaning up for that code now, but I didn't do any changes for the logic itself.
On Sat, 2006-01-28 at 23:11 +0200, Timo Sirainen wrote:
#0 i_stream_read (stream=0x0) at istream.c:48 48 if (stream->closed) (gdb) bt #0 i_stream_read (stream=0x0) at istream.c:48 #1 0x0804ade2 in client_read (client=0x8070410) at client.c:311 #2 0x0804ae3c in client_input (context=0x8070410) at client.c:333
Hmm. Have to look into this. Looking at the code, it seems that client structure was unreferenced too many times, but client_destroy() was never called.
I found one way to crash it, but I'm not sure if it's the same problem as here. Fixed it anyway and added some more asserts to catch the bug earlier.
So not to rush you, I understand you're busy, but when can we expect a fix for these bugs?
Timo Sirainen wrote:
On Sat, 2006-01-28 at 23:11 +0200, Timo Sirainen wrote:
#0 i_stream_read (stream=0x0) at istream.c:48 48 if (stream->closed) (gdb) bt #0 i_stream_read (stream=0x0) at istream.c:48 #1 0x0804ade2 in client_read (client=0x8070410) at client.c:311 #2 0x0804ae3c in client_input (context=0x8070410) at client.c:333
Hmm. Have to look into this. Looking at the code, it seems that client structure was unreferenced too many times, but client_destroy() was never called.
I found one way to crash it, but I'm not sure if it's the same problem as here. Fixed it anyway and added some more asserts to catch the bug earlier.
On Sun, 2006-01-29 at 08:36 -0500, Jared wrote:
So not to rush you, I understand you're busy, but when can we expect a fix for these bugs?
For the two that I found, they're already fixed in CVS, and I'll probably release beta3 soon. The others I can't reproduce, so if you want them fixed, I'd need some description how to reproduce them..
So, which two did you fix, and which two can't you reproduce?
Timo Sirainen wrote:
On Sun, 2006-01-29 at 08:36 -0500, Jared wrote:
So not to rush you, I understand you're busy, but when can we expect a fix for these bugs?
For the two that I found, they're already fixed in CVS, and I'll probably release beta3 soon. The others I can't reproduce, so if you want them fixed, I'd need some description how to reproduce them..
On Sun, 2006-01-29 at 19:48 -0500, Jared wrote:
So, which two did you fix, and which two can't you reproduce?
dovecot-auth crash fixed. imap/pop3-login crash maybe fixed (I fixed one anyway, but not sure if it's the same as what you saw). imap crashes not fixed.
Could you try again if you can reproduce the bugs with:
Timo Sirainen wrote:
On Sun, 2006-01-29 at 19:48 -0500, Jared wrote:
So, which two did you fix, and which two can't you reproduce?
dovecot-auth crash fixed. imap/pop3-login crash maybe fixed (I fixed one anyway, but not sure if it's the same as what you saw). imap crashes not fixed.
Could you try again if you can reproduce the bugs with:
Ok, I'll give it a go and email you directly with my results.
Jared
Timo Sirainen wrote:
On Sun, 2006-01-29 at 19:48 -0500, Jared wrote:
So, which two did you fix, and which two can't you reproduce?
dovecot-auth crash fixed. imap/pop3-login crash maybe fixed (I fixed one anyway, but not sure if it's the same as what you saw). imap crashes not fixed.
Could you try again if you can reproduce the bugs with:
Dovecot-auth and the imap-login crashes listed earlier in this post have now been fixed according to my testing of this lastest binary.
Don't worry about the "warning: exec file is newer than core file." It only said that becuae I copied the binary to the dir after the core instead of having to keep typing the path to the real location of the binary.
Timo Sirainen wrote:
On 28.1.2006 21:33, "-- --" demottjar@yahoo.com wrote:
Yo Timo, did you say that login processes "just don't crash"? Found these while doing load/malformed data tests. Got the kernel patch allow_setid_cores working. Don't have time to check if they're exploitable. Probably time to get a gamma version out for people, eh?
Any suggestions how I could reproduce these myself? I have this one simple stress test tool myself, but I couldn't get it to cause any crashes.
related. Be sure to check if a pointer is valid and that will get rid of most of these.
That won't help. If any of these crashes happen, the memory is already corrupted and there's no point in trying to avoid crashing.
Note: I have plenty of sample cores if you need one to fix.
Those could be helpful, but I'll also need the exact same binaries that produced them.
[root@server dovecot]# gdb ./dovecot-auth -c
..
#0 auth_request_unref (_request=0x807b0f0) at auth-request.c:96
This happened if client did an "AUTHENTICATE" command and then disconnected. Got broken with the large changes I did for beta1 (void pointers are evil). Not exploitable, always crashes while trying to dereference 0x1 pointer. Fixed in CVS.
[root@server 1]# gdb ./dovecot-auth -c core.30100
..
warning: exec file is newer than core file.
So this didn't help, as the core file must be used against the same binary or everything will just show garbage. But I'd guess it's the same problem as above.
[root@server login]# gdb ./imap-login -c core.25116
..
#0 i_stream_read (stream=0x0) at istream.c:48 48 if (stream->closed) (gdb) bt #0 i_stream_read (stream=0x0) at istream.c:48 #1 0x0804ade2 in client_read (client=0x8070410) at client.c:311 #2 0x0804ae3c in client_input (context=0x8070410) at client.c:333
Hmm. Have to look into this. Looking at the code, it seems that client structure was unreferenced too many times, but client_destroy() was never called.
The reference counting code in authentication is pretty horrible. Guess I'll have to clean it up.
[root@server core]# gdb ./imap -c core.15043
..
warning: exec file is newer than core file.
Again this problem, so I don't know how much I can rely on this backtrace.
#1 0x42028a73 in abort () from /lib/tls/libc.so.6 #2 0x0809f9fc in i_internal_panic_handler (fmt=0x80ac360 "file %s: line %d (%s): assertion failed: (%s)", args=0xbfffdd94 "\0251\v\b?") at failures.c:375 #3 0x0809f5cb in i_panic (format=0x80ac360 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:173 #4 0x08082277 in mail_cache_transaction_reserve_more (ctx=0x80cb4a0, block_size=332, commit=true) at mail-cache-transaction.c:257
There's no assert call in line 257. The closest one is at line 252, but I couldn't see how it could happen. I did some cleaning up for that code now, but I didn't do any changes for the logic itself.
participants (3)
-
-- --
-
Jared
-
Timo Sirainen