[Dovecot] Out of memory [repost as a new thread]

[Wolfram Schlich]

* Timo Sirainen <tss at iki.fi> [2008-01-23 16:46]:
> On Wed, 2008-01-23 at 15:29 +0100, Wolfram Schlich wrote:
> 
> > I tried your patch and compiled dovecot with --enable-debug,
> 
> There's no need for --enable-debug. It's mainly useful for developers
> when developing new code.

Ok.

> > /usr/libexec/dovecot/imap: ELF 32-bit LSB shared object, Intel 80386,
> > version 1 (SYSV), for GNU/Linux 2.6.9, not stripped
> > 
> > --8<--
> > uluru tmp # gdb /usr/libexec/dovecot/imap core.x\{imap\}.u\{30010\}.g\{30006\}.p\{20880\}.t\{1201094907\}
> 
> Looks ok..
> 
> > (gdb) bt full
> > #0  0x169917d5 in ?? () from /lib/ld-linux.so.2
> > No symbol table info available.
> > #1  0x169bfba9 in ?? ()
> > No symbol table info available.
> > #2  0xb146d6ff in ?? ()
> > No symbol table info available.
> > #3  0x00000001 in ?? ()
> > No symbol table info available.
> > #4  0x00000000 in ?? ()
> > No symbol table info available.
> 
> Unfortunately this is broken. Even those addresses are impossible (#4
> shows that one call was from NULL pointer, #3 is 1). This just seems to
> happen sometimes for core dumps, possibly even repeatedly..
> 
> The one sure way to get a usable backtrace would be to attach gdb to a
> running imap process, but that would require you to be able to reproduce
> the bug at a specific time (so you can attach gdb, then make it crash)..
> 
> Or maybe if you have more core dumps, one of them shows something? The
> important thing is that if it only shows "??" it's broken. At the very
> least the last lines should contain something like:
> 
> #2  0x00000000004881cd in io_loop_run (ioloop=0x6c9d30) at ioloop.c:301
> #3  0x0000000000424a62 in main (argc=<value optimized out>, 
>     argv=0x7fff1841b9b8, envp=0x7fff1841b9c8) at main.c:293

No chance so far. I even recompiled glibc and kept the debug symbols:
--8<--
GNU gdb 6.7.1
Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".
(no debugging symbols found)
(no debugging symbols found)
Core was generated by `imap [someuser ::ffff:192.168.1.1]'.
Program terminated with signal 11, Segmentation fault.
#0  0x1a2887d5 in _start () from /lib/ld-linux.so.2
(gdb) bt full
#0  0x1a2887d5 in _start () from /lib/ld-linux.so.2
No symbol table info available.
#1  0x1a2a88f1 in ?? ()
No symbol table info available.
#2  0xb136ec5f in ?? ()
No symbol table info available.
#3  0x00000001 in ?? ()
No symbol table info available.
#4  0x00000000 in ?? ()
No symbol table info available.
(gdb)
--8<--

I have to say this machine is a Gentoo Hardened machine using a PaX
kernel and PIE/SSP userland...

Also I noticed that it doesn't *always* core:
--8<--
2008-01-24 05:16:00 +01:00; uluru; kern.alert; kernel: grsec: From 192.168.1.1: denied resource overstep by requesting 1079001088 for RLIMIT_AS against limit 1073741824 for /usr/libexec/dovecot/imap[imap:22235] uid/euid:30001/30001 gid/egid:30006/30006, parent /usr/sbin/dovecot[dovecot:20924] uid/euid:0/0 gid/egid:0/0
2008-01-24 05:16:00 +01:00; uluru; kern.alert; kernel: grsec: From 192.168.1.1: signal 11 sent to /usr/libexec/dovecot/imap[imap:22235] uid/euid:30001/30001 gid/egid:30006/30006, parent /usr/sbin/dovecot[dovecot:20924] uid/euid:0/0 gid/egid:0/0
2008-01-24 05:16:00 +01:00; uluru; mail.err; dovecot: child 22235 (imap) killed with signal 11

2008-01-24 11:17:42 +01:00; uluru; mail.err; dovecot: IMAP(info): pool_system_malloc(): Out of memory
2008-01-24 11:17:42 +01:00; uluru; mail.err; dovecot: child 1964 (imap) returned error 83 (Out of memory)
2008-01-24 11:17:42 +01:00; uluru; kern.alert; kernel: grsec: From 192.168.1.1: denied resource overstep by requesting 1073864704 for RLIMIT_AS against limit 1073741824 for /usr/libexec/dovecot/imap[imap:1964] uid/euid:30010/30010 gid/egid:30006/30006, parent /usr/sbin/dovecot[dovecot:20924] uid/euid:0/0 gid/egid:0/0

2008-01-24 10:58:42 +01:00; uluru; kern.alert; kernel: grsec: From 192.168.1.1: denied resource overstep by requesting 1074552832 for RLIMIT_AS against limit 1073741824 for /usr/libexec/dovecot/imap[imap:28392] uid/euid:30001/30001 gid/egid:30006/30006, parent /usr/sbin/dovecot[dovecot:20924] uid/euid:0/0 gid/egid:0/0
2008-01-24 10:58:42 +01:00; uluru; mail.err; dovecot: IMAP(someuser): pool_system_malloc(): Out of memory
2008-01-24 10:58:42 +01:00; uluru; mail.err; dovecot: child 28392 (imap) returned error 83 (Out of memory)
--8<--

So, only the crash from 05:16 produced a core, the others didn't...
WTF?! :))
And yes, I restarted dovecot right after I installed the
patched version...
-- 
Regards,
Wolfram Schlich <wschlich at gentoo.org>
Gentoo Linux * http://dev.gentoo.org/~wschlich/