[Dovecot] Out of memory [repost as a new thread]

[Wolfram Schlich]

* Timo Sirainen <tss at iki.fi> [2008-01-20 08:49]:
> On Tue, 2008-01-15 at 14:51 +0100, Wolfram Schlich wrote:
> > this happens since a few days on a Gentoo hardened system using a
> > grsecurity enabled kernel running Dovecot 1.0.10, only to 2 of 10
> > users though:
> > 
> > --8<--
> > 
> > kernel: grsec: From 192.168.0.1: denied resource overstep by \
> > 	requesting 537325568 for RLIMIT_AS against limit 536870912 \
> 
> If it's trying to allocate 500MB of memory, there's a bug somewhere.
> Could you get gdb backtrace of this? The attached patch changes Dovecot
> to call abort() so it should write core files to user's home dir. See
> http://dovecot.org/bugreport.html

Hmm.
I tried your patch and compiled dovecot with --enable-debug,
enabled coredumps for the service (with ulimit -c unlimited)
and I got some corefiles now.

/usr/libexec/dovecot/imap: ELF 32-bit LSB shared object, Intel 80386,
version 1 (SYSV), for GNU/Linux 2.6.9, not stripped

--8<--
uluru tmp # gdb /usr/libexec/dovecot/imap core.x\{imap\}.u\{30010\}.g\{30006\}.p\{20880\}.t\{1201094907\}
GNU gdb 6.7.1
Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)
Using host libthread_db library "/lib/libthread_db.so.1".
(no debugging symbols found)
(no debugging symbols found)
Core was generated by `imap [info ::ffff:172.16.223.18]'.
Program terminated with signal 11, Segmentation fault.
#0  0x169917d5 in ?? () from /lib/ld-linux.so.2
(gdb) bt full
#0  0x169917d5 in ?? () from /lib/ld-linux.so.2
No symbol table info available.
#1  0x169bfba9 in ?? ()
No symbol table info available.
#2  0xb146d6ff in ?? ()
No symbol table info available.
#3  0x00000001 in ?? ()
No symbol table info available.
#4  0x00000000 in ?? ()
No symbol table info available.
(gdb) quit
uluru tmp #
--8<--

Looks like this happens in glibc, right?!
I haven't compiled glibc with debugging symbols/unstripped
so far of course...
-- 
Regards,
Wolfram Schlich <wschlich at gentoo.org>
Gentoo Linux * http://dev.gentoo.org/~wschlich/