[Dovecot] [DC2.x ]pop3-login seg faults when compiled with gcc 4.5.2 and gold
Hello, On FreeBSD, Dovecot 2.0.1 compiles fine with gcc45 and the other services have no problem, but even without any optimizations (CFLAGS) at all, pop3-login seg faults when a user tries to connect to Dovecot using POP3.
As this is not the only application that has a problem with the latest version of gcc (mod_php), there may be nothing that can be done on the Dovecot side, but I thought I would mention it, in case it was due to some bug in the pop3-login code.
Cheers,
Olivier
On Mon, 2010-08-30 at 16:52 +0100, interfaSys sàrl wrote:
On FreeBSD, Dovecot 2.0.1 compiles fine with gcc45 and the other services have no problem, but even without any optimizations (CFLAGS) at all, pop3-login seg faults when a user tries to connect to Dovecot using POP3.
If you can figure out a way to get gdb backtrace, I could try to look at why it's crashing there. One sure way to do it would be to patch: diff -r 267385a0ccd3 src/login-common/main.c --- a/src/login-common/main.c Wed Sep 01 18:44:32 2010 +0100 +++ b/src/login-common/main.c Wed Sep 01 20:07:26 2010 +0100 @@ -335,6 +335,7 @@ const char *login_socket = DEFAULT_LOGIN_SOCKET; int c; + sleep(30); master_service = master_service_init(login_binary.process_name, service_flags, &argc, &argv, "DS"); master_service_init_log(master_service, t_strconcat( Then get the process started and attach gdb: gdb /usr/local/libexec/dovecot/pop3-login <pid of pop3-login> cont <wait for crash> bt full
On 01/09/2010 20:08, Timo Sirainen wrote:
On Mon, 2010-08-30 at 16:52 +0100, interfaSys sàrl wrote:
On FreeBSD, Dovecot 2.0.1 compiles fine with gcc45 and the other services have no problem, but even without any optimizations (CFLAGS) at all, pop3-login seg faults when a user tries to connect to Dovecot using POP3.
If you can figure out a way to get gdb backtrace, I could try to look at why it's crashing there. One sure way to do it would be to patch:
diff -r 267385a0ccd3 src/login-common/main.c --- a/src/login-common/main.c Wed Sep 01 18:44:32 2010 +0100 +++ b/src/login-common/main.c Wed Sep 01 20:07:26 2010 +0100 @@ -335,6 +335,7 @@ const char *login_socket = DEFAULT_LOGIN_SOCKET; int c;
+ sleep(30); master_service = master_service_init(login_binary.process_name, service_flags, &argc, &argv, "DS"); master_service_init_log(master_service, t_strconcat(
Then get the process started and attach gdb:
gdb /usr/local/libexec/dovecot/pop3-login <pid of pop3-login> cont <wait for crash> bt full
Adding sleep() didn't change anything, it crashes right from the start and there isn't much info in the trace. It seems it cannot create the first thread. This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)... Core was generated by `pop3-login'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-login.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-login.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /usr/local/lib/libssl.so.7...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libssl.so.7 Reading symbols from /usr/local/lib/libcrypto.so.7...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libcrypto.so.7 Reading symbols from /usr/lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/local/lib/gcc45/libgomp.so.1...done. Loaded symbols for /usr/local/lib/gcc45/libgomp.so.1 Reading symbols from /lib/libthr.so.3...done. Loaded symbols for /lib/libthr.so.3 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x0000000800a29fa1 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 [New LWP 101099] (gdb) bt full #0 0x0000000800a29fa1 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #1 0x0000000800a2acad in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #2 0x0000000800a2b9d5 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #3 0x0000000800a2bbd2 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #4 0x0000000800a313e9 in malloc () from /lib/libc.so.7 No symbol table info available. #5 0x00000008008aabb5 in pthread_kill () from /lib/libthr.so.3 No symbol table info available. #6 0x00000008008ab57c in pthread_join () from /lib/libthr.so.3 No symbol table info available. #7 0x00000008008ae2d7 in __error () from /lib/libthr.so.3 No symbol table info available. #8 0x00000008008a2cee in ?? () from /lib/libthr.so.3 No symbol table info available. #9 0x00000008005331b0 in ?? () from /libexec/ld-elf.so.1 No symbol table info available. #10 0x000000080040bc49 in dlsym () from /libexec/ld-elf.so.1 No symbol table info available. #11 0x000000080040d011 in dlopen () from /libexec/ld-elf.so.1 No symbol table info available. #12 0x0000000800407789 in ?? () from /libexec/ld-elf.so.1 No symbol table info available. #13 0x0000000000000000 in ?? () No symbol table info available. #14 0x0000000000000000 in ?? () No symbol table info available.
On 1.9.2010, at 23.35, interfaSys sàrl wrote:
Adding sleep() didn't change anything, it crashes right from the start and there isn't much info in the trace. It seems it cannot create the first thread. .. #0 0x0000000800a29fa1 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7
I wonder why it's even trying to use pthreads .. Dovecot doesn't use threads. Maybe that's the problem, it's linking wrong libraries.
On 02/09/2010 00:35, Timo Sirainen wrote:
On 1.9.2010, at 23.35, interfaSys sàrl wrote:
Adding sleep() didn't change anything, it crashes right from the start and there isn't much info in the trace. It seems it cannot create the first thread. ... #0 0x0000000800a29fa1 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7
I wonder why it's even trying to use pthreads .. Dovecot doesn't use threads. Maybe that's the problem, it's linking wrong libraries.
Probably because I used the CFLAGS for OpenMP. But even if I don't use CFLAGS, I get a segfault. Could it be because some libraries are using it?
#0 0x0000000800911fa1 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 (gdb) bt full #0 0x0000000800911fa1 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #1 0x0000000800912cad in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #2 0x00000008009139d5 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #3 0x0000000800913bd2 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #4 0x00000008009193e9 in malloc () from /lib/libc.so.7 No symbol table info available. #5 0x00000008008e50a9 in sem_close () from /lib/libc.so.7 No symbol table info available. #6 0x00000008008e5331 in sem_init () from /lib/libc.so.7 No symbol table info available. #7 0x00000008004e8d12 in __do_global_ctors_aux () from /usr/local/lib/gcc45/libgomp.so.1 No symbol table info available. #8 0x00000008004e015e in _init () from /usr/local/lib/gcc45/libgomp.so.1 No symbol table info available. #9 0x00000008005331b0 in ?? () from /libexec/ld-elf.so.1 No symbol table info available. #10 0x000000080040bc49 in dlsym () from /libexec/ld-elf.so.1 No symbol table info available. #11 0x000000080040d011 in dlopen () from /libexec/ld-elf.so.1 No symbol table info available. #12 0x0000000800407789 in ?? () from /libexec/ld-elf.so.1 No symbol table info available. #13 0x0000000000000000 in ?? () No symbol table info available.
On Thu, 2010-09-02 at 01:13 +0100, interfaSys sàrl wrote:
I wonder why it's even trying to use pthreads .. Dovecot doesn't use threads. Maybe that's the problem, it's linking wrong libraries.
Probably because I used the CFLAGS for OpenMP. But even if I don't use CFLAGS, I get a segfault. Could it be because some libraries are using it? .. #8 0x00000008004e015e in _init () from /usr/local/lib/gcc45/libgomp.so.1
Looks like libgomp, which is OpenMP. So it's still using it even when you haven't enabled it with CFLAGS?
On 02/09/2010 16:53, Timo Sirainen wrote:
On Thu, 2010-09-02 at 01:13 +0100, interfaSys sàrl wrote:
I wonder why it's even trying to use pthreads .. Dovecot doesn't use threads. Maybe that's the problem, it's linking wrong libraries.
Probably because I used the CFLAGS for OpenMP. But even if I don't use CFLAGS, I get a segfault. Could it be because some libraries are using it? ... #8 0x00000008004e015e in _init () from /usr/local/lib/gcc45/libgomp.so.1
Looks like libgomp, which is OpenMP. So it's still using it even when you haven't enabled it with CFLAGS?
Indeed pop3-login: libdovecot-login.so.0 => /usr/lib/dovecot/libdovecot-login.so.0 (0x80043c000) libdovecot.so.0 => /usr/lib/dovecot/libdovecot.so.0 (0x800453000) libssl.so.7 => /usr/local/lib/libssl.so.7 (0x80054a000) libcrypto.so.7 => /usr/local/lib/libcrypto.so.7 (0x8005ae000) librt.so.1 => /usr/lib/librt.so.1 (0x800797000) libc.so.7 => /lib/libc.so.7 (0x80089c000) libgomp.so.1 => /usr/local/lib/gcc45/libgomp.so.1 (0x8004dd000)
Maybe because of this?
/usr/local/lib/libcrypto.so.7: libgomp.so.1 => /usr/local/lib/gcc45/libgomp.so.1 (0x800542000)
OpenSSL was compiled with OpenMP
On Thu, 2010-09-02 at 17:09 +0100, interfaSys sàrl wrote:
#8 0x00000008004e015e in _init () from /usr/local/lib/gcc45/libgomp.so.1
Maybe because of this?
/usr/local/lib/libcrypto.so.7: libgomp.so.1 => /usr/local/lib/gcc45/libgomp.so.1 (0x800542000)
OpenSSL was compiled with OpenMP
Yeah, that looks like the problem. I've no idea about OpenMP or pthreads library in general, so I don't know if there's anything Dovecot can do about this.
On 02/09/2010 17:12, Timo Sirainen wrote:
On Thu, 2010-09-02 at 17:09 +0100, interfaSys sàrl wrote:
#8 0x00000008004e015e in _init () from /usr/local/lib/gcc45/libgomp.so.1
Maybe because of this?
/usr/local/lib/libcrypto.so.7: libgomp.so.1 => /usr/local/lib/gcc45/libgomp.so.1 (0x800542000)
OpenSSL was compiled with OpenMP
Yeah, that looks like the problem. I've no idea about OpenMP or pthreads library in general, so I don't know if there's anything Dovecot can do about this.
The strange thing is that it works fine for all the other components of Dovecot, including plugins, so there must be something done a little differently in pop3-login or gcc still has some bugs with OpenMP
On 02/09/2010 17:12, Timo Sirainen wrote:
On Thu, 2010-09-02 at 17:09 +0100, interfaSys sàrl wrote:
#8 0x00000008004e015e in _init () from /usr/local/lib/gcc45/libgomp.so.1
Maybe because of this?
/usr/local/lib/libcrypto.so.7: libgomp.so.1 => /usr/local/lib/gcc45/libgomp.so.1 (0x800542000)
OpenSSL was compiled with OpenMP
Yeah, that looks like the problem. I've no idea about OpenMP or pthreads library in general, so I don't know if there's anything Dovecot can do about this.
I recompiled some of the base libs without OpenMP, but I still get a seg fault. I think the Graphite optimisations automatically link GOMP and pthread
On 02/09/2010 17:12, Timo Sirainen wrote:
On Thu, 2010-09-02 at 17:09 +0100, interfaSys sàrl wrote:
#8 0x00000008004e015e in _init () from /usr/local/lib/gcc45/libgomp.so.1
Maybe because of this?
/usr/local/lib/libcrypto.so.7: libgomp.so.1 => /usr/local/lib/gcc45/libgomp.so.1 (0x800542000)
OpenSSL was compiled with OpenMP
Yeah, that looks like the problem. I've no idea about OpenMP or pthreads library in general, so I don't know if there's anything Dovecot can do about this.
Here is what I see when auto paralellization is removed:
(gdb) bt full #0 0x000000080090dfa1 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #1 0x000000080090ecad in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #2 0x000000080090f9d5 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #3 0x00000008009126fc in _malloc_prefork () from /lib/libc.so.7 No symbol table info available. #4 0x00000008009153d4 in malloc () from /lib/libc.so.7 No symbol table info available. #5 0x0000000800490a8c in data_stack_init () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x000000004c8095b7 in ?? () No symbol table info available. #7 0x0000000000403392 in ?? () No symbol table info available. #8 0x0000000000000048 in ?? () No symbol table info available. #9 0x00000008004a0a67 in lib_init () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #10 0x000000080048b3ff in master_service_init () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #11 0x0000000800449231 in main () from /usr/lib/dovecot/libdovecot-login.so.0 No symbol table info available. #12 0x0000000000401dae in _start () No symbol table info available. #13 0x0000000800436000 in ?? ()
On 02/09/2010 17:12, Timo Sirainen wrote:
On Thu, 2010-09-02 at 17:09 +0100, interfaSys sàrl wrote:
#8 0x00000008004e015e in _init () from /usr/local/lib/gcc45/libgomp.so.1
Maybe because of this?
/usr/local/lib/libcrypto.so.7: libgomp.so.1 => /usr/local/lib/gcc45/libgomp.so.1 (0x800542000)
OpenSSL was compiled with OpenMP
Yeah, that looks like the problem. I've no idea about OpenMP or pthreads library in general, so I don't know if there's anything Dovecot can do about this.
Here is the trace with Dovecot 2.0.2
#0 0x000000080090dfa1 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #1 0x000000080090ecad in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #2 0x000000080090f9d5 in _pthread_mutex_init_calloc_cb () from /lib/libc.so.7 No symbol table info available. #3 0x00000008009126fc in _malloc_prefork () from /lib/libc.so.7 No symbol table info available. #4 0x00000008009153d4 in malloc () from /lib/libc.so.7 No symbol table info available. #5 0x0000000800490a8c in data_stack_init () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #6 0x000000004c88f009 in ?? () No symbol table info available. #7 0x0000000000403392 in ?? () No symbol table info available. #8 0x0000000000000048 in ?? () No symbol table info available. #9 0x00000008004a0a67 in lib_init () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #10 0x000000080048b3ff in master_service_init () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #11 0x0000000800449231 in main () from /usr/lib/dovecot/libdovecot-login.so.0 No symbol table info available. #12 0x0000000000401dae in _start () No symbol table info available.
On 02/09/2010 17:12, Timo Sirainen wrote:
On Thu, 2010-09-02 at 17:09 +0100, interfaSys sàrl wrote:
#8 0x00000008004e015e in _init () from /usr/local/lib/gcc45/libgomp.so.1
Maybe because of this?
/usr/local/lib/libcrypto.so.7: libgomp.so.1 => /usr/local/lib/gcc45/libgomp.so.1 (0x800542000)
OpenSSL was compiled with OpenMP
Yeah, that looks like the problem. I've no idea about OpenMP or pthreads library in general, so I don't know if there's anything Dovecot can do about this.
Did some more tests. pop3-login segfaults if the gold linker (latest SVN) is used. It works fine when using GNU ld.
participants (2)
-
interfaSys sàrl
-
Timo Sirainen