Re: [Dovecot] Segfault in imap_bodystructure_write when searching
Whenever I try to search on my dovecot test install, dovecot dies. It immediately spawns a new process, which the client tries to connect to, and send search commands to, causing that to die, and so forth. This loop means I have to kill either the server or the client.
Your dovecot.index.cache file is probably broken somehow. Try moving dovecot.index* files elsewhere and see if it works then? If it does and dovecot.index.cache doesn't contain anything too sensitive (subjects/from/to addresses), could you send me the dovecot.index* files?
If it crashes even without dovecot.index* files then the mails themselves contain a broken message. It would help if I could get that message.
Bullseye. Why did I fail to mention that I was trying to use uiddb from Courier? :P Using your conversion script fixed it, but simply copying the Courier file crashed Dovecot. According to http://wiki.dovecot.org/Migration/Courier this is supposed to work though, so either that page is wrong, or Dovecot has a bug.
betty - ~vpopmail/domains/bar.com/foo # gdb /usr/lib/dovecot/imap core .. (no debugging symbols found) Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. #0 0x080a539d in imap_bodystructure_write () (gdb) bt full #0 0x080a539d in imap_bodystructure_write () No symbol table info available. #1 0x08083b5c in ?? ()
Unfortunately this backtrace is almost completely broken and it doesn't really help.. You also could try compiling Dovecot itself with debug symbols and getting backtrace from it. That'd probably work.
I tried reproducing now, but it seems it's started working even when moving all dovecot files and copying courierimapuiddb to dovecot-uidlist. If I see it again I'll send you a copy of the file offlist.
Thanks.
-- -==- -=- -==- Christer Mjellem Strand yitzhaq System administrator ICQ: 9557698 GSM +47 922 000 12 JID: yitzhaq@jabber.no -==- -=- -==-
On Sun, 2008-06-01 at 16:47 +0200, Christer Mjellem Strand wrote:
Whenever I try to search on my dovecot test install, dovecot dies. It immediately spawns a new process, which the client tries to connect to, and send search commands to, causing that to die, and so forth. This loop means I have to kill either the server or the client.
Your dovecot.index.cache file is probably broken somehow. Try moving dovecot.index* files elsewhere and see if it works then? If it does and dovecot.index.cache doesn't contain anything too sensitive (subjects/from/to addresses), could you send me the dovecot.index* files?
If it crashes even without dovecot.index* files then the mails themselves contain a broken message. It would help if I could get that message.
Bullseye. Why did I fail to mention that I was trying to use uiddb from Courier? :P Using your conversion script fixed it, but simply copying the Courier file crashed Dovecot. According to http://wiki.dovecot.org/Migration/Courier this is supposed to work though, so either that page is wrong, or Dovecot has a bug.
The Courier and Dovecot v1.0 uidlist files are nearly identical. The only difference is that Dovecot adds the ":2,flags" to filenames which is kind of an optimistic optimization that it's able to find filenames without scanning maildir to find the exact filename. But this isn't relied on in any way. So the problem most likely was caused/fixed by something else.
betty - ~vpopmail/domains/bar.com/foo # gdb /usr/lib/dovecot/imap core .. (no debugging symbols found) Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. #0 0x080a539d in imap_bodystructure_write ()
You said this happens when searching? imap_bodystructure_write() is never called on SEARCH, so this is probably also because the backtrace was corrupted.
Do you know what exactly was the SEARCH command that caused the crash? (rawlog could help: http://wiki.dovecot.org/Debugging/Rawlog) I tried reproducing the crash with the files you sent privately, but I can't seem to cause it to crash.
One way to maybe avoid gdb backtrace corruption is to attach gdb while the process is still running. Like:
- Log in so imap process exists
- gdb /usr/lib/dovecot/imap
pidof imap - cont
- make it crash
- bt full
Whenever I try to search on my dovecot test install, dovecot dies. It immediately spawns a new process, which the client tries to connect to, and send search commands to, causing that to die, and so forth. This loop means I have to kill either the server or the client.
Your dovecot.index.cache file is probably broken somehow. Try moving dovecot.index* files elsewhere and see if it works then? If it does and dovecot.index.cache doesn't contain anything too sensitive (subjects/from/to addresses), could you send me the dovecot.index* files?
If it crashes even without dovecot.index* files then the mails themselves contain a broken message. It would help if I could get that message.
Bullseye. Why did I fail to mention that I was trying to use uiddb from Courier? :P Using your conversion script fixed it, but simply copying the Courier file crashed Dovecot. According to http://wiki.dovecot.org/Migration/Courier this is supposed to work though, so either that page is wrong, or Dovecot has a bug.
The Courier and Dovecot v1.0 uidlist files are nearly identical. The only difference is that Dovecot adds the ":2,flags" to filenames which is kind of an optimistic optimization that it's able to find filenames without scanning maildir to find the exact filename. But this isn't relied on in any way. So the problem most likely was caused/fixed by something else.
This latest crash I can reproduce by just trying to open the mailbox, no searching necessary. With the previous crashes, while other symptoms seemed identical, the mailbox (and messages in it) would open normally, but the crash would happen once I tried searching for something.
One way to maybe avoid gdb backtrace corruption is to attach gdb while the process is still running. Like:
- Log in so imap process exists
- gdb /usr/lib/dovecot/imap
pidof imap- cont
- make it crash
- bt full
I have very limited gdb experience, and can't seem to make this work as intended. I've restarted dovecot and logged in so only one imap process exists, and attached to that. However, it seems that this process keeps on running, and instead spawns children, and they're the ones that die immediately. The main process stays alive, and no backtrace is generated when I just kill dovecot to stop the loop..
Any pointers as to how I could attach gdb to any of those processes, or other ways to proceed?
Please excuse my ignorance here. I've found a successful workaround (just regenerating the index files), but since you state that you wish to fix every crasher, I'll gladly help debug this until you've pinpointed it, if you have the patience to bear with me.. :)
-- -==- -=- -==- Christer Mjellem Strand yitzhaq System administrator ICQ: 9557698 GSM +47 922 000 12 JID: yitzhaq@jabber.no -==- -=- -==-
On Jun 1, 2008, at 7:17 PM, Christer Mjellem Strand wrote:
This latest crash I can reproduce by just trying to open the
mailbox, no searching necessary.
Is it with the files you sent me or some other mailbox?
One way to maybe avoid gdb backtrace corruption is to attach gdb
while the process is still running. Like:
- Log in so imap process exists
- gdb /usr/lib/dovecot/imap
pidof imap- cont
- make it crash
- bt full
I have very limited gdb experience, and can't seem to make this work
as intended. I've restarted dovecot and logged in so only one imap
process exists, and attached to that. However, it seems that this
process keeps on running, and instead spawns children, and they're
the ones that die immediately. The main process stays alive, and no backtrace is generated when I
just kill dovecot to stop the loop..
Only dovecot process forks new processes. I guess it's possible that
your IMAP client works by first creating one connection (first imap
process) and then when selecting the mailbox logs in using a new
connection.
Any pointers as to how I could attach gdb to any of those processes,
or other ways to proceed?
How about manually:
telnet localhost 143 a login user pass << gdb attach >> b select mailbox
Please excuse my ignorance here. I've found a successful workaround
(just regenerating the index files), but since you state that you
wish to fix every crasher, I'll gladly help debug this until you've
pinpointed it, if you have the patience to bear with me.. :)
Yes, fixing the crash would be good.
This latest crash I can reproduce by just trying to open the mailbox, no searching necessary.
Is it with the files you sent me or some other mailbox?
The files I sent you. The search issue I haven't been able to reproduce since last night.
One way to maybe avoid gdb backtrace corruption is to attach gdb while the process is still running. Like:
- Log in so imap process exists
- gdb /usr/lib/dovecot/imap
pidof imap- cont
- make it crash
- bt full
I have very limited gdb experience, and can't seem to make this work as intended. I've restarted dovecot and logged in so only one imap process exists, and attached to that. However, it seems that this process keeps on running, and instead spawns children, and they're the ones that die immediately. The main process stays alive, and no backtrace is generated when I just kill dovecot to stop the loop..
Only dovecot process forks new processes. I guess it's possible that your IMAP client works by first creating one connection (first imap process) and then when selecting the mailbox logs in using a new connection.
Aha, that does indeed sound probable, and seems to fit well with what I'm seeing in the logs.
Any pointers as to how I could attach gdb to any of those processes, or other ways to proceed?
How about manually:
telnet localhost 143 a login user pass << gdb attach >> b select mailbox
No crash :\
I could always fire up wireshark or something to try to figure out what my client (Mulberry) is actually doing, but that still wouldn't log the crasher..
-- -==- -=- -==- Christer Mjellem Strand yitzhaq System administrator ICQ: 9557698 GSM +47 922 000 12 JID: yitzhaq@jabber.no -==- -=- -==-
On Jun 1, 2008, at 7:50 PM, Christer Mjellem Strand wrote:
I could always fire up wireshark or something to try to figure out
what my client (Mulberry) is actually doing, but that still wouldn't
log the crasher..
The last sent command should be the one that causes the crash. Instead
of wireshark you could also use http://wiki.dovecot.org/Debugging/
Rawlog (works with SSL connections too).
I could always fire up wireshark or something to try to figure out what my client (Mulberry) is actually doing, but that still wouldn't log the crasher..
The last sent command should be the one that causes the crash. Instead of wireshark you could also use http://wiki.dovecot.org/Debugging/Rawlog (works with SSL connections too).
Found out Mulberry has built-in logging, so I just used that. http://trac.mulberrymail.com/mulberry/wiki/logging btw, in case it's useful.
I suspect the SORT command might be involved here, since just opening the mailbox didn't cause the crash. I'll send you a log copy off-list. If that doesn't do the trick, I can play around with Rawlog.
-- -==- -=- -==- Christer Mjellem Strand yitzhaq System administrator ICQ: 9557698 GSM +47 922 000 12 JID: yitzhaq@jabber.no -==- -=- -==-
On Jun 1, 2008, at 8:12 PM, Christer Mjellem Strand wrote:
Found out Mulberry has built-in logging, so I just used that. http://trac.mulberrymail.com/mulberry/wiki/logging btw, in case it's
useful.
Oh, Mulberry. I bet this fixes it: http://hg.dovecot.org/dovecot-1.0/rev/538f8892a2f1
I was going to release 1.0.14 this weekend. I suppose I still could. :)
participants (2)
-
Christer Mjellem Strand
-
Timo Sirainen