Just mbox fixes since 1.0-test16. The logic is simpler and more correct now. Can anyone break it anymore? I actually tested it a while with Evolution and several mailboxes and it didn't break at least immediately. :)
Now maybe a few more days and I dare trying this thing myself with my real mboxes (yes, I'm still using them).
Dovecot mailing list archives could soon be served with 1.0-tests too, as soon as I get support for read-only mboxes working again (storing and comparing MD5 summed headers).
Timo Sirainen wrote:
Just mbox fixes since 1.0-test16. The logic is simpler and more correct now. Can anyone break it anymore?
_o/, I can.
Every time when indexes are rebuilt, new mail etc, Dovecot segfaults with this error in logs:
Info: pop3-login: Login: joe [127.0.0.1] Error: POP3(joe): file mbox-sync.c: line 745 (mbox_sync_handle_eof_updates): assertion failed: (sync_ctx->expunged_space == 0) Error: child 21489 (pop3) killed with signal 6
Here's a backtrace of it.
(gdb) bt #0 0x45dc0721 in kill () from /lib/libc.so.6 #1 0x45dc04c5 in raise () from /lib/libc.so.6 #2 0x45dc19e8 in abort () from /lib/libc.so.6 #3 0x0807de25 in i_internal_panic_handler (fmt=0x0, args=0x0) at failures.c:369 #4 0x0807d8f9 in i_panic (format=0x0) at failures.c:173 #5 0x0805f154 in mbox_sync_handle_eof_updates (sync_ctx=0x2e9, mail_ctx=0xba6eed80) at mbox-sync.c:813 #6 0x0805f8e1 in mbox_sync_update_imap_base (sync_ctx=0xba6eee50) at mbox-sync.c:952 #7 0x0805fc54 in mbox_sync (ibox=0x80a7eb8, last_commit=-1, lock=0) at mbox-sync.c:1053 #8 0x0805fdfb in mbox_storage_sync (box=0x80a7eb8, flags=0) at mbox-sync.c:1087 #9 0x08072746 in mailbox_sync (box=0x0, flags=1173120272) at mail-storage.c:372 #10 0x080674af in index_storage_get_status (box=0x80a7eb8, items=9, status=0xba6eefb0) at index-status.c:35 #11 0x0807271d in mailbox_get_status (box=0x45ec6510, items=0, status=0x0) at mail-storage.c:367 #12 0x08051d85 in init_mailbox (client=0x80a7a20) at client.c:54 #13 0x0805207c in client_create (hin=134780866, hout=0, storage=0x80a7930) at client.c:149 #14 0x08053723 in main_init () at main.c:119 #15 0x0805384c in main (argc=1, argv=0x0, envp=0x0) at main.c:152 (gdb)
This system is a Debian GNU/Linux testing/unstable with kernel 2.4.26-grsec.
-- Tomi Hakala
Tomi Hakala wrote:
_o/, I can.
Nevermind that last backtrace, I had a sligth typo on a configure --prefix path and new binaries got installed somewhere else :-/
But I can get some segfaults with actual test17.
This happens when more than one message is delivered to an empty box with no indexes present.
Info: Dovecot v1.0-test17 starting up Info: pop3-login: Login: joe [127.0.0.1] Error: POP3(joe): file mbox-sync-rewrite.c: line 341 (mbox_sync_fill_leftover): assertion failed: (start_offset < end_offset) Error: child 13957 (pop3) killed with signal 6
(gdb) bt #0 0x46e45721 in kill () from /lib/libc.so.6 #1 0x46e454c5 in raise () from /lib/libc.so.6 #2 0x46e469e8 in abort () from /lib/libc.so.6 #3 0x0807e6d5 in i_internal_panic_handler (fmt=0x0, args=0x0) at failures.c:369 #4 0x0807e1a9 in i_panic (format=0x0) at failures.c:173 #5 0x08063118 in mbox_sync_fill_leftover (sync_ctx=0xb1f8d640, mails=0x80a67c8, seq=1, idx=0, start_offset=55, end_offset=43) at mbox-sync-rewrite.c:341 #6 0x080634f0 in mbox_sync_rewrite (sync_ctx=0xb1f8d640, first_seq=1, last_seq=134899656, extra_space=2985875008) at mbox-sync-rewrite.c:471 #7 0x0805f8c6 in mbox_sync_handle_eof_updates (sync_ctx=0xb1f8d640, mail_ctx=0xb1f8d560) at mbox-sync.c:819 #8 0x0805fd50 in mbox_sync_do (sync_ctx=0xb1f8d640) at mbox-sync.c:954 #9 0x080601e7 in mbox_sync (ibox=0x80a4348, last_commit=0, lock=0) at mbox-sync.c:1077 #10 0x080604eb in mbox_storage_sync (box=0x80a4348, flags=0) at mbox-sync.c:1141 #11 0x08072fb6 in mailbox_sync (box=0x0, flags=1190442256) at mail-storage.c:372 #12 0x08067d0f in index_storage_get_status (box=0x80a4348, items=9, status=0xb1f8d7a0) at index-status.c:35 #13 0x08072f8d in mailbox_get_status (box=0x46f4b510, items=0, status=0x0) at mail-storage.c:367 #14 0x08051e35 in init_mailbox (client=0x80a3eb0) at client.c:54 #15 0x0805212c in client_create (hin=134783714, hout=0, storage=0x80a3dc0) at client.c:149 #16 0x080537d3 in main_init () at main.c:119 #17 0x080538fc in main (argc=1, argv=0x0, envp=0x0) at main.c:152 (gdb) quit
Now mailbox is corrupted, first and second message changed their place and first line of a mailbox contains "From" and "Return-Path" lines.
-- Tomi Hakala
Hello Timo and list,
I'm still getting some maildir errors with 1.0-test17:
sp dovecot: IMAP(moe): file mail-transaction-log.c: line 1159 (mail_transaction_log_sync_lock): assertion failed: (!log->index->log_locked) sp dovecot: child 6497 (imap) killed with signal 6
It happens about once an hour, I'm not sure how it can be reliably reproduced, yet.
regards
On Fri, Jun 18, 2004 at 03:30:50AM +0300, Timo Sirainen wrote:
Just mbox fixes since 1.0-test16. The logic is simpler and more correct now. Can anyone break it anymore? I actually tested it a while with Evolution and several mailboxes and it didn't break at least immediately. :)
Now maybe a few more days and I dare trying this thing myself with my real mboxes (yes, I'm still using them).
Dovecot mailing list archives could soon be served with 1.0-tests too, as soon as I get support for read-only mboxes working again (storing and comparing MD5 summed headers).
--
sed 's/mbox-support/shared-folders/g'
and now these start appearing, too:
sp dovecot: IMAP(moe): Maildir /home/moe/Maildir/.INBOX.ac-cron sync: UID inserted in the middle of ma ilbox (835 > 834, file = msg.jGI9:2,) sp dovecot: Killed with signal 15
switching back to dovecot-old...
On Fri, Jun 18, 2004 at 03:33:41PM +0200, Moe Wibble wrote:
Hello Timo and list,
I'm still getting some maildir errors with 1.0-test17:
sp dovecot: IMAP(moe): file mail-transaction-log.c: line 1159 (mail_transaction_log_sync_lock): assertion failed: (!log->index->log_locked) sp dovecot: child 6497 (imap) killed with signal 6
It happens about once an hour, I'm not sure how it can be reliably reproduced, yet.
regards
On Fri, Jun 18, 2004 at 03:30:50AM +0300, Timo Sirainen wrote:
Just mbox fixes since 1.0-test16. The logic is simpler and more correct now. Can anyone break it anymore? I actually tested it a while with Evolution and several mailboxes and it didn't break at least immediately. :)
Now maybe a few more days and I dare trying this thing myself with my real mboxes (yes, I'm still using them).
Dovecot mailing list archives could soon be served with 1.0-tests too, as soon as I get support for read-only mboxes working again (storing and comparing MD5 summed headers).
-- sed 's/mbox-support/shared-folders/g'
timo_todo_new.txt #;)
On 18.6.2004, at 22:20, Moe Wibble wrote:
and now these start appearing, too:
sp dovecot: IMAP(moe): Maildir /home/moe/Maildir/.INBOX.ac-cron sync: UID inserted in the middle of ma ilbox (835 > 834, file = msg.jGI9:2,) sp dovecot: Killed with signal 15
Hmm. You sure this wasn't because of old broken indexes in that mailbox?
On Fri, Jun 18, 2004 at 10:39:06PM +0300, Timo Sirainen wrote:
On 18.6.2004, at 22:20, Moe Wibble wrote:
and now these start appearing, too:
sp dovecot: IMAP(moe): Maildir /home/moe/Maildir/.INBOX.ac-cron sync: UID inserted in the middle of ma ilbox (835 > 834, file = msg.jGI9:2,) sp dovecot: Killed with signal 15
Hmm. You sure this wasn't because of old broken indexes in that mailbox?
Now that you mention it, actually I'm not sure. I'll shutdown, find . -iname "*dovecot*" -exec rm -f {} \;' and give test17 a 2nd shot.
On Fri, 2004-06-18 at 23:06, Moe Wibble wrote:
On Fri, Jun 18, 2004 at 10:39:06PM +0300, Timo Sirainen wrote:
On 18.6.2004, at 22:20, Moe Wibble wrote:
and now these start appearing, too:
sp dovecot: IMAP(moe): Maildir /home/moe/Maildir/.INBOX.ac-cron sync: UID inserted in the middle of ma ilbox (835 > 834, file = msg.jGI9:2,) sp dovecot: Killed with signal 15
Hmm. You sure this wasn't because of old broken indexes in that mailbox?
Now that you mention it, actually I'm not sure. I'll shutdown, find . -iname "*dovecot*" -exec rm -f {} \;' and give test17 a 2nd shot.
I really hope that fixes it, otherwise I have again no idea why it's breaking, and this time I can't even test it myself as it has worked correctly in my tests every time.
On Fri, Jun 18, 2004 at 11:10:09PM +0300, Timo Sirainen wrote:
On Fri, 2004-06-18 at 23:06, Moe Wibble wrote:
On Fri, Jun 18, 2004 at 10:39:06PM +0300, Timo Sirainen wrote:
On 18.6.2004, at 22:20, Moe Wibble wrote:
and now these start appearing, too:
sp dovecot: IMAP(moe): Maildir /home/moe/Maildir/.INBOX.ac-cron sync: UID inserted in the middle of ma ilbox (835 > 834, file = msg.jGI9:2,) sp dovecot: Killed with signal 15
Hmm. You sure this wasn't because of old broken indexes in that mailbox?
Now that you mention it, actually I'm not sure. I'll shutdown, find . -iname "*dovecot*" -exec rm -f {} \;' and give test17 a 2nd shot.
I really hope that fixes it, otherwise I have again no idea why it's breaking, and this time I can't even test it myself as it has worked correctly in my tests every time.
Sorry, even after erasing *dovecot* before the upgrade this is what I got again this morning:
sp dovecot: IMAP(moe): file mail-transaction-log.c: line 1159 (mail_transaction_log_sync_lock): assertion failed: (!log->index->log_locked) sp dovecot: child 25754 (imap) killed with signal 6
It seems to usually happen right after I deleted some messages and try to expunge them (maybe it's only when there is mail being delivered at the same time). But that might just be coincidence (I'm guessing).
moe
On 19.6.2004, at 12:24, Moe Wibble wrote:
I really hope that fixes it, otherwise I have again no idea why it's breaking, and this time I can't even test it myself as it has worked correctly in my tests every time.
Sorry, even after erasing *dovecot* before the upgrade this is what I got again this morning:
sp dovecot: IMAP(moe): file mail-transaction-log.c: line 1159 (mail_transaction_log_sync_lock): assertion failed: (!log->index->log_locked) sp dovecot: child 25754 (imap) killed with signal 6
But that's not a syncing problem. Fixing this one would only require looking through the code and making sure the assert can't ever happen. Not too difficult once I get around doing it :)
On Sat, Jun 19, 2004 at 07:01:53PM +0300, Timo Sirainen wrote:
On 19.6.2004, at 12:24, Moe Wibble wrote:
I really hope that fixes it, otherwise I have again no idea why it's breaking, and this time I can't even test it myself as it has worked correctly in my tests every time.
Sorry, even after erasing *dovecot* before the upgrade this is what I got again this morning:
sp dovecot: IMAP(moe): file mail-transaction-log.c: line 1159 (mail_transaction_log_sync_lock): assertion failed: (!log->index->log_locked) sp dovecot: child 25754 (imap) killed with signal 6
But that's not a syncing problem. Fixing this one would only require looking through the code and making sure the assert can't ever happen. Not too difficult once I get around doing it :)
sounds good, lookin' forward to it. ;)
On Sat, 2004-06-19 at 19:48, Moe Wibble wrote:
sp dovecot: IMAP(moe): file mail-transaction-log.c: line 1159 (mail_transaction_log_sync_lock): assertion failed: (!log->index->log_locked) sp dovecot: child 25754 (imap) killed with signal 6
But that's not a syncing problem. Fixing this one would only require looking through the code and making sure the assert can't ever happen. Not too difficult once I get around doing it :)
sounds good, lookin' forward to it. ;)
Actually it is a syncing problem, but not like the others. It happens when Dovecot tries to update maildir (flags, expunges) but finds out that some file is missing, so it tries to sync the maildir to find it. Except it is already in the middle of syncing index, so it crashes. Have to think a while what is the right fix for this.
On Sat, Jun 19, 2004 at 08:33:23PM +0300, Timo Sirainen wrote:
On Sat, 2004-06-19 at 19:48, Moe Wibble wrote:
sp dovecot: IMAP(moe): file mail-transaction-log.c: line 1159 (mail_transaction_log_sync_lock): assertion failed: (!log->index->log_locked) sp dovecot: child 25754 (imap) killed with signal 6
But that's not a syncing problem. Fixing this one would only require looking through the code and making sure the assert can't ever happen. Not too difficult once I get around doing it :)
sounds good, lookin' forward to it. ;)
Actually it is a syncing problem, but not like the others. It happens when Dovecot tries to update maildir (flags, expunges) but finds out that some file is missing, so it tries to sync the maildir to find it. Except it is already in the middle of syncing index, so it crashes. Have to think a while what is the right fix for this.
Sounds confusing. Shouldn't the index-sync just be delayed if there's one in progress already? I imagine something like while ( log->index->log_locked ) usleep(500);. After all it's locked, isn't it?
best regards
On 19.6.2004, at 20:50, Moe Wibble wrote:
Actually it is a syncing problem, but not like the others. It happens when Dovecot tries to update maildir (flags, expunges) but finds out that some file is missing, so it tries to sync the maildir to find it. Except it is already in the middle of syncing index, so it crashes. Have to think a while what is the right fix for this.
Sounds confusing. Shouldn't the index-sync just be delayed if there's one in progress already? I imagine something like while ( log->index->log_locked ) usleep(500);. After all it's locked, isn't it?
But it's then syncing the index itself, so sleeping wouldn't help :) It basically goes like:
- begin index sync
- commit each change to maildir
- maildir files may change their names, so if it's lost, we have to
find it again:
- read everything in maildir and store them
- begin index sync, so we can store the changes in it
- oops, we're already syncing index, crash
- maildir files may change their names, so if it's lost, we have to
find it again:
- finish index sync
Anyway, the fix was pretty simple. Just update the maildir filenames and leave the index syncing later.
I'll release test19 sometimes later after some more mbox fixes.
Timo Sirainen wrote:
Just mbox fixes since 1.0-test16. The logic is simpler and more correct now. Can anyone break it anymore? I actually tested it a while with Evolution and several mailboxes and it didn't break at least immediately. :)
well I can break maildir here - although I'm quite unfair :-)
I get a shitload of:
Jun 18 22:39:58 cronos dovecot: IMAP(mm-mailinglist@madness.at): file mail-transaction-log-view.c: line 122 (mail_transaction_log_view_set): assertion failed: (min_file_seq != max_file_seq || min_file_offset <= max_file_offset) Jun 18 22:39:58 cronos dovecot: child 42945 (imap) killed with signal 6 Jun 18 22:39:58 cronos dovecot: imap-login: Login: mm-mailinglist@madness.at [195.70.118.73] Jun 18 22:39:58 cronos dovecot: IMAP(mm-mailinglist@madness.at): file mail-transaction-log-view.c: line 122 (mail_transaction_log_view_set): assertion failed: (min_file_seq != max_file_seq || min_file_offset <= max_file_offset)
what I'm doing here is quite "strange" - though :-)
i have courier-imap and dovecot serving the same(!) maildir (on different ports) for evaluation and comparison. what happens now is that new mail gets delivered into the inbox, mozilla-thunderbird (using the dovecot-connection) "sees" it and displays the headers, mozilla-thunderbird courier sees it too and "moves" the message into a subfolder due to a filterrule -> *booom*.
while I agree that this is not a typical situation I would much prefer if dovecot could handle that a litte better then just crashing :-(
OS is FreeBSD 4.10, client is mozilla-thunderbird. authentication is going through pgsql.
Stefan
On 18.6.2004, at 23:53, Stefan Kaltenbrunner wrote:
well I can break maildir here - although I'm quite unfair :-)
I get a shitload of:
Jun 18 22:39:58 cronos dovecot: IMAP(mm-mailinglist@madness.at): file mail-transaction-log-view.c: line 122 (mail_transaction_log_view_set): assertion failed: (min_file_seq != max_file_seq || min_file_offset <= max_file_offset) Jun 18 22:39:58 cronos dovecot: child 42945 (imap) killed with signal 6
Oh, that's "normal". I haven't gotten around to figure out that bug yet, but it's completely different from what used to be breaking before.
what I'm doing here is quite "strange" - though :-)
i have courier-imap and dovecot serving the same(!) maildir (on different ports) for evaluation and comparison. what happens now is that new mail gets delivered into the inbox, mozilla-thunderbird (using the dovecot-connection) "sees" it and displays the headers, mozilla-thunderbird courier sees it too and "moves" the message into a subfolder due to a filterrule -> *booom*.
while I agree that this is not a typical situation I would much prefer if dovecot could handle that a litte better then just crashing :-(
Sure, if Dovecot can't handle other clients accessing the maildir it can't be called maildir-compatible.
Stefan Kaltenbrunner wrote:
what I'm doing here is quite "strange" - though :-)
i have courier-imap and dovecot serving the same(!) maildir (on different ports) for evaluation and comparison. what happens now is that new mail gets delivered into the inbox, mozilla-thunderbird (using the dovecot-connection) "sees" it and displays the headers, mozilla-thunderbird courier sees it too and "moves" the message into a subfolder due to a filterrule -> *booom*.
while I agree that this is not a typical situation I would much prefer if dovecot could handle that a litte better then just crashing :-(
I'm not on any test yet as I'm at the stage where I do hope most things work. My own scripting is causing enough problems! :)
I hope this isn't strange. My other "mua" is actually crontab scripts that move some files from folder to folder. Namely spam filtering/processing.
bogofilter filters procmail delivers crontab archives *boom* ??
On 19.6.2004, at 13:32, Tom Allison wrote:
I hope this isn't strange. My other "mua" is actually crontab scripts that move some files from folder to folder. Namely spam filtering/processing.
bogofilter filters procmail delivers crontab archives *boom* ??
With 0.99.10 there shouldn't be such problems.
participants (5)
-
Moe Wibble
-
Stefan Kaltenbrunner
-
Timo Sirainen
-
Tom Allison
-
Tomi Hakala