[Dovecot] Fwd: Weird problem opening mbox in beta1/2

[Chris Wakelin]

Chris Wakelin wrote:
> Chris Wakelin wrote:
>> Hmm, this is peculiar!
>>
>> This mailbox has been unchanged on my test server since September. I've
>> been using it to test each new version of Dovecot just after I've
>> compiled it.
>>
>> When beta1 was released, I opened this mbox with no problems. However,
>> when I upgraded to beta2, I got "file size unexpectedly shrinked in mbox
>> file /export/mail/a/24/vis99003/INBOX (16895967 vs 16899267)".
>>
>> Reverting to beta1, it also failed with the same error! I've deleted
>> indexes, restored the mbox from a backup made in October (no changes)
>> and I still get the error. It *does* open OK in 1.0-stable.
>>
>> I've put it through mboxcrypt.pl, deleted the UW-IMAP pseudo-message
>> (which wasn't converted properly by mboxcrypt) and it still fails to
>> open with a similar error, so I've sent it to Timo (gzipped, of course!)
>>
>> I've e-mailed this to the list minus the attachment as sending
>> unsolicited 1MB attachments is unfriendly! However, if anybody wants to
>> look at it, please let me know.
>>
>> Best Wishes,
>> Chris
>>
>> P.S. The past participle of "shrink" is "shrunk" not "shrinked" - it's
>> yet another irregular English verb :)
>>
> 
> Right! Some more testing seems to give :-
> 
> 1.0-alpha5 opens it fine
> dovecot-20060110 opens it fine
> dovecot-20060114 (plus the minor fixes for compile errors) fails
> 1.0-beta1 and 1.0-beta2 fail
> 
> dovecot-20060110 plus the patches from the CVS commits
> 
> 2006-01-11 20:45  Timo Sirainen <tss at iki.fi>
> 
> 	* src/lib-storage/index/mbox/: istream-raw-mbox.c, mbox-mail.c:
> 	  Handle unexpectedly breaking mboxes better without crashing.
> 
> 2006-01-14 13:59  Timo Sirainen <tss at iki.fi>
> 
> 	* src/lib-storage/index/mbox/mbox-sync-update.c: Use longer line
> 	  wrapping with X-IMAP, X-IMAPbase and X-Keywords headers since
> 	  UW-IMAP doesn't like the wrapping.
> 
> works (but I've just realised that the latter was after dovecot-20060114
> anyway).
> 
> So it looks like it might be something to do with the int -> bool changes.
> 
> The really strange thing, is I'm pretty sure I checked this mbox with
> all the above versions when they came out, and it was fine! I've also
> got another mbox giving the same sort of error in beta1/2, and I've
> tried a third one with beta2 and it was fine.
> 
> I've tried using UW-IMAP's mailutil to copy the mbox to a new folder
> (which sometimes fixes mbox problems), and that new folder still fails.
> 
> I've got a few users (including me, of course!) using beta1 on the live
> server without problems, but I'm not keen on deploying beta1 or beta2
> everywhere without getting to the bottom of this :(
> 
> Best Wishes,
> (Puzzled) Chris
> 

I'm getting somewhere at last. I tried chopping one of the mboxes into
smaller and smaller bits until I got the error with only one message!

It seems I've got a couple of test messages at the end of the suspect
mboxes with what looks like incorrect "Content-Length" headers.

But that leaves the question of what changed? The only thing in CVS I
can think of is the int -> bool change to parse_content_length() in
src/lib-storage/index/mbox/mbox-sync-parse.c; perhaps it never
successfully parsed them before and now does (we're still returning
"TRUE" or "FALSE" rather than -1 or 0)?

That still doesn't explain how come I opened them OK, even with beta1
until this week? Maybe I'm going bonkers and actually didn't open them
successfully after all!

Chris

-- 
--+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
Christopher Wakelin,                           c.d.wakelin at reading.ac.uk
IT Services Centre, The University of Reading,  Tel: +44 (0)118 378 8439
Whiteknights, Reading, RG6 2AF, UK              Fax: +44 (0)118 975 3094