On Sat, 2007-12-29 at 17:54 +0100, Gerhard Wiesinger wrote:
Hello Timo!
What was exactly wrong in v1.0.8 and v1.0.9 with mbox handling?
v1.0.8 added optimizations to get message's full size without reading through the entire message. This helped performance a lot with large messages, especially if they were being FETCHed in small blocks.
This optimization exposed bugs in mbox file reading, causing a FETCH to sometimes return the next message's contents and maybe disconnect with "got too little data" error.
All of this is related to how input streams work internally.. They layer on top of each others, so that when IMAP reads data it goes through multiple input streams:
- Header filter stream to drop some mbox headers
- Limit stream to make stream's virtual offset=0 point to beginning of the mbox message (and not to beginning of mbox file)
- mbox raw stream to return EOF when reaching the next "From " line
- File stream to actually read the data from mbox file
The problem was then that some data was first read by header filter stream, but in the middle of it mbox raw stream was accessed to get the message's size. Header filter stream didn't know about this and still returned data which pointed to raw stream's buffer, which at this point contained wrong data.
I fixed this with some kludges for v1.0. For v1.1 I did larger stream changes to make sure the streams know about changes in each others.