[Dovecot] Embedded From_ lines breaking Content-Length (and Dovecot)

[Lennart Lövstrand]

I feel like I'm going totally crazy.  Is it just me, or have embedded  
From_ lines really been breaking mbox messages since (at least)  
dovecot 1.0?  I found a whole lot of broken messages in an old mailbox  
of mine and when I looked closer at it, it seemed like Dovecot was  
ignoring the Content-Length header and truncating the messages at the  
first embedded From_ line instead.  Figuring that this probably just  
was a fluke / old bug, I erased the index and upgraded Dovecot, first  
to 1.0.10, then to 1.1.beta14, but I'm still seeing the bug -- and  
it's even gotten worse!  Earlier versions would just quietly split the  
message at the From_ line, but 1.1.beta14 is more diligent and will  
complain with:

> dovecot[44795]: IMAP(lennart): FETCH for mailbox Drafts UID 196 got  
> too little data: 590 vs 719

before dropping the connection and exiting... which cause Thunderbird  
to reopen the connection and reissue the same request... which will  
cause Dovecot to complain and exit again... etc ad nauseam...

It's trivial to reproduce too -- just mail yourself a message with a  
valid From_ line in it (assuming that your delivery system isn't doing  
 >From-escaping), or put it in a draft plain text message and save  
it.  Then go and look in your Drafts folder...

(Like this one: "From someone Tue Jan 29 03:15:52 2008")

Just be sure that you can stop your spinning mail client and/or server  
if you're using 1.1.beta14...

---

OK, I am definitely not very familiar with the Dovecot source, but I  
spent some time trying to track this down today and it looks to me  
like i_stream_raw_mbox_read is being asked to find the end of the  
message (at the From_ line) before mbox_sync_parse_next_mail has had a  
chance to parse the Content-Length header and find out how big it is.   
Maybe someone who knows the code better can take a look at it and tell  
me if I'm barking up the totally wrong tree...

Thanks,
--Lennart