[Dovecot] IMP reading entire mailbox

[Michael M. Slusarz]

Quoting Rick Romero <rick at havokmon.com>:

> Quoting "Timo Sirainen" <tss at iki.fi>:
>
>> On Jun 9, 2009, at 9:18 AM, Chris O'Regan wrote:
>>
>>> IMP is periodically checking this user's mail every five minutes. Each
>>> time it does, the his entire mailbox is being read. I initially thought
>>> it might be an indexing problem since I am also seeing messages such as
>>> this in the logs:
>>>
>>> Corrupted index cache file  
>>> /local/data/dovecot/indexes/mail/X/XXXXXXXX/.imap/delivery.log/dovecot.index.cache: Broken MIME parts for mail UID  
>>> 30880
>>
>> Is it always this same UID? How often does it happen? Every 5  
>> minutes? Can you reproduce this manually somehow? If there's a way  
>> to reproduce this, I'd like to get a copy of the mailbox. You can  
>> put the mailbox through http://dovecot.org/tools/mbox-anonymize.pl  
>> and hopefully it will then still break.
>>
>> Dovecot anyway detects that data in cache file is corrupted, so it  
>> deletes the entire cache file. If IMP actually fetches something  
>> like BODY or BODYSTRUCTURE for all messages, that causes all the  
>> messages to be read since the cache was just deleted.
>
> IMP also has an option to preview messages.  It reads the entire  
> contents of the messages (should only be 20 or so for the 'current'  
> INBOX window), and will display the first 200 characters or so of  
> the email.  Is it possible the numbers you're seeing reflect the  
> total size of his most current 20 emails?

Unfortunately, with IMP 4.x we are limited to the API provided by  
c-client/PHP imap module.  This API is *horribly* inefficient and does  
all sorts of terrible IMAP things that are  
unnecessary/inefficient/embarassing.  So it is conceivable that the  
entire mailbox is scanned when a mailbox refresh occurs.

Good news is that IMP 5 fixes all this with a new IMAP library  
(created, in part, with advice directly from Timo himself :)).   
Although none of this explains why the dovecot cache is apparently  
being corrupted.

michael