[Dovecot] IMP reading entire mailbox

[Timo Sirainen]

On Jun 9, 2009, at 11:44 AM, Chris O'Regan wrote:

>>> Corrupted index cache file /local/data/dovecot/indexes/mail/X/
>>> XXXXXXXX/.imap/delivery.log/dovecot.index.cache: Broken MIME parts
>>> for mail UID 30880
>>
>> Is it always this same UID? How often does it happen? Every 5  
>> minutes?
>
> The UID is increasing by one each time. There does not seem to be a
> pattern. Here's a sample from the past couple of days:
>
> Jun 8 05:56:57 ... UID 30859
> Jun 8 06:14:44 ... UID 30860
> Jun 8 09:03:01 ... UID 30861

That's kind of weird. So it doesn't seem like the contents of the  
mailbox actually matter.

>> Dovecot anyway detects that data in cache file is corrupted, so it
>> deletes the entire cache file. If IMP actually fetches something like
>> BODY or BODYSTRUCTURE for all messages, that causes all the messages
>> to be read since the cache was just deleted.
>
> Is there an easy way to tell if the cache file has been deleted and
> recreated? For example, will the inode number change?

The inode might change, but then again after the next recreation it  
could get the original inode back. But you could run idxview program  
and it'll show the cache file's file_seq which always increases:

idxview ~/Maildir
..
file_seq ............. = 1234639580 (2009-02-14 11:26:20) (9  
compressions)

> I enabled
> mail_debug for a short while this morning and did not notice anything
> unusual with the logs for this user: connect, set up namespace,
> disconnect just like everyone else.

mail_debug isn't really helpful in figuring out bugs.

> At any rate, the Dovecot stats report "bytes out" of about 500MB every
> five minutes (when the user's session closes); would this include the
> data read to build the index? I suspect not.

Hmm. Right, no, it wouldn't. The indexing issue sounds like a bug I  
should fix though. As for sending the 500 MB it sounds like an IMP  
issue. You could enable rawlog to see what commands exactly IMP sends: http://wiki.dovecot.org/Debugging/Rawlog

And to avoid it writing 500 MB log files every time, you can use  
rawlog -o parameter to log only the client -> server traffic (looks  
like I've messed up the -i and -o parameter order, guess I should  
switch them some day..)