[Dovecot] IMP reading entire mailbox

[Chris O'Regan]

> > Corrupted index cache file /local/data/dovecot/indexes/mail/X/ 
> > XXXXXXXX/.imap/delivery.log/dovecot.index.cache: Broken MIME parts  
> > for mail UID 30880
> 
> Is it always this same UID? How often does it happen? Every 5 minutes?

The UID is increasing by one each time. There does not seem to be a
pattern. Here's a sample from the past couple of days:

Jun 8 05:56:57 ... UID 30859
Jun 8 06:14:44 ... UID 30860
Jun 8 09:03:01 ... UID 30861
Jun 8 09:44:27 ... UID 30862
Jun 8 10:25:48 ... UID 30863
Jun 8 11:30:46 ... UID 30864
Jun 8 12:00:49 ... UID 30865
Jun 8 12:48:12 ... UID 30866
Jun 8 13:53:12 ... UID 30867
Jun 8 14:17:04 ... UID 30868
Jun 8 14:29:44 ... UID 30869
Jun 8 17:35:01 ... UID 30870
Jun 8 18:34:49 ... UID 30871
Jun 8 18:41:14 ... UID 30872
Jun 8 22:25:33 ... UID 30873
Jun 8 23:36:45 ... UID 30874
Jun 9 01:28:36 ... UID 30875
Jun 9 03:25:57 ... UID 30876
Jun 9 06:33:33 ... UID 30877
Jun 9 10:11:08 ... UID 30878
Jun 9 10:23:47 ... UID 30879
Jun 9 10:29:56 ... UID 30880
Jun 9 12:40:48 ... UID 30881
Jun 9 13:40:08 ... UID 30882
Jun 9 13:58:18 ... UID 30883
Jun 9 14:28:23 ... UID 30884

> Can you reproduce this manually somehow? If there's a way to reproduce  
> this, I'd like to get a copy of the mailbox. You can put the mailbox  
> through http://dovecot.org/tools/mbox-anonymize.pl and hopefully it  
> will then still break.

It is doubtful that I would be permitted to send a copy of the mailbox,
anonymized or not. It is questionable whether I would even be permitted
to make a copy of it as an experiment.

> Dovecot anyway detects that data in cache file is corrupted, so it  
> deletes the entire cache file. If IMP actually fetches something like  
> BODY or BODYSTRUCTURE for all messages, that causes all the messages  
> to be read since the cache was just deleted.

Is there an easy way to tell if the cache file has been deleted and
recreated? For example, will the inode number change? I enabled
mail_debug for a short while this morning and did not notice anything
unusual with the logs for this user: connect, set up namespace,
disconnect just like everyone else.

At any rate, the Dovecot stats report "bytes out" of about 500MB every
five minutes (when the user's session closes); would this include the
data read to build the index? I suspect not.


-- 
Chris O'Regan <chris at encs.concordia.ca>
Senior Unix Systems Administrator, Academic IT Services
Faculty of Engineering and Computer Science
Concordia University, Montreal, Canada