[Dovecot] corrupted binary tree and index data files

[Gary Sandine]

Hi.

I am using dovecot 0.99.14 as an imaps server with mbox folders (in an
NFS-mounted directory) in x86 Debian testing.  I have searched the
archives of this list and have seen other evidence of the problems I am
having.  I get lots of log messages like these:

Jul 29 07:47:09 HOST imap(USER): Corrupted binary tree file /home/USER/IMAP_MBOX_DIR/.imap/INBOX/.imap.index.tree: UID to be inserted isn't higher than existing (290 <= 291)
...
Jul 29 07:47:13 HOST imap(USER): Corrupted index data file /home/USER/IMAP_MBOX_DIR/.imap/INBOX/.imap.index.data: Field 1 size points outside file (14648 / 14656) for record 290

This is from the midst of a herd of messages starting at 82 <= 144:

Jul 28 08:01:31 HOST imap(USER): Corrupted binary tree file /home/USER/IMAP_MBOX_DIR/FOLDER_DIR/.imap/MBOX1/.imap.index.tree: UID to be inserted isn't higher than existing (82 <= 144)
...
Jul 28 08:01:32 HOST imap(USER): Corrupted binary tree file /home/USER/IMAP_MBOX_DIR/FOLDER_DIR/.imap/MBOX1/.imap.index.tree: UID to be inserted isn't higher than existing (143 <= 144)
Jul 28 08:01:32 HOST imap(USER): Corrupted binary tree file /home/USER/IMAP_MBOX_DIR/FOLDER_DIR/.imap/MBOX1/.imap.index.tree: UID to be inserted isn't higher than existing (144 <= 144)
Jul 28 08:01:32 HOST imap(USER): Corrupted binary tree file /home/USER/IMAP_MBOX_DIR/FOLDER_DIR/.imap/MBOX1/.imap.index.tree: UID to be inserted isn't higher than existing (92 <= 154)
Jul 28 08:01:32 HOST imap(USER): Corrupted binary tree file /home/USER/IMAP_MBOX_DIR/FOLDER_DIR/.imap/MBOX1/.imap.index.tree: UID to be inserted isn't higher than existing (93 <= 154)
...
Jul 28 08:01:32 HOST imap(USER): Corrupted binary tree file /home/USER/IMAP_MBOX_DIR/FOLDER_DIR/.imap/MBOX1/.imap.index.tree: UID to be inserted isn't higher than existing (101 <= 154)
Jul 28 08:01:34 HOST imap(USER): Corrupted binary tree file /home/USER/IMAP_MBOX_DIR/FOLDER_DIR/SUBFOLDER_DIR/.imap/MBOX2/.imap.index.tree: UID to be inserted isn't higher than existing (79 <= 80)
Jul 28 08:01:34 HOST imap(USER): Corrupted binary tree file /home/USER/IMAP_MBOX_DIR/FOLDER_DIR/SUBFOLDER_DIR/.imap/MBOX2/.imap.index.tree: UID to be inserted isn't higher than existing (80 <= 80)
Jul 28 08:11:27 HOST imap(USER): Corrupted binary tree file /home/USER/IMAP_MBOX_DIR/FOLDER_DIR/.imap/MBOX1/.imap.index.tree: UID to be inserted isn't higher than existing (102 <= 164)
...

I don't know how to reproduce this.  It happens (repeatedly) for some
users and seems to be independent of the number of mboxes, mbox sizes,
mail client, ....  I have tried deleting .imap directories, and this
helps for a while, but the problem always comes back for a few users.

The proposed solution for "Corrupted binary tree" or "Corrupted index
data file" log messages is usually to change to a different version of
dovecot.  I will try that, but want to check in here first in case
something obvious is happening that I don't understand.

Thanks,
-- 
Gary Sandine <gars at lanl.gov>