failed: Cached message size smaller than expected

[Aki Tuomi]

> On 19/04/2021 13:21 FUSTE Emmanuel <emmanuel.fuste at thalesgroup.com> wrote:
> 
>  
> Le 19/04/2021 à 09:01, Aki Tuomi a écrit :
> >> On 17/04/2021 23:07 Michael Grant <mgrant at grant.org> wrote:
> >>
> >>   
> >> On Fri, Apr 02, 2021 at 04:45:36PM -0400, Michael Grant wrote:
> >>> Every few days, my mailbox seizes up.  No mail come in to my imap clients.
> >>>
> >>> I'm getting these errors over and over with my mailbox:
> >>>
> >>>    Error: Mailbox INBOX: Deleting corrupted cache record uid=371208: UID 371208: Broken physical size in mailbox INBOX: read(/var/mail/mgrant) failed: Cached message size smaller than expected (17212 < 17222, box=INBOX, UID=371208)
> >>>    Error: Mailbox INBOX: UID=371208: read(/var/mail/mgrant) failed: Cached message size smaller than expected (17212 < 17222, box=INBOX, UID=371208) (FETCH BODY[])
> >>>    Error: Mailbox INBOX: Deleting corrupted cache record uid=371203: UID 371203: Broken physical size in mailbox INBOX: read(/var/mail/mgrant) failed: Cached message size smaller than expected (3904 < 3914, box=INBOX, UID=371203)
> >>>    Error: Mailbox INBOX: UID=371203: read(/var/mail/mgrant) failed: Cached message size smaller than expected (3904 < 3914, box=INBOX, UID=371203) (FETCH BODY[])
> >>>
> >>> My inbox is an mbox file.  I'm running dovecot installed on Debian
> >>> Bullseye, the dovecot packages are all: 1:2.3.13+dfsg1-1
> >>>
> >>> I am running sendmail and using procmail for local delivery.
> >>>
> >>> I suspect, but am not certain, that this may be some locking issue
> >>> between procmail and dovecot but I have never been able to prove
> >>> that. The final procmail rule which appends messages to my mailbox
> >>> looks like this, the trailing ':' causes procmail to use a lockfile:
> >>>
> >>> :0:
> >>> /var/mail/mgrant
> >>>
> >>> The locking config lines in 10-mail.conf are commented, but I have
> >>> also tried uncommenting them, did not help:
> >>>
> >>> #mbox_read_locks = fcntl
> >>> #mbox_write_locks = fcntl dotlock
> >>>
> >>> Though sometimes it seems to fix itself after a few hours, the only
> >>> way I have found to fix this quickly is to manually remove the cache
> >>> files and restart dovecot:
> >>>
> >>> rm ~/mail/.imap/INBOX/*
> >>> systemctl restart dovecot
> >>>
> >>> I am not even sure this is a locking issue.  Something definitely gets
> >>> corrupted though. I do have several IMAP clients hitting the same
> >>> mailbox (phone, laptop, desktop).  On the phone, I run K9 and also the
> >>> gmail client which talks imap.  Also using thunderbird, outlook, and
> >>> w10 mail, though typically not all at the same time.  You could
> >>> definitely say I am stress testing this setup a bit!
> >>>
> >>> Any ideas on how to resolve this?
> >> I still see this corruption every day or so.  Anyone have any ideas how to debug this or resolve it?
> >>
> >> Michael Grant
> > Hi!
> >
> > We don't really fix issues with mbox files anymore, other than read issues.. Our focus is enabling people to move to other formats, such as maildir. I would strongly recommend you to consider using maildir instead of mbox.
> >
> > I would also recommend you use dovecot-lda in procmail to deliver mail, if you are not already doing so.
> >
> > Aki
> So please put mbox code read only, or kill it.
> Corruption is not acceptable.
> At it is not at the expected level or quality dovecot used to be or 
> claim to be.
> 
> Mbox code is slow and you will do nothing to get it faster. Ok we could 
> buy it.
> Optimisation and feature on the other formats could make the Mbox code 
> slower and slower because no investements in the Mbox code. Ok too, make 
> sense.
> 
> But no, corruption is not acceptable. It is a bug.
> You're each time mbox corruption reports pops-up ask to switch to 
> another format as the only answer. It make me nervous as beeing in my 
> opinion an unfair and incomplete answer.
> This time I allow myself to react : Please put this code read only or 
> disable it.
> If what you need is funding for proper basic maintenance of R/W (or even 
> RO) Mbox code, it will make it more obvious to your users / customers.
> 
> Regards,
> Emmanuel.

None of our customers use mbox and there is no mail corruption in mbox. Just dovecot cache is disagreeing with the mail contents, which is automatically healed by removing the cache entry from cache.

I am not sure what's going on with that, hard to say without knowing more about how the system is set up and configured.

Aki