huge mdbox broken / Log synchronization error

[Timo Sirainen]

On 7. May 2021, at 16.16, Florian Lohoff <f at zz.de> wrote:
> 
> On Fri, May 07, 2021 at 09:17:26AM +0200, Florian Lohoff wrote:
>> So i am pretty shure nothing else is accessing this mailbox - This morning 
>> i opened the screen session and i am here:
> 
> Okay - ran doveadm force-resync and it finished without further notices.
> 
> I changed password back and immediatly the web frontend started to try
> to fetch mails. (No deliveries enabled).
> 
> First accesses took ages - it seems the imapd tried
> to rebuild cache files or the like - nothing in the log.
> 
> After ~10 Minutes this happened and now the mailbox is broken again:
> 
> May 07 16:07:44 imap(inbox at wasteland)<26758><5rud0r3BwL0qAQT4AgGS8wAAAAAAAA8Q>: Warning: Transaction log file /var/vmail/wasteland/inbox/mdbox/dovecot.list.index.log was locked for 554 seconds (rotating while syncing)
> May 07 16:07:44 imap(inbox at wasteland)<30094><eM6p6r3BqsEqAQT4AgGS8wAAAAAAAA8Q>: Warning: Locking transaction log file /var/vmail/wasteland/inbox/mdbox/dovecot.list.index.log took 151 seconds (syncing)
> May 07 16:07:44 imap(inbox at wasteland)<26758><5rud0r3BwL0qAQT4AgGS8wAAAAAAAA8Q>: Info: Connection closed (UID FETCH finished 0.000 secs ago) in=101 out=973 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0
> May 07 16:07:47 imap(inbox at wasteland)<26219><OcR9zr3BMr0qAQT4AgGS8wAAAAAAAA8Q>: Error: Corrupted transaction log file /var/vmail/wasteland/inbox/mdbox/dovecot.list.index.log seq 3: indexid changed: 1620395840 -> 1620395910 (sync_offset=0)
> May 07 16:07:47 imap(inbox at wasteland)<26219><OcR9zr3BMr0qAQT4AgGS8wAAAAAAAA8Q>: Warning: Transaction log file /var/vmail/wasteland/inbox/mdbox/dovecot.list.index.log was locked for 627 seconds (rotating while syncing)
..
> My feeling tells me that mdbox is not very stable. If you hammer hard enough,
> or your access times are slow enough, or your mailbox is large enough it will
> eat its own mailbox for breakfast.

All these errors were also about dovecot.list.index, not mdbox. The "indexid changed" means that the index files were recreated. That does seem to be a bug, since there's no reason for it to recreate them. Anyway, as suggested in the previous mail, disabling mailbox list indexes should avoid these issues. Also 220k folders is way much more than I ever thought anyone using.