On 30/08/2024 15:32 EEST Nikolaos Pyrgiotis via dovecot dovecot@dovecot.org wrote:
Hello,
We have been running dovecot version 2.3.21 since July on debian bookworm system. After upgrading we have seen in the logs similar messages to the following:
2024-08-29T19:33:27.755399+03:00 server1 dovecot: lmtp(user1@modulus.gr)<481240><7TMeHNei0GYpeTkA1g8ogQ:P1>: Warning: mdbox /path/user1/mdbox/storage: Inconsistency in map index (2213,40 != 2213,56). 2024-08-29T19:33:27.755399+03:00 server1 dovecot: lmtp(user1@modulus.gr)<481240><7TMeHNei0GYpeTkA1g8ogQ:P1>: Warning: mdbox /path/user1/mdbox/storage: Inconsistency in map index (2213,40 != 2213,56) 2024-08-29T19:33:27.838990+03:00 server1 dovecot: lmtp(user1@modulus.gr)<481240><7TMeHNei0GYpeTkA1g8ogQ:P1>: sieve: msgid=39f9c444-0c08-4bda-bfc9-512c34584470@modulus.gr: fileinto action: stored mail into mailbox 'Replies' 2024-08-29T19:33:27.840222+03:00 server1 dovecot: lmtp(user1@modulus.gr)<481240><7TMeHNei0GYpeTkA1g8ogQ:P1>: Warning: fscking index file /path/user1/mdbox/storage/dovecot.map.index 2024-08-29T19:33:27.953281+03:00 server1 dovecot: lmtp(user1@modulus.gr)<481240><7TMeHNei0GYpeTkA1g8ogQ:P1>: Warning: mdbox /path/user1/mdbox/storage: rebuilding indexes 2024-08-29T19:37:27.989792+03:00 server1 dovecot: lmtp(user1@modulus.gr)<481249>vhzsLROj0GYQVgcAfAk26A:P1: Error: Timeout (180s)while waiting for lock for transaction log file /path/user1/mdbox/storage/dovecot.map.index.log (WRITE lock held by pid 481240) 2024-08-29T19:37:27.995913+03:00 server1 dovecot: lmtp(user1@modulus.gr)<481249>vhzsLROj0GYQVgcAfAk26A:P1: Error: sieve: msgid=39f9c444-0c08-4bda-bfc9-512c34584470@modulus.gr: fileinto action: failed to store into mailbox 'Mailbox': Timeout (180s) while waiting for lock for transaction log file /path/user1/mdbox/storage/dovecot.map.index.log (WRITE lock held by pid 481240)
This has happened a few times since we upgraded always by a lmtp process. It seems to be happening randomly and we have not been able to reproduce it.
The mailboxes are stored in a glusterfs server, and we use the dovecot director with shared mailboxes architecture.
Usually this was mitigated automatically by dovecot index rebuilding, but last time it happened on a mailbox with a lot of giga byte of mail data the following logs were produced for more than an hour:
2024-08-29T19:45:30.506694+03:00 server1 dovecot: lmtp(user1@modulus.gr)<481240><7TMeHNei0GYpeTkA1g8ogQ:P1>: Error: mmap_anon(/path/user/mdbox/mailboxes/Mailbox/dbox-Mails/dovecot.index.cache, 74076160) failed: Cannot allocate memory
During that period segfaults starting being generated for the imap processes accessing that mailbox probably caused by timeouts from imap processes attempting to fetch emails from the affected mailbox:
[3912187.557029] imap[481783]: segfault at 38 ip 00007fb03248abf8 sp 00007ffc88b264f0 error 4 in libdovecot-storage.so.0.0.0[7fb0323c7000+d8000] likely on CPU 6 (core 3, socket 0)
Are you by chance accessing user's mailbox from multiple different servers, like delivering via LMTP on one, and user accessing on another?
Also you probably want to hike up the vsize_limit, the default 256M is usually too little for mail processses. mmap'd files are counted against this limit.
Aki