Error: Corrupted index cache file
Hi All,
In mail.log exists an error messages "Error: Corrupted index cache file /data/mail/vhosts/domain.com/user@domain.com/Maildir/dovecot.index.cache: invalid record size" for some accounts.
I tried these steps to fix it:
- Running commands
doveadm -D -v index -u user@domain.com INBOX
# doveadm -v force-resync -u user@domain.com INBOX
after this I saw that the following files are updated in ../Maildir/: dovecot.index.cache dovecot.index.log dovecot-uidlist
But after some time error messages occurred again.
Added NFS mount option 'nordirplus' to /etc/fstab with remount /data partition, add 'mmap_disable = yes' and 'mail_fsync = always' to 10-mail.conf as advised on https://doc.dovecot.org/configuration_manual/nfs/ (because /data partition mounted via NFS);
Renaming 3 files with/without Dovecot service pre-stop.
mv dovecot.index old.dovecot.index
# mv dovecot.index.cache old.dovecot.index.cache # mv dovecot.index.log old.dovecot.index.log
Unfortunately, all done steps didn't fix issue. What is the proper way to fix corrupted index cache errors?
dovecot package: 1:2.2.33.2-1ubuntu4.8 (Ubuntu-18.04).
p.s. for some of email accounts also exists error messages "Error: Broken file ../Maildir/dovecot-uidlist line ###: Invalid data:" besides mentioned cache error.
-- Best wishes Sohin Vyacheslav
On 10/02/2023 17:15 EET Sohin Vyacheslav slawa@vivaldi.net wrote:
Hi All,
In mail.log exists an error messages "Error: Corrupted index cache file /data/mail/vhosts/domain.com/user@domain.com/Maildir/dovecot.index.cache: invalid record size" for some accounts.
I tried these steps to fix it:
- Running commands
doveadm -D -v index -u user@domain.com INBOX
# doveadm -v force-resync -u user@domain.com INBOX
after this I saw that the following files are updated in ../Maildir/: dovecot.index.cache dovecot.index.log dovecot-uidlist
But after some time error messages occurred again.
Added NFS mount option 'nordirplus' to /etc/fstab with remount /data partition, add 'mmap_disable = yes' and 'mail_fsync = always' to 10-mail.conf as advised on https://doc.dovecot.org/configuration_manual/nfs/ (because /data partition mounted via NFS);
Renaming 3 files with/without Dovecot service pre-stop.
mv dovecot.index old.dovecot.index
# mv dovecot.index.cache old.dovecot.index.cache # mv dovecot.index.log old.dovecot.index.log
Unfortunately, all done steps didn't fix issue. What is the proper way to fix corrupted index cache errors?
dovecot package: 1:2.2.33.2-1ubuntu4.8 (Ubuntu-18.04).
p.s. for some of email accounts also exists error messages "Error: Broken file ../Maildir/dovecot-uidlist line ###: Invalid data:" besides mentioned cache error.
-- Best wishes Sohin Vyacheslav
It sounds like your NFS is somehow managing to break the files, this can happen easily if same user is accessed by two backends concurrently.
Aki
It sounds like your NFS is somehow managing to break the files, this can happen easily if same user is accessed by two backends concurrently.
how it's possible to resolve?
By not allowing the same user to access two backends concurrently.
One option and the best way is using dovecot director. However, it is soon to be removed from free community dovecot.
Another way is using a proxy (example HaProxy) with sticky IP. The disadvantage of sticky IP is if they are using multiple IP's such as a desktop and mobile connection at the same time.
11.02.2023 07:47, dovecot@ptld.com пишет:
By not allowing the same user to access two backends concurrently.
One option and the best way is using dovecot director. However, it is soon to be removed from free community dovecot.
Another way is using a proxy (example HaProxy) with sticky IP. The disadvantage of sticky IP is if they are using multiple IP's such as a desktop and mobile connection at the same time.
Thank you! In my configuration already uses a Dovecot director and error messages occurred on the backend mail servers (mentioned in option director_mail_servers).
By the way, I saw that in mail.log of 1st backend error messages more than in mail.log of 2nd backend.
-- Best wishes Sohin Vyacheslav
Hi Sohin,
I've run into this error in the past in some situations:
one was during the migration from Centos 6 to 7, probably the NFS client in the kernel had some differences in cache management
one was with Ubuntu and NFS server in Google Cloud, I don't remember exactly how I solved it in that case but the problem was the NFS server (maybe because the NFS server only supported version 4.1 and there were locking issues)
one was where I used Director but local delivery via LDA, I solved it by switching to delivery via LMTP
What NFS server/storage are you using? And with what NFS version your Maildir are mounted?
Are you using LDA or LMTP for delivery?
After what change did the problem start?
Ciao
Il 10/02/23 16:15, Sohin Vyacheslav ha scritto:
Hi All,
In mail.log exists an error messages "Error: Corrupted index cache file /data/mail/vhosts/domain.com/user@domain.com/Maildir/dovecot.index.cache: invalid record size" for some accounts.
I tried these steps to fix it:
- Running commands
doveadm -D -v index -u user@domain.com INBOX
# doveadm -v force-resync -u user@domain.com INBOX
after this I saw that the following files are updated in ../Maildir/: dovecot.index.cache dovecot.index.log dovecot-uidlist
But after some time error messages occurred again.
Added NFS mount option 'nordirplus' to /etc/fstab with remount /data partition, add 'mmap_disable = yes' and 'mail_fsync = always' to 10-mail.conf as advised on https://doc.dovecot.org/configuration_manual/nfs/ (because /data partition mounted via NFS);
Renaming 3 files with/without Dovecot service pre-stop.
mv dovecot.index old.dovecot.index
# mv dovecot.index.cache old.dovecot.index.cache # mv dovecot.index.log old.dovecot.index.log
Unfortunately, all done steps didn't fix issue. What is the proper way to fix corrupted index cache errors?
dovecot package: 1:2.2.33.2-1ubuntu4.8 (Ubuntu-18.04).
p.s. for some of email accounts also exists error messages "Error: Broken file ../Maildir/dovecot-uidlist line ###: Invalid data:" besides mentioned cache error.
-- Alessio Cecchi Postmaster @ http://www.qboxmail.it https://www.linkedin.com/in/alessice
12.02.2023 21:36, Alessio Cecchi пишет:
I've run into this error in the past in some situations:
one was during the migration from Centos 6 to 7, probably the NFS client in the kernel had some differences in cache management
one was with Ubuntu and NFS server in Google Cloud, I don't remember exactly how I solved it in that case but the problem was the NFS server (maybe because the NFS server only supported version 4.1 and there were locking issues)
one was where I used Director but local delivery via LDA, I solved it by switching to delivery via LMTP
What NFS server/storage are you using? And with what NFS version your Maildir are mounted? We use NFS on Ubuntu nfs-kernel-server 1:1.3.4-2.1ubuntu5.5 nfs-common 1:1.3.4-2.1ubuntu5.5
$ nfsstat | grep nfs Server nfs v4:
Now Maildir mounted with these options: type nfs4 (rw,noatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,acregmin=1800,acregmax=1800,acdirmin=1800,acdirmax=1800,hard,nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=IP-address,local_lock=none,addr=ip-address)
Are you using LDA or LMTP for delivery? Uses LMTP for delivery.
After what change did the problem start? It seems that problem exists a long time.
-- Best wishes Sohin Vyacheslav
Hi Sohin,
I don't remember how was solved the problem with Ubuntu, but in my current setup (some hundreds of thousands of mailboxes), and in general with Maildir and small size files I prefer to use NFSv3 that have less "problem" with lock since it is stateless. Try also to remove all "ac" options from fstab, my fstab for NFS Maildir is very simple:
rw,nfsvers=3,noatime,nodiratime,_netdev,nordirplus
But I'm not using Linux as NFS for server. Let me know if you have any improvements with my mount options.
Ciao
Il 13/02/23 10:39, Sohin Vyacheslav ha scritto:
12.02.2023 21:36, Alessio Cecchi пишет:
I've run into this error in the past in some situations:
one was during the migration from Centos 6 to 7, probably the NFS client in the kernel had some differences in cache management
one was with Ubuntu and NFS server in Google Cloud, I don't remember exactly how I solved it in that case but the problem was the NFS server (maybe because the NFS server only supported version 4.1 and there were locking issues)
one was where I used Director but local delivery via LDA, I solved it by switching to delivery via LMTP
What NFS server/storage are you using? And with what NFS version your Maildir are mounted? We use NFS on Ubuntu nfs-kernel-server 1:1.3.4-2.1ubuntu5.5 nfs-common 1:1.3.4-2.1ubuntu5.5
$ nfsstat | grep nfs Server nfs v4:
Now Maildir mounted with these options: type nfs4 (rw,noatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,acregmin=1800,acregmax=1800,acdirmin=1800,acdirmax=1800,hard,nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=IP-address,local_lock=none,addr=ip-address)
Are you using LDA or LMTP for delivery? Uses LMTP for delivery.
After what change did the problem start? It seems that problem exists a long time.
participants (4)
-
Aki Tuomi
-
Alessio Cecchi
-
dovecot＠ptld.com
-
Sohin Vyacheslav