[Dovecot] cache files corruption
Here what one of my users sometimes gets in logs when using IMAP:
dovecot: IMAP(skribtsov@pawlin.mass.hc.ru): Disconnected in IDLE dovecot: IMAP(skribtsov@pawlin.mass.hc.ru): Maildir /usr/local/mail/pawlin.mass.hc.ru/skribtsov/Maildir sync: UID inserted in the middle of mailbox (59 > 16, file = 1198267793.H747026P68033.mail5.hc.ru:2,RS) dovecot: IMAP(skribtsov@pawlin.mass.hc.ru): Disconnected: Mailbox is in inconsistent state, please relogin. dovecot: IMAP(skribtsov@pawlin.mass.hc.ru): Maildir /usr/local/mail/pawlin.mass.hc.ru/skribtsov/Maildir sync: UID inserted in the middle of mailbox (59 > 16, file = 1198267793.H747026P68033.mail5.hc.ru:2,RS) dovecot: IMAP(skribtsov@pawlin.mass.hc.ru): Disconnected: Mailbox is in inconsistent state, please relogin. dovecot: IMAP(skribtsov@pawlin.mass.hc.ru): Corrupted transaction log file /usr/local/mail/pawlin.mass.hc.ru/skribtsov/Maildir/dovecot.index.log.2: invalid indexid (1198254020 != 1198498301) dovecot: IMAP(skribtsov@pawlin.mass.hc.ru): Corrupted index cache file /usr/local/mail/pawlin.mass.hc.ru/skribtsov/Maildir/dovecot.index.cache: indexid changed
After that, some messages could "disappear" from messages list. And then maybe "appear" again. When I delete dovecot.* files, all goes ok. I use NFS storage.
BTW, it seems like this problem exists only when working with M$ Outlook :)
# 1.0.5: /usr/local/etc/dovecot.conf syslog_facility: local7 protocols: imap pop3 imaps pop3s ssl_cert_file: /etc/ssl/mail5.hc.ru.pem ssl_key_file: /etc/ssl/mail5.hc.ru.pem disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_greeting: Mail server ready. login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no login_max_processes_count: 256 max_mail_processes: 2048 verbose_proctitle: yes first_valid_uid: 90 last_valid_uid: 65000 first_valid_gid: 90 last_valid_gid: 90 mail_extra_groups: mail mail_location: maildir:~/Maildir mmap_disable: yes lock_method: dotlock mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 imap_client_workarounds(default): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(pop3): outlook-idle pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login cram-md5 verbose: yes debug: yes debug_passwords: yes worker_max_count: 150 passdb: driver: sql args: /usr/local/etc/dovecot-sql.conf userdb: driver: prefetch userdb: driver: sql args: /usr/local/etc/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: vexim plugin: quota: maildir
-- Dmitry Pryadko http://www.hc.ru
On Mon, 2007-12-24 at 17:08 +0300, Dmitry Pryadko wrote:
/usr/local/mail/pawlin.mass.hc.ru/skribtsov/Maildir sync: UID inserted in the middle of mailbox (59 > 16, file = 1198267793.H747026P68033.mail5.hc.ru:2,RS) .. After that, some messages could "disappear" from messages list. And then maybe "appear" again. When I delete dovecot.* files, all goes ok.
Doesn't it get fixed itself without deleting them?
I use NFS storage.
Do you use one or more Dovecot servers?
Timo Sirainen пишет:
On Mon, 2007-12-24 at 17:08 +0300, Dmitry Pryadko wrote:
/usr/local/mail/pawlin.mass.hc.ru/skribtsov/Maildir sync: UID inserted in the middle of mailbox (59 > 16, file = 1198267793.H747026P68033.mail5.hc.ru:2,RS)
..
After that, some messages could "disappear" from messages list. And then maybe "appear" again. When I delete dovecot.* files, all goes ok.
Doesn't it get fixed itself without deleting them?
Well I am not sure, this user was so angry that I had to insert dovecot.* files deletion into crontab, every 5 minutes. Strange that my mail cluster handles over 40 000 mailboxes, but logs say that only few have such problem.
I use NFS storage.
Do you use one or more Dovecot servers?
Two dovecot servers with round-robin pf balancing and NetApp NFS storage over gigabit ethernet
-- Dmitry Pryadko http://www.hc.ru
On 28.12.2007, at 8.48, Dmitry Pryadko wrote:
I use NFS storage.
Do you use one or more Dovecot servers?
Two dovecot servers with round-robin pf balancing and NetApp NFS
storage over gigabit ethernet
That pretty much explains it then. See http://wiki.dovecot.org/NFS
Timo Sirainen пишет:
On 28.12.2007, at 8.48, Dmitry Pryadko wrote:
I use NFS storage.
Do you use one or more Dovecot servers?
Two dovecot servers with round-robin pf balancing and NetApp NFS storage over gigabit ethernet
That pretty much explains it then. See http://wiki.dovecot.org/NFS
Thanks! It seems that my problem is fixed in v1.1 with mail_nfs_storage = yes and mail_nfs_index = yes, am I right ?
-- Dmitry Pryadko http://www.hc.ru
On Fri, 2007-12-28 at 18:24 +0300, Dmitry Pryadko wrote:
Timo Sirainen пишет:
On 28.12.2007, at 8.48, Dmitry Pryadko wrote:
I use NFS storage.
Do you use one or more Dovecot servers?
Two dovecot servers with round-robin pf balancing and NetApp NFS storage over gigabit ethernet
That pretty much explains it then. See http://wiki.dovecot.org/NFS
Thanks! It seems that my problem is fixed in v1.1 with mail_nfs_storage = yes and mail_nfs_index = yes, am I right ?
Yep. Although it still would work faster if you could do per-user (or at least per-IP) redirecting instead of randomly directing the connections.
IIRC in my stress tests accessing a mailbox from only one server was 2-5 times faster than if it was accessed from two servers simultaneously.
participants (2)
-
Dmitry Pryadko
-
Timo Sirainen