[Dovecot] Investigating suspected cache corruption

[Maria Elinger]

Hi David and all, 
I´ve read the mail you posted in the Dovecot list.
We´re are having this same problem in a site which is very similar to yours. 
We also migrated from Courier to Dovecot nearly 9 months ago, and every now and then users complain about slowness in 
the imap connection. We fix it by removing the index files in the user's maildir.
 
We use Horde as webmail, but the problem appears with any imap or pop connection, so we feel that the problem is in Dovecot.
 
Did you find any solution or workaround at your site?, it would be very useful if you gave us a hint on this.
 
Thanks in advance,
María Elinger
 
 
 

[Dovecot] Investigating suspected cache corruption


David Warden
Wed, 14 Oct 2009 13:18:17

 
Hi All,

I'm running Dovecot 1.1.11 for a site of about 7000 active users with a Squirrelmail frontend. We migrated from Courier (big shout to everyone who worked on the wonderful part of the Dovecot wiki that deals with Courier migration) and I have been extremely happy with the performance gains Dovecot has given us. Not only is the IMAP part of Squirrelmail faster, but our same hardware is under far less stress than it was before the migration. 
To date, the only problem we have experienced is a seemingly random but significant slowdown in Squirrelmail's loading of the messages in a user's Inbox. Each time, I have been able to fix the problem by deleting the user's dovecot.index, dovecot.index.cache and dovecot.index.log files in the user's INDEX directory for their dovecot Inbox. 
I tried having users experiencing this problem log in to a test Squirrelmail box with mail_debug on, but I didn't see anything out of the ordinary and they report the same slowness on the test server. I don't see anything but normal login/logout IMAP lines in the production dovecot logs. 
I have hesitated a bit in posting to the list since this seems to be happening to a small percentage of users and it technically may be a problem with Squirrelmail, but the fact that I'm able to fix the problem by nuking Dovecot's index files has me concerned - does anyone have any suggestions for troubleshooting? My first idea when I get another one of these is to do a telnet IMAP session to Dovecot on one of the Squirrelmail boxes and see if I can get it to slow down when retrieving messages from the Inbox. 
We are using Maildir over NFS with the control/index files stored on NFS as well. We're doing FS quota over NFS using rquotad to query the NFS server for each user's quota. Here's the output of dovecot -n: 
# 1.1.11: /etc/dovecot.conf
Warning: fd limit 1024 is lower than what Dovecot can use under full load (more than 1536). Either grow the limit or change login_max_processes_count and max_mail_processes settings # OS: Linux 2.6.18-128.1.1.0.1.el5 x86_64 Red Hat Enterprise Linux Server release 5.3 (Tikanga) 
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(pop3): /usr/libexec/dovecot/pop3-login
login_process_per_connection: no
login_processes_count: 10
login_max_processes_count: 512
verbose_proctitle: yes
first_valid_uid: 0
mail_location: maildir:~/Maildir:CONTROL=/Mail/mailhome/ %u/.dovecot:INDEX=/Mail/mailhome/%u/.dovecot 
mmap_disable: yes
mail_nfs_storage: yes
mail_nfs_index: yes
mail_drop_priv_before_exec: yes
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(pop3): /usr/libexec/dovecot/pop3
mail_plugins(default): quota imap_quota
mail_plugins(imap): quota imap_quota
mail_plugins(pop3):
mail_plugin_dir(default): /usr/lib64/dovecot/imap
mail_plugin_dir(imap): /usr/lib64/dovecot/imap
mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3
imap_client_workarounds(default): outlook-idle delay-newmail
imap_client_workarounds(imap): outlook-idle delay-newmail
imap_client_workarounds(pop3):
imap_logout_format(default): bytes(in/out)=%i/%o
imap_logout_format(imap): bytes(in/out)=%i/%o
imap_logout_format(pop3): bytes=%i/%o
pop3_uidl_format(default): %08Xu%08Xv
pop3_uidl_format(imap): %08Xu%08Xv
pop3_uidl_format(pop3): UID%u-%v
pop3_client_workarounds(default):
pop3_client_workarounds(imap):
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s
pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s
pop3_logout_format(pop3): top(count/bytes)=%t/%p, retr(count/bytes)=%r/ %b, del(deleted/pre-deleted)=%d/%m, mailbox-size(bytes)=%s 
namespace:
  type: private
  prefix: INBOX.
  inbox: yes
  list: yes
  subscriptions: yes
auth default:
  mechanisms: plain login
  passdb:
    driver: pam
  userdb:
    driver: passwd
plugin:
  quota: fs:Mail Quota:user

Many thanks,
David Warden