[Dovecot] Investigating suspected cache corruption

[David Warden]

Hi All,

I'm running Dovecot 1.1.11 for a site of about 7000 active users with  
a Squirrelmail frontend. We migrated from Courier (big shout to  
everyone who worked on the wonderful part of the Dovecot wiki that  
deals with Courier migration) and I have been extremely happy with the  
performance gains Dovecot has given us. Not only is the IMAP part of  
Squirrelmail faster, but our same hardware is under far less stress  
than it was before the migration.

To date, the only problem we have experienced is a seemingly random  
but significant slowdown in Squirrelmail's loading of the messages in  
a user's Inbox. Each time, I have been able to fix the problem by  
deleting the user's dovecot.index, dovecot.index.cache and  
dovecot.index.log files in the user's INDEX directory for their  
dovecot Inbox.

I tried having users experiencing this problem log in to a test  
Squirrelmail box with mail_debug on, but I didn't see anything out of  
the ordinary and they report the same slowness on the test server. I  
don't see anything but normal login/logout IMAP lines in the  
production dovecot logs.

I have hesitated a bit in posting to the list since this seems to be  
happening to a small percentage of users and it technically may be a  
problem with Squirrelmail, but the fact that I'm able to fix the  
problem by nuking Dovecot's index files has me concerned - does anyone  
have any suggestions for troubleshooting? My first idea when I get  
another one of these is to do a telnet IMAP session to Dovecot on one  
of the Squirrelmail boxes and see if I can get it to slow down when  
retrieving messages from the Inbox.

We are using Maildir over NFS with the control/index files stored on  
NFS as well. We're doing FS quota over NFS using rquotad to query the  
NFS server for each user's quota. Here's the output of dovecot -n:

# 1.1.11: /etc/dovecot.conf
Warning: fd limit 1024 is lower than what Dovecot can use under full  
load (more than 1536). Either grow the limit or change  
login_max_processes_count and max_mail_processes settings
# OS: Linux 2.6.18-128.1.1.0.1.el5 x86_64 Red Hat Enterprise Linux  
Server release 5.3 (Tikanga)
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(pop3): /usr/libexec/dovecot/pop3-login
login_process_per_connection: no
login_processes_count: 10
login_max_processes_count: 512
verbose_proctitle: yes
first_valid_uid: 0
mail_location: maildir:~/Maildir:CONTROL=/Mail/mailhome/ 
%u/.dovecot:INDEX=/Mail/mailhome/%u/.dovecot
mmap_disable: yes
mail_nfs_storage: yes
mail_nfs_index: yes
mail_drop_priv_before_exec: yes
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(pop3): /usr/libexec/dovecot/pop3
mail_plugins(default): quota imap_quota
mail_plugins(imap): quota imap_quota
mail_plugins(pop3):
mail_plugin_dir(default): /usr/lib64/dovecot/imap
mail_plugin_dir(imap): /usr/lib64/dovecot/imap
mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3
imap_client_workarounds(default): outlook-idle delay-newmail
imap_client_workarounds(imap): outlook-idle delay-newmail
imap_client_workarounds(pop3):
imap_logout_format(default): bytes(in/out)=%i/%o
imap_logout_format(imap): bytes(in/out)=%i/%o
imap_logout_format(pop3): bytes=%i/%o
pop3_uidl_format(default): %08Xu%08Xv
pop3_uidl_format(imap): %08Xu%08Xv
pop3_uidl_format(pop3): UID%u-%v
pop3_client_workarounds(default):
pop3_client_workarounds(imap):
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
pop3_logout_format(default): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s
pop3_logout_format(imap): top=%t/%p, retr=%r/%b, del=%d/%m, size=%s
pop3_logout_format(pop3): top(count/bytes)=%t/%p, retr(count/bytes)=%r/ 
%b, del(deleted/pre-deleted)=%d/%m, mailbox-size(bytes)=%s
namespace:
   type: private
   prefix: INBOX.
   inbox: yes
   list: yes
   subscriptions: yes
auth default:
   mechanisms: plain login
   passdb:
     driver: pam
   userdb:
     driver: passwd
plugin:
   quota: fs:Mail Quota:user

Many thanks,
David Warden