In case anyone else is experiencing this issue, using the following in a postlogin script is a reasonable workaround:
if [[ ! -e "$HOME/.lastlogin_imap" && "$HOME/Maildir/dovecot.index.cache" -ot "$HOME/Maildir/new" && "$HOME/Maildir/dovecot-uidlist" -ot "$HOME/Maildir/new" ]]; then rm -f "$HOME/Maildir/dovecot.index.cache" fi
(whenever a user logs in via imap we touch .lastlogin_imap, so we only kill the cache file if the user is pop3-only and both the cache and uidlist are older than the new directory).
Mark
-- Mark Zealey -- Shared Hosting Team Leader Product Development * Webfusion 123-reg.co.uk, webfusion.co.uk, donhost.co.uk, supanames.co.uk
This mail is subject to http://www.gxn.net/disclaimer
-----Original Message----- From: dovecot-bounces+mark.zealey=webfusion.com@dovecot.org [mailto:dovecot-bounces+mark.zealey=webfusion.com@dovecot.org] On Behalf Of Mark Zealey Sent: 02 July 2008 11:01 To: Dovecot Mailing List Subject: Re: [Dovecot] Poor pop3 over nfs performance
rc11 had this fix which may be relevant:
- dovecot-uidlist is now recreated if it results in file shrinking over 25%.
Not sure this fix is relevent; but I have upgraded to 1.1.1
With v1.1 INDEX=MEMORY shouldn't matter that much anymore, since it should cache the virtual message size to dovecot-uidlist.
- Do you have ,W=<size> in maildir filenames? Apparently not?
No; yesterday though I did switch exim to delivering through lda - surprisingly though there was not much effect on the graphs.
- Do you have W=<size> added to dovecot-uidlist?
I have found something here which I believe is a bug; will detail below
- Are your POP3 users download + delete or do they leave mail on the server?
A mixture; I believe at least 80% will download+delete; probably closer to 95% but we do have some very large mail stores - most of these would be accessed by a combination of pop and imap.
You should be able to use dotlock_use_excl=yes. ... dotlock is the slowest locking method. Did you try how well fcntl would have worked?
Not tried fcntl; I don't really trust nfs locking. I have enabled the excl in the config; again not much change.
Now what I discovered in testing a particularly problematic user with an 800mb cur directory (under v1.1.1; I have never used v1.0 on this mbox but it was converted with your courier convert script); initially they had the following:
drwx------ 2 exim exim 2048 Apr 11 2007 courierimapkeywords -rw-r--r-- 1 exim exim 47 Apr 22 15:56 courierimapsubscribed -rw-r--r-- 1 exim exim 142493 Jun 23 19:57 courierimapuiddb -rw-r--r-- 1 exim exim 195309 Jun 24 18:52 courierpop3dsizelist drwx------ 2 exim exim 499712 Jul 2 10:35 cur -rw------- 1 exim exim 20912 Jul 2 10:51 dovecot.index -rw------- 1 exim exim 33056 Jun 17 17:34 dovecot.index.cache -rw------- 1 exim exim 31336 Jul 2 10:37 dovecot.index.log -rw------- 1 exim exim 158465 Jul 2 10:13 dovecot-uidlist drwx------ 2 exim exim 4096 Jul 2 10:29 new -rw------- 1 exim exim 23 Jun 24 22:49 subscriptions drwx------ 2 exim exim 2048 Jul 2 10:42 tmp
As you can see, the cache file is very old, and the uidlist file is behind the last time that the logged in (which was 10:35). When I log in over pop and do a stat/list, it re-reads everything in cur/new but didn't update the dovecot-uidlist file or the index.cache file (the log file is updated though). Dovecot-uidlist file does not have the extra W= tags in it. I removed the dovecot.index.cache file and performed the same login/stat/list operation and logged out, and suddenly the dovecot-uidlist file was updated to the new v1.1 format, and the directory listing looks like:
total 1068 drwx------ 2 exim exim 2048 Apr 11 2007 courierimapkeywords -rw-r--r-- 1 exim exim 47 Apr 22 15:56 courierimapsubscribed -rw-r--r-- 1 exim exim 142493 Jun 23 19:57 courierimapuiddb -rw-r--r-- 1 exim exim 195309 Jun 24 18:52 courierpop3dsizelist drwx------ 2 exim exim 499712 Jul 2 10:35 cur -rw------- 1 exim exim 20912 Jul 2 10:53 dovecot.index -rw------- 1 exim exim 31428 Jul 2 10:53 dovecot.index.log -rw------- 1 exim exim 175508 Jul 2 10:53 dovecot-uidlist drwx------ 2 exim exim 4096 Jul 2 10:29 new -rw------- 1 exim exim 23 Jun 24 22:49 subscriptions drwx------ 2 exim exim 2048 Jul 2 10:42 tmp
This looks like a bug to me; I will have a look in the source and see if I can find anything, but I thought I'd send this email as you'll be able to find the bug faster than I can.
Thanks,
Mark