[Dovecot] Various uidlist and index errors with 1.1 on NFS

[Adam McDougall]

On Sun, Nov 11, 2007 at 06:19:21AM +0200, Timo Sirainen wrote:

  On Sat, 2007-11-10 at 19:51 -0500, Adam McDougall wrote:
  > On Sun, Nov 11, 2007 at 02:34:33AM +0200, Timo Sirainen wrote:
  > 
  >   I'm beginning to understand the problem. Or at least one of them. :)
  >   
  >   Dovecot is currently flushing attribute cache for files it wants to
  >   stat/open. But that's not enough. It also has to flush attribute cache
  >   for the directory, or it might stat/open a file that has already been
  >   unlinked (but which still has links somewhere in the filesystem so that
  >   it doesn't fail with ESTALE).
  >   
  >   dotlock_use_excl=yes seems to help somewhat with this problem for now.
  >   It also gives better performance.
  >   
  > I'm assuming that only helps if I use dotlock locking, right?
  
  No, maildir always uses dotlocks.
  
  This is getting difficult though. After many hours I've finally gotten
  it to work properly without indexes. But even that required a pretty
  evil hack. It looks like the only way I can get FreeBSD to flush its
  filename cache (or whatever it's called that maps filenames to inodes)
  is to call rmdir() to the directory and hope that it fails with
  ENOTEMPTY. So this can't be safely done if the directory may be empty,
  as is easily possible with Maildir/cur and Maildir/new directories..

Wow, scary, but at least finding one awful way to make it work is starting
on the road to finding a less awful way :) If I had more time, I'd start
looking around the kernel source myself, but I have to get the rest of this
project off the ground.
  
  With indexes enabled it then starts giving errors immediately about
  transaction logs. I think I'll leave figuring that out for later. I'm
  guessing the read cache flushing code doesn't work properly either.

No problem, thanks for looking into it.  I think I will be satisfied for 
now with per-server index directories.  Do you think it should be safe to
have them on NFS as long as only one host accesses that index directory?
(with possible, and likely multiple dovecot processes for one user on that
host)  Also, should I start using dotlock_use_excl anyway as long as it 
appears to work?
  
  I hope Linux supports cache flushes better..