This is nonsense. The problem is that the behavior of readdir() is confusing.
Why should unlink() or rename() invalidate data that your C library ALREADY READ from the directory?
This is like saying "I fgetc()'d a byte, but now lseek() shows that my offset is 1024!" - it's silly.
Just put a: if (stat(d->d_name, &sb) == -1)continue;
After your check for the "." in the first character of the d->d_name (about line 41) and all will be good. No amount of twiddling with USE_UNLINK or FILES is going to affect it.
So you say, "I need to stat() each entry? That's going to create a large number of syscalls!"
Of course. For readdir() to be atomic, it would need to do a system call for each directory entry. This is exactly why readdir() doesn't, so that you do one syscall for every (say) 50 entries, and if you want validity, you'll do a stat() yourself.
Now: Maildir quite obviously wasn't designed with IMAP in mind. IMAP has some (largely ridiculous) requirements that Maildir simply doesn't make easy.
The largest problem (with Maildir) is this renaming of file identifiers and moving things in and out of cur/. It's only necessary so programs don't have to open() in order to read flags (after all, they JUST did a readdir())...
Since the names aren't going to change in cur/, you can get away with just doing a stat() in there [[ after all, you just rename()'d it into cur/ if you're working on new ]]
Unfortunately, cur/ is often bigger than new/.
On Sun, 2004-10-24 at 20:02, Timo Sirainen wrote:
Looks like maildir can't be used very realiably without quite a lot of locking. Writing and scanning the directory would have to be locked, but reading wouldn't (as long as the file hasn't been renamed which would require scanning to find it). So much for "no locks needed"..
The problem is that opendir()/readdir() may temporarily not return some files if there has been changes in the directory since the opendir(). That means Dovecot thinks a message is expunged, while in fact it really isn't, and the next scan would usually show it again.
Currently when that happens, Dovecot usually prints an error message about it and rebuilds indexes. Of course, in real life clients aren't often bombing the same mailbox with tons of changes in multiple connections, which is usually needed to trigger this.
I wrote a test program which tests this:
http://dovecot.org/tmp/readdir.c
I'd like to hear if you can run it in some system without errors. I tested Linux 2.4 and 2.6 with ext2, ext3, xfs and reiser3, Solaris 8/ufs and OpenBSD 3.5/sparc64. Only OpenBSD passed the test, but I'm not sure if it's only because the computer was so slow and didn't switch between processes hard enough. I'd be especially interested about FreeBSD and various NFS systems.
If it actually works properly in some systems, I guess I'll make the extra locking configurable.
Geo Carncross geocar@internetconnection.net Internet Connection Reliable Web Hosting http://www.internetconnection.net/