[Dovecot] readdir and rename don't mix; patch included
Enclosed please find a patch to dovecot-1.1.16 that fixes a common problem we see under load on Mac OS X and HFS+. The symptom is that sometimes readdir() returns EINVAL. The problem is that readdir() and rename() don't mix well, and maildir_scan_dir() renames messages from new/ to cur/. The solution I chose is just to retry scanning the directory, since code to rescan is already present. An alternative solution would be to read the entire directory first, then process the contents; that is, to do all the readdirs before any renames. The patch is like Apple's previous ones, all tagged with /*APPLE*/ and formatted to simplify our merges. Please reformat it as you desire. It should also apply easily to 1.2.0. --- dovecot-1.1.16/src/lib-storage/index/maildir/maildir-sync.c 2009-05-20 16:42:49.000000000 -0500 +++ dovecot/src/lib-storage/index/maildir/maildir-sync.c 2009-07-07 19:50:37.000000000 -0500 @@ -354,7 +354,8 @@ return -1; } -static int maildir_scan_dir(struct maildir_sync_context *ctx, bool new_dir) +static int maildir_scan_dir(struct maildir_sync_context *ctx, bool new_dir, + bool final) /* APPLE */ { struct mail_storage *storage = &ctx->mbox->storage->storage; const char *path; @@ -486,6 +487,11 @@ } } + /* APPLE - rename can cause readdir to fail with EINVAL; force + quiet rescan unless this is the final such rescan already */ + if (errno == EINVAL && move_count && !final) + move_count = MAILDIR_RENAME_RESCAN_COUNT + 1; + else /* APPLE reduce code deltas */ if (errno != 0) { mail_storage_set_critical(storage, "readdir(%s) failed: %m", path); @@ -772,7 +778,8 @@ that new/ dir is checked as well. it's a good idea anyway. */ unsigned int count = 0; - while ((ret = maildir_scan_dir(ctx, TRUE)) > 0) { + while ((ret = maildir_scan_dir(ctx, TRUE, + count == MAILDIR_SCAN_DIR_MAX_COUNT)) > 0) { /* APPLE */ /* rename()d at least some files, which might have caused some other files to be missed. check again (see MAILDIR_RENAME_RESCAN_COUNT). */ @@ -783,7 +790,8 @@ return -1; if (cur_changed) { - if (maildir_scan_dir(ctx, FALSE) < 0) + if (maildir_scan_dir(ctx, FALSE, + FALSE) < 0) /* APPLE */ return -1; }
On Tue, 2009-07-07 at 20:20 -0500, Mike Abbott wrote:
Enclosed please find a patch to dovecot-1.1.16 that fixes a common
problem we see under load on Mac OS X and HFS+.The symptom is that sometimes readdir() returns EINVAL.
Committed: http://hg.dovecot.org/dovecot-1.2/rev/4e74f4651f5b
and immediately noticed I forgot: http://hg.dovecot.org/dovecot-1.2/rev/797474444551
participants (2)
-
Mike Abbott
-
Timo Sirainen