2.4.4 fts-flatcurve: fts optimize silently destroys a folder's index when the index is on NFS

Hello,

Summary

=======

On Dovecot 2.4.4 with fts_flatcurve and mail_index_path on NFS, "doveadm fts optimize" leaves behind an empty shard directory. That directory remains the shard every subsequent write opens, so from that point on the folder can no longer index new mail. The operation reports success and logs nothing at the time. The damage is permanent and, on a folder that

receives no new mail, invisible.

Two separate defects are involved, and both are reported below:

(1) fts optimize deletes a shard directory whose files it still holds open. On a local filesystem the unlink succeeds invisibly; NFS cannot express "unlink but keep open" and silly-renames instead, so the directory survives.

(2) unlink_directory() reports success when rmdir() failed, which is what makes (1) silent rather than loud.

Environment

===========

Dovecot 2.4.4 CE (8b687aa65c)

container image

sha256:d6b2f80db2e656a9250583712a82397e2752b35ad7e4e2e71c85fdc0d510137d

base Debian 13 trixie

fts_flatcurve (Xapian glass)

libxapian30 1.4.29-3 (libxapian.so.30.14.1)

substring_search = yes, min_term_size = 2

optimize_limit at its default of 10

dovecot_storage_version = 2.3.0

mail_index_path on NFSv3; mail storage on a separate NFS export

NFS client is the host, not the container:

Debian 12 bookworm, kernel 6.1.0-51-amd64 x86_64

nfs-common 1:2.6.2-4+deb12u1

mount options: rw,noatime,vers=3,rsize=32768,wsize=32768,hard,

               nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys,

               local_lock=none

Impact

======

An operator does not have to invoke optimize to be exposed: with the default optimize_limit = 10, flatcurve runs it itself once a folder accumulates ten shards. Every active folder therefore reaches this eventually.

Already-indexed mail stays searchable, so the folder looks healthy. Only indexing of new mail fails. With fts_search_add_missing = yes, the first SEARCH after new mail arrives attempts to index it, fails, and the client receives NO [SERVERBUG].

I hit this while migrating a host from 2.3 to 2.4: 114 mailboxes were left in this state in a single run, and it surfaced only because a monitoring mailbox began answering SERVERBUG.

Defect 1: fts optimize leaves an empty shard directory on NFS

=============================================================

Steps to reproduce

1. Configure fts_flatcurve with the index path on an NFSv3 mount.

2. Take a mailbox with mail in INBOX (a few hundred messages is enough) and build the index:

   doveadm index -u <user> INBOX

   Confirm one shard exists:

   ls <index>/.INBOX/fts-flatcurve/ -> current.<id>

3. Run:

   doveadm fts optimize -u <user> INBOX

   (Alternatively, let it fire on its own by driving the folder past optimize_limit shards.)

4. While it runs, list the shard directory repeatedly - .nfsXXXX entries appear inside current.<id>.

5. After it finishes, note the exit status is 0 and no error was logged.

   List the directory again: index.<n> was created as expected, and current.<id> is still there, now empty.

6. Deliver one new message, then:

   doveadm index -u <user> INBOX

Expected vs actual

Expected: after optimize, only the compacted index.<n> remains, and subsequent indexing writes to a new shard.

Actual: the stale empty current.<id> remains and is reopened for writing, so step 6 fails:

Cannot open DB (RW, current.<id>): .../current.<id>/iamglass:

Failed to open glass revision file for reading

(No such file or directory)

cmd index: Mailbox INBOX: Precache for UID=59 failed

and IMAP SEARCH on that folder returns NO [SERVERBUG].

What I observed while diagnosing

strace of "doveadm fts optimize" on NFSv3 shows the old shard's files being unlinked while handles on them are still open, so the directory cannot be removed:

unlink(...) = 0 (x7, every file in the shard)

rmdir(...) = -1 ENOTEMPTY (Directory not empty)

rename(...) = 0 (optimize -> index.N, proceeds regardless)

Once the process exits, the NFS client reaps the .nfsXXXX files and the directory is simply empty.

What does not help

NFSv4.1, mounting with nolock, "doveadm fts rescan", "doveadm index", and full re-indexing all take the same write path into the same empty shard and fail identically. Removing the empty directory (or the folder's whole fts-flatcurve tree) and re-indexing restores the folder; setting optimize_limit = 0 avoids the situation but leaves the index spread across many shards.

Defect 2: unlink_directory() returns success when rmdir() failed

================================================================

The ENOTEMPTY above never reaches the caller. unlink_directory() in

src/lib/unlink-directory.c returns 1 when the final rmdir() fails, the

same value it returns on success:

if (rmdir(dir) < 0 && errno != ENOENT) {

      *error_r = t_strdup_printf("rmdir(%s) failed: %m", dir);

      return errno == ENOENT ? 0 : 1;

}

The "errno != ENOENT" guard already establishes that errno is not ENOENT, so the ternary always yields 1. Callers cannot distinguish "directory removed" from "directory still there", and the error message it composed is discarded.

Any condition that makes rmdir() fail with something other than ENOENT reproduces this, so the return value is wrong independently of NFS - every caller of unlink_directory() is affected. The code path is unchanged in current main.

--

Best regards,

Ihor Ru

Hello,

Summary

=======

On Dovecot 2.4.4 with fts_flatcurve and mail_index_path on NFS, "doveadm fts optimize" leaves behind an empty shard directory. That directory remains the shard every subsequent write opens, so from that point on the folder can no longer index new mail. The operation reports success and logs nothing at the time. The damage is permanent and, on a folder that

receives no new mail, invisible.

Two separate defects are involved, and both are reported below:

 (1) fts optimize deletes a shard directory whose files it still holds

open. On a local filesystem the unlink succeeds invisibly; NFS cannot express "unlink but keep open" and silly-renames instead, so the directory survives.

 (2) unlink_directory() reports success when rmdir() failed, which is

what makes (1) silent rather than loud.

Environment

===========

 Dovecot 2.4.4 CE (8b687aa65c)

   container image

sha256:d6b2f80db2e656a9250583712a82397e2752b35ad7e4e2e71c85fdc0d510137d

   base Debian 13 trixie

 fts_flatcurve (Xapian glass)

   libxapian30 1.4.29-3 (libxapian.so.30.14.1)

 substring_search = yes, min_term_size = 2

 optimize_limit at its default of 10

 dovecot_storage_version = 2.3.0

 mail_index_path on NFSv3; mail storage on a separate NFS export

 NFS client is the host, not the container:

   Debian 12 bookworm, kernel 6.1.0-51-amd64 x86_64

   nfs-common 1:2.6.2-4+deb12u1

   mount options: rw,noatime,vers=3,rsize=32768,wsize=32768,hard,

                  nordirplus,proto=tcp,timeo=600,retrans=2,sec=sys,

                  local_lock=none

Impact

======

An operator does not have to invoke optimize to be exposed: with the default optimize_limit = 10, flatcurve runs it itself once a folder accumulates ten shards. Every active folder therefore reaches this eventually.

Already-indexed mail stays searchable, so the folder looks healthy. Only indexing of new mail fails. With fts_search_add_missing = yes, the first SEARCH after new mail arrives attempts to index it, fails, and the client receives NO [SERVERBUG].

I hit this while migrating a host from 2.3 to 2.4: 114 mailboxes were left in this state in a single run, and it surfaced only because a monitoring mailbox began answering SERVERBUG.

Defect 1: fts optimize leaves an empty shard directory on NFS

=============================================================

Steps to reproduce

1. Configure fts_flatcurve with the index path on an NFSv3 mount.

2. Take a mailbox with mail in INBOX (a few hundred messages is enough) and build the index:

   doveadm index -u <user> INBOX

   Confirm one shard exists:

   ls <index>/.INBOX/fts-flatcurve/ -> current.<id>

3. Run:

   doveadm fts optimize -u <user> INBOX

   (Alternatively, let it fire on its own by driving the folder past optimize_limit shards.)

4. While it runs, list the shard directory repeatedly - .nfsXXXX entries appear inside current.<id>.

5. After it finishes, note the exit status is 0 and no error was logged.

   List the directory again: index.<n> was created as expected, and current.<id> is still there, now empty.

6. Deliver one new message, then:

   doveadm index -u <user> INBOX

Expected vs actual

Expected: after optimize, only the compacted index.<n> remains, and subsequent indexing writes to a new shard.

Actual: the stale empty current.<id> remains and is reopened for writing, so step 6 fails:

 Cannot open DB (RW, current.<id>): .../current.<id>/iamglass:

   Failed to open glass revision file for reading

   (No such file or directory)

 cmd index: Mailbox INBOX: Precache for UID=59 failed

and IMAP SEARCH on that folder returns NO [SERVERBUG].

What I observed while diagnosing

strace of "doveadm fts optimize" on NFSv3 shows the old shard's files being unlinked while handles on them are still open, so the directory cannot be removed:

 unlink(...) = 0                 (x7, every file in the shard)

 rmdir(...)  = -1 ENOTEMPTY      (Directory not empty)

 rename(...) = 0                 (optimize -> index.N, proceeds

regardless)

Once the process exits, the NFS client reaps the .nfsXXXX files and the directory is simply empty.

What does not help

NFSv4.1, mounting with nolock, "doveadm fts rescan", "doveadm index", and full re-indexing all take the same write path into the same empty shard and fail identically. Removing the empty directory (or the folder's whole fts-flatcurve tree) and re-indexing restores the folder; setting optimize_limit = 0 avoids the situation but leaves the index spread across many shards.

Defect 2: unlink_directory() returns success when rmdir() failed

================================================================

The ENOTEMPTY above never reaches the caller. unlink_directory() in

src/lib/unlink-directory.c returns 1 when the final rmdir() fails, the

same value it returns on success:

 if (rmdir(dir) < 0 && errno != ENOENT) {

         *error_r = t_strdup_printf("rmdir(%s) failed: %m", dir);

         return errno == ENOENT ? 0 : 1;

 }

The "errno != ENOENT" guard already establishes that errno is not ENOENT, so the ternary always yields 1. Callers cannot distinguish "directory removed" from "directory still there", and the error message it composed is discarded.

Any condition that makes rmdir() fail with something other than ENOENT reproduces this, so the return value is wrong independently of NFS - every caller of unlink_directory() is affected. The code path is unchanged in current main.

--

Best regards,

Ihor Ru