Timo Sirainen wrote:
Index File
[...] The file is modified by creating first exclusively owned "index.lock" file, updating it and then rename()ing it over the index file. This lock file shouldn't be held for a long time, so if it exists, and it doesn't get modified at all within 30 seconds, it can be overwritten. Before rename()ing over the index file, you should check with stat() that the index.lock file is still the same as you expect it to be (in case your process was temporarily stopped for over 30 secs).
The locking and handling of the index strikes me as a regression back to the mbox problems that Maildir tried to solve.
Have you considered other approaches, such as having the index be under control of a daemon, and use IPC to communicate events to that daemon, which could then exclusively handle modifying the file?
Or maybe a fixed size record structure, which would permit more of a random record access and modification with less chance of inter process conflicts?
Any way you slice it, though, these are just approximations of a database server. Maybe embedding SQLite (just for indexes) is the answer?
-Tom