On Monday, Sep 8, 2003, at 21:15 Europe/Helsinki, Eric S. Johansson wrote:
mbox accesse is never going to be fast or efficient. If it was possible to make mbox access faster, it would be because enough bright people have thought about the problem to find a solution if there was one. I'm not saying there isn't an undiscovered magic option like marking a message as replaced, writing an updated copy to the end of the file, and sorting out the order at read time. Only that it's highly unlikely.
Well .. mbox performance does depend on what the client does. Synchronizing can do all kinds of tricks to speed it up when Dovecot is the only one accessing the mbox. I'll probably do these once I get mbox working in general.
Sync optimization would work so that whenever Dovecot modifies mbox or synchronizes it, it keeps the mbox read-locked for at least two seconds. This guarantees that if someone else modifies the mbox, the timestamp changes and we know it. If it doesn't change, our index is up to date and we don't need any syncing.
Only problem is that if there's lots of unsynced mailboxes, we don't really want to just keep waiting for those 2 seconds. We'd rather just want to keep the lock and remove it two seconds later. But we don't necessarily want to keep all the mboxes open and eat all available file decriptors.. Maybe it should keep max. 10 mboxes open before waiting for one of them to finish the wait.
Expunging can't do much optimizations or it wouldn't be mbox-compatible anymore. But luckily most expunges are done for new messages and then it doesn't have to move that much data.
That sync optimization would actually help with your slowdown, if Dovecot also indexed mails at the time they were copied/appended. That also makes UIDPLUS extension possible with little effort..