On 17 Feb 2003, Timo Sirainen wrote:
On Mon, 2003-02-17 at 20:59, Andreas Aardal Hanssen wrote: But looks like I've missed the ENOTEMPTY check there, adding. I'm renaming directories there, not files.
My bad.
/* move the file into new/ directory - syncing will pick it up from there */ if (rename(tmp_path, new_path) == 0) failed = FALSE;Here you can lose emails if the new/ folder contains a message whose base name is equal.
In theory, yes. In practice, I'd say not. It can only happen with broken MUAs, are there any? User could of course deliberately break it, but is there some gain in it?
It seems the qmail community disagrees with you here - I followed the discussion on the qmail mailing list. I guess the Dovecot community will be knocking on your door when important emails start disappearing unprovoked, and Dovecot is the service that deleted them.
The bottom line is that you can not rely on other Maildir clients following your server's conventions. They will only oblige to the Maildir standard, and using Dovecot, emails will disappear. But there's no sweat - it's a simple fix.
The only way to avoid this is:
- use link and unlink, not rename ..and if it crashes (or loses NFS link) between those calls, you'll suddenly see two mails. I prefer atomic operations.
Well, then Dovecot has to have a way to communicate this to the sysadmin as an error condition. It's indefinitely better than deleting innocent (blog) users' emails.
Anyway, Maildir has a strict consistency criteria which says that all messages that are linked from tmp/ into new/ _must_ use time(NULL) plus a number that is guaranteed not to lapse within one second, and which does not collide with other messages in new/. new/ is the single entry point into cur/, and messages in new/ can not be "older" than the messages in cur/. It only has requirement that the file name is unique. There's other ways to do that than time(NULL). http://cr.yp.to/proto/maildir.html lists some.
"A unique name has three pieces, separated by dots. On the left is the result of time() or the second counter from gettimeofday(). On the right is the result of gethostname(). (To deal with invalid host names, replace / with \057 and : with \072.) In the middle is a delivery identifier, discussed below."
Quite clear.
flags. It follows that when moving a message from new/ to cur/, it is required that the server only picks messages that are older than one second. I'm not sure what you mean by this.. What's special in files that were just created? What changes after it's older than one second? Or are you changing the base name when moving it to cur/? (What makes you sure that the new base name still doesn't exist?)
I create a message in tmp and link this to new, then unlink the original.
I create a new message in tmp. I'm about to link this message to new/, when suddenly
You move my original message from new/ to cur/.
I now complete my link from tmp/ to new/.
You move my new message from new/ to cur.
Boom. My original message is gone. What's the point of only moving messages from new/ older than one second?
All messages delivered to new/ in the same second will get a new middle-part name, sometimes with a _0, _1 ... _n next to the pid. When you move the messages from new/ to cur/ after one second, all new messages delivered to new/ will have a different time part. No collisions. But if you move the messages _before_ one second has lapsed, then a message in cur/ may be overwritten when you either
- move the message from new/ to cur/
- change the flags of two messages whose base is equal.
Using link avoids the loss of emails, and allows the sysadmin to clean up the server's buggy behavior. rename means bye-bye.
I don't think you should rely on checking timestamps in any case. What if the maildir is accessed via NFS and some other computer with different time created the file?
Maildir relies on timestamps - like it or not.
But about the flags, looks like maildir spec says they could be used only in cur/ directory, so I guess my code is broken because it allows setting them already in new/.. Well, I'll fix it anyway later by moving mails directly from tmp/ into cur/ when implementing UIDPLUS extension.
You will lose emails if you move messages from tmp to cur directly. Simply because you have no idea wether or not messages in cur have the same base as your message in tmp.
Andy
-- Andreas Aardal Hanssen http://www.andreas.hanssen.name/gpg