[dovecot] Re: [bincimap] Re: Re: bincimap

[Andreas Aardal Hanssen]

On 17 Feb 2003, Timo Sirainen wrote:
>On Mon, 2003-02-17 at 20:59, Andreas Aardal Hanssen wrote:
>But looks like I've missed the ENOTEMPTY check there, adding. I'm
>renaming directories there, not files.

My bad.

>>         /* move the file into new/ directory - syncing will pick it
>>            up from there */
>>         if (rename(tmp_path, new_path) == 0)
>>                 failed = FALSE;
>> Here you can lose emails if the new/ folder contains a message whose base 
>> name is equal.
>In theory, yes. In practice, I'd say not. It can only happen with broken
>MUAs, are there any? User could of course deliberately break it, but is
>there some gain in it?

It seems the qmail community disagrees with you here - I followed the
discussion on the qmail mailing list. I guess the Dovecot community will
be knocking on your door when important emails start disappearing
unprovoked, and Dovecot is the service that deleted them.

The bottom line is that you can not rely on other Maildir clients
following your server's conventions. They will only oblige to the Maildir
standard, and using Dovecot, emails will disappear. But there's no sweat - 
it's a simple fix.

>> The only way to avoid this is:
>> 1) use link and unlink, not rename
>..and if it crashes (or loses NFS link) between those calls, you'll
>suddenly see two mails. I prefer atomic operations.

Well, then Dovecot has to have a way to communicate this to the sysadmin
as an error condition. It's indefinitely better than deleting innocent
(blog) users' emails.

>> Anyway, Maildir has a strict consistency criteria which says that all
>> messages that are linked from tmp/ into new/ _must_ use time(NULL) plus a
>> number that is guaranteed not to lapse within one second, and which does
>> not collide with other messages in new/. new/ is the single entry point
>> into cur/, and messages in new/ can not be "older" than the messages in
>> cur/.
>It only has requirement that the file name is unique. There's other ways
>to do that than time(NULL). http://cr.yp.to/proto/maildir.html lists
>some.

"A unique name has three pieces, separated by dots. On the left is the 
result of time() or the second counter from gettimeofday(). On the right 
is the result of gethostname(). (To deal with invalid host names, replace 
/ with \057 and : with \072.) In the middle is a delivery identifier, 
discussed below."

Quite clear.

>> flags. It follows that when moving a message from new/ to cur/, it is
>> required that the server only picks messages that are older than one
>> second.
>I'm not sure what you mean by this.. What's special in files that were
>just created? What changes after it's older than one second? Or are you
>changing the base name when moving it to cur/? (What makes you sure that
>the new base name still doesn't exist?)

1) I create a message in tmp and link this to new, then unlink the
original.

2) I create a new message in tmp. I'm about to link this message to new/, 
when suddenly

3) You move my original message from new/ to cur/.

4) I now complete my link from tmp/ to new/.

5) You move my new message from new/ to cur.

Boom. My original message is gone. What's the point of only moving
messages from new/ older than one second?

All messages delivered to new/ in the same second will get a new
middle-part name, sometimes with a _0, _1 ... _n next to the pid. When you
move the messages from new/ to cur/ after one second, all new messages
delivered to new/ will have a different time part. No collisions. But if 
you move the messages _before_ one second has lapsed, then a message in 
cur/ may be overwritten when you either

  1) move the message from new/ to cur/
  2) change the flags of two messages whose base is equal.

Using link avoids the loss of emails, and allows the sysadmin to clean up 
the server's buggy behavior. rename means bye-bye.

>I don't think you should rely on checking timestamps in any case. What
>if the maildir is accessed via NFS and some other computer with
>different time created the file?

Maildir relies on timestamps - like it or not.

>But about the flags, looks like maildir spec says they could be used
>only in cur/ directory, so I guess my code is broken because it allows
>setting them already in new/.. Well, I'll fix it anyway later by moving
>mails directly from tmp/ into cur/ when implementing UIDPLUS extension.

You will lose emails if you move messages from tmp to cur directly. Simply 
because you have no idea wether or not messages in cur have the same base 
as your message in tmp.

Andy

-- 
Andreas Aardal Hanssen
http://www.andreas.hanssen.name/gpg