[Dovecot] Apple patches 1-5
Here are the first few simple patches from Apple, based on
dovecot-1.1.7. The comments with "APPLE" in them helped us merge in
your new releases; feel free to remove them. Please let me know if
you want subsequent patches in a different format, or if you have any
questions.
Patch #1. Some versions of Mac OS X have buggy CMSG_* macros.
Patch #2. Don't set "<username>=1" in the environment.
Patch #3. Some versions of Mac OS X return near-duplicate kevents.
Patch #4. Null renames are actually pretty slow on HFS (Apple's file
system), so avoid them.
Patch #5. Required by Apple's lawyers.
On Mon, Dec 15, 2008 at 12:39 PM, Mike Abbott michael.abbott@apple.com wrote:
Patch #5. Required by Apple's lawyers.
Patch 5 seems spurious to me, regardless of what Apple's lawyers want.
Three of the four patches are working around bugs in Apple's own OS. Most are a few lines.
Sending patches upstream are great, but in any project I've been a part of, these are not conditions under which someone gets to claim copyright, especially on something as elusive as "portions".
--Jeff
On Mon, 2008-12-15 at 11:39 -0600, Mike Abbott wrote:
Here are the first few simple patches from Apple, based on
dovecot-1.1.7. The comments with "APPLE" in them helped us merge in
your new releases; feel free to remove them. Please let me know if
you want subsequent patches in a different format, or if you have any
questions.Patch #1. Some versions of Mac OS X have buggy CMSG_* macros.
Is the nopen() check really necessary? I'd like to know at least what kind of a bug it works around for before applying it. The fstat()+inode comparing doesn't catch it?
Patch #2. Don't set "<username>=1" in the environment.
I put the fix to auth-client.c: http://hg.dovecot.org/dovecot-1.1/rev/3145be9f66ae
Patch #3. Some versions of Mac OS X return near-duplicate kevents.
This probably won't affect other BSDs, so I think the duplicate removal should be around #ifdef __APPLE__?
Patch #4. Null renames are actually pretty slow on HFS (Apple's file
system), so avoid them.
I've thought about adding a similar check before, but it removes an intentional side effect: It makes sure that the file exists. If it doesn't exist (flags had changed), its new name is looked up and the rename is tried again. Would it be faster to instead stat() to see if the file exists?
v1.2 suppresses unnecessary flag changes, so these dummy renames (or stat()s) should happen less often.
Patch #5. Required by Apple's lawyers.
That's fine by me.
On Tue, 2008-12-16 at 05:01 +0200, Timo Sirainen wrote:
Patch #3. Some versions of Mac OS X return near-duplicate kevents.
This probably won't affect other BSDs, so I think the duplicate removal should be around #ifdef __APPLE__?
Hmm. Actually I think when I was writing that code I noticed the same thing and tried to fix it with:
/* there can be multiple events for a single io.
call the callback only once if that happens. */
if (io->refcount == 2 && io->io.callback != NULL)
io->io.callback(io->io.context);
But later (although inside the same commit..) I probably added the assert to:
for (i = 0; i < ret; i++) {
io = (void *)events[i].udata;
i_assert(io->refcount == 1);
io->refcount++;
}
Which makes the refcount == 2 check pointless. I think it would work correctly simply if the assert was removed?
Hmm. Actually I think when I was writing that code I noticed the same thing and tried to fix it with:
/* there can be multiple events for a single io. call the callback only once if that happens. */ if (io->refcount == 2 && io->io.callback != NULL) io->io.callback(io->io.context);
When there are near-duplicates in the returned events -- that is, two
or more entries in the events array having the same udata but
different fflags -- the refcount==2 check actually prevents the
callback from being called at all (assuming Apple's patch to this area
has not been applied, and the assert is removed). The first loop
increments the refcount twice (or more), from 1 to 3 (or higher).
Then the second loop skips the callback entirely because the refcount
does not equal 2. The callback is not even called once.
I think it would work correctly simply if the assert was removed?
Removing the assert would cause the callback to be called twice (or
more). Probably not good. I know the O(n^2) algorithm to find and
remove duplicates is undesirable but in practice n is small.
On Dec 16, 2008, at 6:47 PM, Mike Abbott wrote:
Hmm. Actually I think when I was writing that code I noticed the same thing and tried to fix it with:
/* there can be multiple events for a single io. call the callback only once if that happens. */ if (io->refcount == 2 && io->io.callback != NULL) io->io.callback(io->io.context);
When there are near-duplicates in the returned events -- that is,
two or more entries in the events array having the same udata but
different fflags -- the refcount==2 check actually prevents the
callback from being called at all (assuming Apple's patch to this
area has not been applied, and the assert is removed). The first
loop increments the refcount twice (or more), from 1 to 3 (or
higher). Then the second loop skips the callback entirely because
the refcount does not equal 2. The callback is not even called once.
But it decreases the refcount every time. So if refcount starts at 3,
the second loop skips the first callback -> decreases refcount to 2 ->
calls callback -> decreases refcount to 1.
Patch #1. Some versions of Mac OS X have buggy CMSG_* macros.
Is the nopen() check really necessary?
It might detect buggy CMSG macros on non-Apple systems.
I'd like to know at least what kind of a bug it works around for
before applying it.
The 64-bit CMSG macros are broken in such a way that two descriptors
are sometimes passed (depending on the value in uninitialized memory,
often 0), not one. The nopen() check looks for that.
I will try to be more descriptive up front with the remaining
patches. This is why I started with some small ones. Thank you for
your cooperation.
Patch #2. Don't set "<username>=1" in the environment.
I put the fix to auth-client.c: http://hg.dovecot.org/dovecot-1.1/rev/3145be9f66ae
Two points:
- Yes, if you see a better way to achieve the same goal for any of
Apple's patches, please do adopt the better way. - However, in this case your change will prevent one of our future
patches from working properly. It will need the user name in the
extra_fields array in deliver.c. May I suggest that you take patch #2
as-is?
Patch #3. Some versions of Mac OS X return near-duplicate kevents.
This probably won't affect other BSDs, so I think the duplicate
removal should be around #ifdef __APPLE__?
I have no data to support that conclusion.
Patch #4. Null renames are actually pretty slow on HFS (Apple's file system), so avoid them.
I've thought about adding a similar check before, but it removes an intentional side effect: It makes sure that the file exists. If it doesn't exist (flags had changed), its new name is looked up and the rename is tried again. Would it be faster to instead stat() to see if the file exists?
I wondered about that. Sure, a stat() would be faster.
Patch #5. Required by Apple's lawyers.
That's fine by me.
Phew!
On Dec 16, 2008, at 6:26 PM, Mike Abbott wrote:
Patch #1. Some versions of Mac OS X have buggy CMSG_* macros.
Is the nopen() check really necessary?
It might detect buggy CMSG macros on non-Apple systems.
I'd like to know at least what kind of a bug it works around for
before applying it.The 64-bit CMSG macros are broken in such a way that two descriptors
are sometimes passed (depending on the value in uninitialized
memory, often 0), not one. The nopen() check looks for that.
Hmm. OK, I guessed it was probably something like that. There's
probably a better way to test it by seeing if two fds were received,
but since fd_read() doesn't support it I guess the nopen() check is
fine.
Patch #2. Don't set "<username>=1" in the environment.
I put the fix to auth-client.c: http://hg.dovecot.org/dovecot-1.1/rev/3145be9f66ae
Two points:
- Yes, if you see a better way to achieve the same goal for any of
Apple's patches, please do adopt the better way.- However, in this case your change will prevent one of our future
patches from working properly. It will need the user name in the
extra_fields array in deliver.c. May I suggest that you take patch
#2 as-is?
Hmm. Maybe. I'll see about it when you send the future patch. :)
Anyway in v1.2 the auth lookup API has changed already and the
username is received in a different parameter (and this bug was
already fixed there already).
Patch #4. Null renames are actually pretty slow on HFS (Apple's
file system), so avoid them.I've thought about adding a similar check before, but it removes an intentional side effect: It makes sure that the file exists. If it doesn't exist (flags had changed), its new name is looked up and the rename is tried again. Would it be faster to instead stat() to see if the file exists?
I wondered about that. Sure, a stat() would be faster.
OK, I'll add that.
Summary:
On Mon, 2008-12-15 at 11:39 -0600, Mike Abbott wrote:
Here are the first few simple patches from Apple, based on
dovecot-1.1.7. The comments with "APPLE" in them helped us merge in
your new releases; feel free to remove them. Please let me know if
you want subsequent patches in a different format, or if you have any
questions.Patch #1. Some versions of Mac OS X have buggy CMSG_* macros.
http://hg.dovecot.org/dovecot-1.1/rev/a217d9ae130b
Patch #2. Don't set "<username>=1" in the environment.
Waiting for your next patch to see if I should do it this way or some other way.
Patch #3. Some versions of Mac OS X return near-duplicate kevents.
http://hg.dovecot.org/dovecot-1.1/rev/d178293dde50
Patch #4. Null renames are actually pretty slow on HFS (Apple's file
system), so avoid them.
http://hg.dovecot.org/dovecot-1.1/rev/92921985e4f5
Patch #5. Required by Apple's lawyers.
Following up.
You checked in slightly different versions of patches 1, 3 and 4 and
released them with 1.1.8. We will test your solutions for these and
adopt them if they work. Thanks!
For all the changes you checked into 1.1 on our behalf, will they also
be included in 1.2?
participants (3)
-
Jeff Mitchell
-
Mike Abbott
-
Timo Sirainen