[Dovecot] testing needed: log file concurrency

[Troy Engel]

OS: CentOS 4.5 (Final) (RHEL4 clone)

$ /usr/bin/time -f "total time: %E\ni/o waits: %w\n" ./concurrency
writing, page size = 4096
Command terminated by signal 2
total time: 10:41.53
i/o waits: 312177

$ /usr/bin/time -f "total time: %E\ni/o waits: %w\n" ./concurrency 1
reading, page size = 4096
page size cut
Command terminated by signal 2
total time: 10:39.67
i/o waits: 314930


Machine:

$ uname -srvmpoi
Linux 2.6.9-42.0.10.ELsmp #1 SMP Tue Feb 27 10:11:19 EST 2007 i686 i686 
i386 GNU/Linux

$ cat /proc/cpuinfo | egrep "(processor|model name)"
processor       : 0
model name      : Intel(R) Xeon(R) CPU            5120  @ 1.86GHz
processor       : 1
model name      : Intel(R) Xeon(R) CPU            5120  @ 1.86GHz
processor       : 2
model name      : Intel(R) Xeon(R) CPU            5120  @ 1.86GHz
processor       : 3
model name      : Intel(R) Xeon(R) CPU            5120  @ 1.86GHz

Timo Sirainen wrote:
> http://dovecot.org/tmp/concurrency.c
> 
> I'd want to know what results this program gives with different systems.
> Please test and reply (but don't bother if someone already replied with
> the same OS+result). I expect it to print:
> 
>  - SMP kernels: "page size cut" once in a while
>  - UP (uniprocessor) kernels: Nothing
>  - The most important thing is that it never prints "broken data"
> 
> It might take a while for it to print anything. With my computer it
> takes anything from a few seconds to a minute or so. See the file itself
> for compiling/running instructions.
> 
> So far I've tested only with Linux 2.6.21 x86-64/SMP and a slow
> Solaris/Sparc/UP.
> 
> If you're interested in knowing what this is about:
> 
> Dovecot writes to dovecot.index.log files by first writing the
> transaction with its size being 0. After that it writes the 4 size bytes
> again (using a bit special format with all bytes ORed with 0x80).
> 
> I expected that when another process is read()ing the file and it
> notices the size being valid (all bytes having 0x80) that the whole
> transaction could always be read. But looks like if the size happens to
> be just before a memory page boundary, it's possible that the updated
> size is read, but the rest of the transaction isn't.
> 

-- 
Troy Engel | Systems Engineer
Fluid, Inc | http://www.fluid.com