[Dovecot] Indexing Performance Question (was tpop3d vs dovecot)

[Nate]

At 09:55 AM 2/2/2007, Kenny Dail wrote:
> > What's sad with these tests is it takes 2m15s to read all the data in
> > 71570 individual files, and only 15s to do it in 1 file.  Granted
> > there's a bit of overhead from the shell script looping, but not
> > *that* much!  I wonder how these tests would go on a different file
> > system.  I'm currently using reiserfs.
>
>actually the loop adds a lot of overhead:
>These commands were run right in a row so there is some speedup due to
>cacheing, So I repeted test a few times.
>Admittedly, this is a small maildir.
>
>bash-2.05b# ls | wc
>     1009    1009   75500
>
>bash-2.05b# time for x in `ls` ; do cat $x > /dev/null ; done
>
>real    0m3.670s
>user    0m0.347s
>sys     0m1.508s
>bash-2.05b# time cat * > /dev/null
>
>real    0m0.497s
>user    0m0.015s
>sys     0m0.071s
>bash-2.05b# time for x in `ls` ; do cat $x > /dev/null ; done
>
>real    0m2.237s
>user    0m0.356s
>sys     0m1.435s
>bash-2.05b# time cat * > /dev/null
>
>real    0m0.504s
>user    0m0.001s
>sys     0m0.086s
>bash-2.05b# time for x in `ls` ; do cat $x > /dev/null ; done
>
>real    0m2.415s
>user    0m0.356s
>sys     0m1.550s

wow, that is a ton of extra overhead.  I'm doing some testing on ext3 
vs reiserfs with different options and it seems that a lot of it has 
to do with the filesystem and how well it handles tons of files in a directory.

What I've gotten so far using the same shell loop with 71569 small 
files in the dir.  Between each test the mount was unmounted and 
remounted to clear cache.
ext3 defaults:  12m3s  (~400k/sec of writes going on)
ext3 noatime:  11m31s  (0kb/sec writes)
ext3 noatime,nodiratime:  11m43s  (0k/sec of writes)

reiserfs defaults:  13m35s  (~500k/sec of writes)
reiserfs noatime:   8m47s (0k/sec of writes)
reiserfs noatime,nodiratime:  9m51s  (0k/sec of writes)
reiserfs noatime,nodiratime,notail: 9m59s  (0k/sec writes)

The clear winner seems to be reiserfs with only the noatime option 
set.  I don't know why nodiratime actually slows it down (i ran these 
tests multiple times to confirm it wasn't flukish results), but 
perhaps it's an optimization issue with reiserfs.

- Nate