[Dovecot] Indexing Performance Question (was tpop3d vs dovecot)
nm_list at visp.net
Thu Feb 1 19:31:49 UTC 2007
At 10:50 PM 1/31/2007, FiL wrote:
>10 minutes to read 748Mb? Kinda slow, isn't it ?
>For me reading 2Gb mailbox (not maildir) takes MUCH less time than
>that (around 20 sec).
>It should be within 40secs to read the whole maildir.
I whole-heartedly agree, that is slow; however, not necessarily any
problem with dovecot. More a filesystem issue. Because it's not 1
large synchronous file, the disk has to seek like crazy to read each
file. Here's some testing to show it's normal:
I looped through all the files and cat them to /dev/null. This is
independent of dovecot. This is reading the entire contents of every
file to /dev/null. I don't know why or if dovecot would have to do
that to build an index, should be able to get away with the first 100
lines max I'd think.
[root at test cur]# time for x in `ls` ; do cat $x > /dev/null ; done
Now if I copy the contents of cur/ to another directory (on an
otherwise idle machine), so it's all written synchronously on the
disk, the results change dramatically. (a reboot is performed to
clear all cache after the copy)
[root at test testspool]# time for x in `ls` ; do cat $x > /dev/null ; done
Finally, to simulate a mbox like you mention I tar all the files to 1
large file, reboot, and cat the new tarfile to /dev/null.
[root at test natetest]# time cat cur.tar >/dev/null
What's sad with these tests is it takes 2m15s to read all the data in
71570 individual files, and only 15s to do it in 1 file. Granted
there's a bit of overhead from the shell script looping, but not
*that* much! I wonder how these tests would go on a different file
system. I'm currently using reiserfs.
More information about the dovecot