At 10:50 PM 1/31/2007, FiL wrote:
10 minutes to read 748Mb? Kinda slow, isn't it ? For me reading 2Gb mailbox (not maildir) takes MUCH less time than that (around 20 sec). It should be within 40secs to read the whole maildir.
I whole-heartedly agree, that is slow; however, not necessarily any problem with dovecot. More a filesystem issue. Because it's not 1 large synchronous file, the disk has to seek like crazy to read each file. Here's some testing to show it's normal:
I looped through all the files and cat them to /dev/null. This is independent of dovecot. This is reading the entire contents of every file to /dev/null. I don't know why or if dovecot would have to do that to build an index, should be able to get away with the first 100 lines max I'd think.
[root@test cur]# time for x in ls ; do cat $x > /dev/null ; done
real 15m28.568s
user 0m20.993s
sys 1m27.609s
Now if I copy the contents of cur/ to another directory (on an
otherwise idle machine), so it's all written synchronously on the
disk, the results change dramatically. (a reboot is performed to
clear all cache after the copy)
[root@test testspool]# time for x in ls ; do cat $x > /dev/null ; done
real 2m15.945s
user 0m20.493s
sys 1m10.796s
Finally, to simulate a mbox like you mention I tar all the files to 1 large file, reboot, and cat the new tarfile to /dev/null. [root@test natetest]# time cat cur.tar >/dev/null real 0m15.162s user 0m0.044s sys 0m1.024s
What's sad with these tests is it takes 2m15s to read all the data in 71570 individual files, and only 15s to do it in 1 file. Granted there's a bit of overhead from the shell script looping, but not *that* much! I wonder how these tests would go on a different file system. I'm currently using reiserfs.
- Nate