Timo Sirainen wrote:
On Mon, 2007-04-16 at 09:37 -0400, Justin McAleer wrote:
While doing some testing with converting accounts while simulating incoming mail load (no other pop/imap processes going but 4 processes converting users), we found that we were maxing out the local disk in the server with the index activity. To find out that it was the index activity, I mounted a tmpfs for dovecot to keep indexes on, and the system load dropped from 70 to 3 :)
How exactly were you converting the users? I guess if the system is building new index files for tons of users that could take a while. fsync_disable=yes could help a lot with reducing the disk writes, and maybe also mmap_disable=yes.
Ok, I'll give more detail about the setup. I plan to use the convert plugin to migrate from CommuniGate to dovecot for our ~160,000 accounts. We will probably migrate one domain at a time, letting users basically migrate their mail at first login. But, we will also go through all the domain's accounts logging in to pop3 just to ensure everything gets moved.
So, for testing, I've copied a few of our domains' mail spools over to the test server and started a script that forks and goes through all their accounts and simply logs in to pop3 and waits for a message list to come back. We have the maildirs being stored on an NFS mount though, and indexes on localdisk, so the only localdisk activity was logging and indexes. I'll try setting both of those options and see how much difference it makes, although I'm not sure that losing mail upon server reset is acceptible for us (disabling fsync).
For Dovecot v2.0 I'm planning on reducing at least dovecot.index.log file sizes. I think currently it uses way too much space when building the initial indexes.
Anyway, Timo and others, do you have any thoughts about keeping indexes on a tmpfs partition? I realize the obvious issues with this: running out of space, and the fact that the partition is lost on reboot.
I think memory would be better used for keeping mailbox data cached that's actually useful at the time.
Perhaps, but one way or another, we apparently cannot use a simple localdisk for indexes. So, the options are either memory or some sort of raid setup. Since we use maildirs, isn't the disk cache aspect less of a concern?
Also if you've POP3 users who keep messages in the server and dovecot.index.cache file is lost, all the messages are read to calculate the messages' virtual sizes when the user logs in the next time. That could be expensive.