Thank you very much for all the responses in this thread. Now I have more questions:
- I have "slow" I/O (about 3.5000-4.000 IOPS, measured via imapsync), if I enable zlib compression in my maildirs, that should lower the number the IOPS (less to read, less to write, less IOPS, more CPU). Dovecot 2.0 is better for zlib (lda support) than dovecot 1.2.X..
- I understand that indexes should go to the fastest storage I own. Somebody talked about storing them in a ramdisk and then backup them to disk on shutdown. I have several questions about that:
- In my setup I have 25.000+ users, almost 7.000.000 messages in my maildir. How much memory should I need in a ramdisk to hold that?
- What happens if something fails? I think that if I lose the indexes (ej: kernel crash) the next time I boot the system the ramdisk will be empty, so the indexes should be recreated. Am I right?
- If I buy a SSD system and export that little and fast storage via iSCSI, does zlib compression applies to indexes?
- Any additional filesystem info? I am using ext3 on RHEL 5.5, in RHEL 5.6 ext4 will be supported. Any performance hint/tuning (I already use noatime, 4k blocksize)?
Regards
Javiermail_location = maildir:~/Maildir:INDEX=MEMORY
The ":INDEX=MEMORY" disables writing the index files to disk, and as the name implies, I believe, simply keeps indexes in memory. I think maybe I shoudn't have called it INDEX=MEMORY, but rather more like INDEX=DISABLE.
"If you really want to, you can also disable the index files completely by appending :INDEX=MEMORY."
My read of that is that indexing isn't disabled completely, merely storing the indexes to disk is disables. The indexes are still built and maintained in memory.
Timo, is that correct? It's a per-connection in-memory index. Also there is no kind of caching of anything (dovecot.index.cache file, which is where most of Dovecot performance usually comes from).
I don't know if, or how much, storing them in RAM via :INDEX=MEMORY consumes, as compared to using a ramdisk. The memory consumption may be less or it may be more. Timo should be able to answer this, and give a recommendation as to whether this is even a sane thing to do. I think INDEX=MEMORY performance is going to suck. http://imapwiki.org/Benchmarking explains IMAP performance a bit more. By default Dovecot is the "Dynamically caching server", but with INDEX=MEMORY it becomes "Non-caching server".