[Dovecot] Question about "slow" storage but fast cpus, plenty of ram and dovecot

[Javier de Miguel Rodríguez]

Thank you very much for all the responses in this thread. Now I have 
more questions:

     - I have "slow" I/O (about 3.5000-4.000 IOPS, measured via 
imapsync), if I enable zlib compression in my maildirs, that should 
lower the number the IOPS (less to read, less to write, less IOPS, more 
CPU). Dovecot 2.0 is better for zlib (lda support) than dovecot 1.2.X..

     - I understand that indexes should go to the fastest storage I own. 
Somebody talked about storing them in a ramdisk and then backup them to 
disk on shutdown. I have several questions about that:

                 - In my setup I have 25.000+ users, almost 7.000.000 
messages in my maildir. How much memory                 should I need in 
a ramdisk to hold that?

                  - What happens if something fails? I think that if I 
lose the indexes (ej: kernel crash) the next time I                 boot 
the system the ramdisk will be empty, so the indexes should be 
recreated. Am I right?

                 - If I buy a SSD system and export that little and fast 
storage via iSCSI, does zlib compression                         applies 
to indexes?

     - Any additional filesystem info? I am using ext3 on RHEL 5.5, in 
RHEL 5.6 ext4 will be supported. Any performance hint/tuning (I already 
use noatime, 4k blocksize)?


     Regards

     Javier

>> mail_location = maildir:~/Maildir:INDEX=MEMORY
>>
>> The ":INDEX=MEMORY" disables writing the index files to disk, and as the
>> name implies, I believe, simply keeps indexes in memory.
> I think maybe I shoudn't have called it INDEX=MEMORY, but rather more like INDEX=DISABLE.
>
>> "If you really want to, you can also disable the index files completely
>> by appending :INDEX=MEMORY."
>>
>> My read of that is that indexing isn't disabled completely, merely
>> storing the indexes to disk is disables.  The indexes are still built
>> and maintained in memory.
>>
>> Timo, is that correct?
> It's a per-connection in-memory index. Also there is no kind of caching of anything (dovecot.index.cache file, which is where most of Dovecot performance usually comes from).
>
>> I don't know if, or how much, storing them in RAM via :INDEX=MEMORY
>> consumes, as compared to using a ramdisk.  The memory consumption may be
>> less or it may be more.  Timo should be able to answer this, and give a
>> recommendation as to whether this is even a sane thing to do.
> I think INDEX=MEMORY performance is going to suck. http://imapwiki.org/Benchmarking explains IMAP performance a bit more. By default Dovecot is the "Dynamically caching server", but with INDEX=MEMORY it becomes "Non-caching server".