[Dovecot] Question about "slow" storage but fast cpus, plenty of ram and dovecot

[Stan Hoeppner]

Javier de Miguel Rodríguez put forth on 12/12/2010 1:26 PM:
> 
> Thank you very much for all the responses in this thread. Now I have
> more questions:
> 
>     - I have "slow" I/O (about 3.5000-4.000 IOPS, measured via
> imapsync), if I enable zlib compression in my maildirs, that should
> lower the number the IOPS (less to read, less to write, less IOPS, more
> CPU). Dovecot 2.0 is better for zlib (lda support) than dovecot 1.2.X..
> 
>     - I understand that indexes should go to the fastest storage I own.
> Somebody talked about storing them in a ramdisk and then backup them to
> disk on shutdown. I have several questions about that:

For that many users I'm guessing you can't physically stuff enough RAM
into the machines in your ESX cluster to use a ramdisk for the index
files, and if you could, you probably couldn't, or wouldn't want to,
afford the DIMMs required to meet the need.

>                 - In my setup I have 25.000+ users, almost 7.000.000
> messages in my maildir. How much memory                 should I need in
> a ramdisk to hold that?
> 
>                  - What happens if something fails? I think that if I
> lose the indexes (ej: kernel crash) the next time I                 boot
> the system the ramdisk will be empty, so the indexes should be
> recreated. Am I right?

Given the size of your mail user base, I'd probably avoid the ramdisk
option, and go with a couple of striped (RAID 0) 100+ GB SSDs connected
on the iSCSI SAN.  This is an ESX cluster of more than one machine
correct?  You never confirmed this, but it seems a logical assumption
based on what you've stated.  If it's a single machine you should
obviously go with locally attached SATA II SSDs as it's far cheaper with
much greater real bandwidth by a factor of 100:1 vs iSCSI connection.

>                 - If I buy a SSD system and export that little and fast
> storage via iSCSI, does zlib compression                         applies
> to indexes?

Timo will have to answer this regarding zlib on indexes.

>     - Any additional filesystem info? I am using ext3 on RHEL 5.5, in
> RHEL 5.6 ext4 will be supported. Any performance hint/tuning (I already
> use noatime, 4k blocksize)?

I'm shocked you're running 25K mailboxen with 7 million messages on
maildir atop EXT3!  On your  fast iSCSI SAN array, I assume with at
least 14 spindles in the RAID group LUN where the mail is stored, you
should be using XFS.

Formatted with the correct parameters, and mounted with the correct
options, XFS will give you _at minimum_ a factor of 2 performance gain
over EXT3 with 128 concurrent users.  As you add more concurrent users,
this ratio will grow even greater in XFS' favor.

XFS was designed specifically, and has been optimized since 1994, for
large parallel workloads.  EXT3 has traditionally been optimized for use
as a single user desktop filesystem.  Its performance with highly
parallel workloads pales in comparison to XFS:

http://btrfs.boxacle.net/repository/raid/2.6.35-rc5/2.6.35-rc5/2.6.35-rc5_Mail_server_simulation._num_threads=128.html


If you do add the two ~100 GB SSDs and stripe them via the RAID hardware
or via mdadm or LVM, format the resulting striped device with XFS and
give it 16 allocation groups.  If using kernel 2.6.36 or grater, mount
the resulting filesystem with the delaylog option.  You will be doing
yourself a huge favor, and will be amazed at the index performance.

-- 
Stan