[Dovecot] Please advise on very fast search

[Alexander Chekalin]

Hello, Stan,

 > This is why I recommended mbox in the first place.  If your only writes
 > to these mailbox files are appends of new messages, mbox is the best
 > format by far.  It's faster at appending than any other format, and it's
 > faster for searching than any other.

I now seriously consider to use mdbox due to its nice self-regulation. 
After all it I believe mdbox should do file compression on its own, no 
cron scripts required.

 > It's an archive.  You're not going to use maildir so you don't need
 > random IOPS performance.  Thus RAID5/6 are a much better fit for an
 > archive as you get better read performance, with more than adequate
 > write performance, and you use less disks.  And as this is an archive,
 > you don't need real time automatic/transparent compression.  Thus I
 > recommend something like:
 >
 > 1.  Debian 6 w/linux-image-2.6.39-bpo.2-amd64 or a custom rolled
 >      2.6.39 or later kernel
 > 2.  hardware RAID5 w/large (2TB) SATA disks, 512B native sectors
 >      e.g. MegaRAID SAS 9261-8i, 4 Seagate Constellation ES ST2000NM0011
 >      Specify a strip size of 256KB for the array
 >      Perma set /sys/block/sdX/read_ahead_kb to 512 so you're reading
 >      ahead 1024 sectors at a time instead of the default of 256.  This
 >      will speed up your searches quite a bit.
 > 3.  XFS filesystem on the RAID device, created with mkfs.xfs defaults
 > 4.  mbox w/zlib plugin.  Compress daily files each night with a script
 > 5.  You don't need LVM with a good RAID card (or with mdraid).  This
 >      controller can expand the RAID5 up to 8 drives (up to 32 drives max
 >      using SAS expanders)

We are considering to get HP DL180G6 server for 8 or 14 drives bays 
(base model price is somewhat equal, but additional drives adds up cost) 
with HP Smart Array P410 RAID controller (some servers are equipped with 
this controller by default) with 256 Mb battery-backed cache, but I'll 
check your suggestions!

What memory size should I plan in the server? You're talking about AMD64 
OS image, and 64-bit software are like to consume more memory that 
32-bit, so looks like your talking about pretty huge RAM, and I don't 
believe it's necessary, or maybe I'm wrong?

Problem is I have no experience with XFS and not sure I can tune it in 
the best way, so I'll go with mkfs.xfs defaults, I think.

Hope we'll see Dovecot 2.1.x stable soon, as I'd like to use fts plugins 
and 2.1 handle that much better, but I don't like the idea of use 
unstable in production.

Thank you for taking your time on my case,
   yours,
     Alexander