It's the real question of where the file is stored. For me, using nfs,
using compression given me several times the performance of nfs, vs
not using compression, for html files.
I would imagine the same benifits with email.
I would say something very close to the same with mdbox, if it's going
over the network to a database server, compression will should clearly
speed things up.
Now if your storing it on the local machine, then yes, compression
will do nothing but slow things down, but can give you increased
storage space.
Quoting Timo Sirainen <tss@iki.fi>:
On 6.2.2010, at 3.23, Timo Sirainen wrote:
I was thinking that the compression would be delayed so that it
would be done only after mdbox already decided that it wouldn't
write any more data to it.Oh, and this is actually why I was thinking that maybe it could be a
good idea. If it's only done for older mails, they aren't accessed
that often. So maybe a hybrid solution would be a good idea for
mdbox users with alt storage:
- primary storage: SSD disks, mdbox file size = 100k, compress each
mail separately- alt storage: spinning disks, mdbox file size = 2 MB, compress the
entire fileMails would be moved to alt storage after n days, perhaps
dynamically depending on available SSD disk space.SSDs can read data pretty fast though, so it would be nice to look
at some benchmarks that read tons of emails concurrently compressed
vs. uncompressed. Is the bottleneck CPU or I/O? Hmm. A quick test
with my Intel SSD shows that it can read 243 MB/s from a single
large file, while zlib input is only 100 MB/s with Macbook's one CPU
core. Faster CPUs and more cores would make zlib faster though.