[Dovecot] (Single instance) attachment storage
bill+dovecot at blunn.org
Mon Jul 19 22:42:29 EEST 2010
Timo Sirainen wrote:
> I was thinking it would be nice to be able to compress attachments after they've already been delivered. Like maybe keep the attachments decoded for a few weeks and then compress them. Similar to how some people do it with Maildir. This can't work without a small header, otherwise you can't know if the attachment was originally compressed or not.
What about doing what people do with files and codify the compression
status in the filename?
Assume the attachment file has a base name of "foo".
If you want to compress it, create a file "foo.gz" (or "foo.bz2" or
"foo.xz"), and remove the original file "foo".
This assumes that if an open file is unlinked it will remain in
existence until closed.
Code which wishes to read attachment files should first try to open the
file using the base name. If the base name does not exist then it can
try suffixes until it finds a file which opens. The suffix which worked
then determines the decompression algorithm to be used.
The cost of an extra directory search (on messages which are understood
to be historical in any case) should be dwarfed by the cost of
decompressing the data.
This also solves the problem of how you compress a file in place. By
creating the compressed version alongside, you never upset any readers.
This also helps out sysadmins because they would then be able to examine
compressed attachment files straight off (zcat, bzcat, xzcat, et. al.)
without having to think about any Dovecot-specific way of how the files
might be laid out.
More information about the dovecot