Timo Sirainen wrote:
I was thinking it would be nice to be able to compress attachments after they've already been delivered. Like maybe keep the attachments decoded for a few weeks and then compress them. Similar to how some people do it with Maildir. This can't work without a small header, otherwise you can't know if the attachment was originally compressed or not.
What about doing what people do with files and codify the compression status in the filename?
Assume the attachment file has a base name of "foo".
If you want to compress it, create a file "foo.gz" (or "foo.bz2" or "foo.xz"), and remove the original file "foo".
This assumes that if an open file is unlinked it will remain in existence until closed.
Code which wishes to read attachment files should first try to open the file using the base name. If the base name does not exist then it can try suffixes until it finds a file which opens. The suffix which worked then determines the decompression algorithm to be used.
The cost of an extra directory search (on messages which are understood to be historical in any case) should be dwarfed by the cost of decompressing the data.
This also solves the problem of how you compress a file in place. By creating the compressed version alongside, you never upset any readers.
This also helps out sysadmins because they would then be able to examine compressed attachment files straight off (zcat, bzcat, xzcat, et. al.) without having to think about any Dovecot-specific way of how the files might be laid out.
Bill