Coming back to this now almost a year later.
Thanks for your response Patrick.
On 1/12/22 13:25, Patrick Domack wrote:
It's because it doesn't move email by email to alt storage, but the whole file. So the emails move as is, be them compressed or uncompressed.
I don't believe the above is correct, going on https://doc.dovecot.org/admin_manual/mailbox_formats/dbox/#dbox-mbox-format: "The granularity at which data is moved to alternate storage is individual messages. This is true even for |mdbox| when multiple messages are stored in a single |m.*| storage file. If individual messages from an |m.*| storage file need to be moved to alternate storage, the message data is written out to a different |m.*| storage file (either new or existing) in the alternate storage area and the “map index” updated accordingly."
Also, it says "only the actual message data (stored in files |u.*| under |sdbox| and |m.*| under |mdbox|) is moved" and (presumably) not the dovecot.map.index, which maybe suggests that when a message is moved it can't be compressed/uncompressed, since the dovecot.map.index may not change, in the sdbox case at least??
But the fact that the map index _does_ get updated when altmove happens for mdbox according to the paragraph above did give me hope that it might be possible. And I saw an email on this list from someone who claimed/implied to be compressing using:
doveadm -o zlib_save=gz -o zlib_save_level=9 altmove
However haven't been able to get it to work, it seems on my testing that indeed messages can only be compressed on initial delivery and compressing them with altmove isn't (any longer) possible.
I'm still kind of curious whether it's a limitation in the mdbox format and the way indexes are handled that precludes compress on altmove, or whether it's a limitation of the altmove command, or whether altmove maybe can do it but I just haven't got the options right yet. If I had more time I'd dig in to the code :)
I think you could do what you want, but passing the correct options, but I don't think it would be easy to do.
I would just store them compressed all the time, saves disk space, io bandwidth, and if it goes over the network, actually will give a huge performance improvement.
I have moved all mine to zstd currently, from gzip I had been using since it was possible.
I will likely do the same.
Thanks again, Alex
Quoting Alex King alex@king.net.nz:
Using 2.3.13 (89f716dc2) (debian package), with mdbox storage.
I considering adding the zlib plugin. I understand that reading mail can cope with either compressed or uncompressed mailboxes.
I'm wondering if I can leave the main storage uncompressed, while having the alternate storage compressed. There does not seem to be a way to configure this, from https://doc.dovecot.org/settings/plugin/zlib-plugin/ it seems compression (on saving emails) is an all-or-nothing option?
Can I leave zlib_save unset in my main configuration, and have doveadm use some alternate configuration with zlib_save set? It doesn't look hopeful because I don't see the ability to use a different config at https://wiki.dovecot.org/Tools/Doveadm/Altmove.
Perhaps I would need multiple copies of the daemon running?
Or is it better just to compress everything? I plan to store older mails (perhaps 3months+) in the alternative storage, and was thinking it would be nice to have those compressed, while I don't need to compress the main store.
Any feedback welcome.
Thanks, Alex