ZLIB/ZSTD Compressed MBOX in Maildir++ layout

[Pedro Ribeiro]

Hello again and thanks for the quick answer!

Our base (inbox) namespace is based on MDBOX with SIS

SIS is fantastic optimizing the storage required to the active (daily 
accessed/moved) email.

But for archival and easy recover of some specific user mailbox/folder 
it isn't practical to rely on the shared attachment storage of SIS, also 
I think is more universal format in case some legal obligation require 
us to give the e-mail of some user to the authorities, including all the 
attachments.

I would opt for some tradeoff solution if I find a way to enable the 
altstorage or additional namespace (also with MDBOX) but without SIS so 
that I can store the old mails with the attachments included and 
compressed. (zstd does a great job!)

Is there any way to do it?

It seems to me SIS can't be enabled/disabled for each namespace.

zlib_save would benefit if it could be different for each namespace (eg. 
lz4 or none for the inbox namespace and zstd for the archival)

Regards.

On 6/28/21 13:42, Josef 'Jeff' Sipek wrote:
> On Mon, Jun 28, 2021 at 10:00:59 +0100, Pedro Ribeiro wrote:
>> Hello!
>>
>> Thank you for the suggestion.
>>
>> That solved the problem of using layout FS with MBOX storage format.
>>
>> After copying all the mail of the test user to the namespace in this
>> format I've compressed all the storage files with ZSTD and that changed
>> the name with the additional suffix .zst
>>
>> Dovecot seems to need this to be .zstd so I've fixed the names.
>>
>> But ... it doesn't work.
>>
>>> Jun 28 08:56:39 ribble dovecot:
>>> imap(pribeiro at net.ipl.pt)<82609><vi0h1M7FGMMKZP4/>: Error: Mailbox
>>> @HISTZLIB/IPLNet:
>>> stat(/mnt/mail-hist/historic-fs/net.ipl.pt/ro/pribeiro/mailboxes/IPLNet/mBoX-MeSsAgEs)
>>> failed: No such file or directory
>> In that directory the filename is:
>>
>>> -rw------- 1 vmail vmail 853M Jun 25 17:25 mBoX-MeSsAgEs.zstd
>> What am I doing wrong?
> Hm.  It is possible that it just doesn't work.  mbox doesn't get a lot of
> attention these days - its support is more of a migration tool.
>
> Out of curiosity, why do you use mbox?  In your earlier email you said that
> you want a self contained format.  Why?  What's the goal?
>
> Jeff.
>
>> regards!
>>
>> On 24/06/2021 14:53, Josef 'Jeff' Sipek wrote:
>>> On Thu, Jun 24, 2021 at 10:35:24 +0100, Pedro Ribeiro wrote:
>>>> Hello everyone!
>>>>
>>>> Our archive namespace is kept in MBOX format with Maildir++ layout due to:
>>>>
>>>>     * Need to have the email from all users/folders self contained (our
>>>>       base namespace uses MDBOX with single instance storage of attachments)
>>>>     * MBOX/FS layout isn't supported because some folders have messages
>>>>       and subfolders.
>>> You should be able to use the fs layout if you specify a dirname:
>>> https://doc.dovecot.org/configuration_manual/mail_location/mbox/mboxchildfolders/#messages-in-named-file
>>>
>>> Then, the compression suffix being added shouldn't matter.
>>>
>>> Jeff.
>> -- 
>> Best regards,
>>
>> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
>> Pedro Ribeiro
>> Politécnico de Lisboa, Serviços da Presidência
>> Departamento de Sistemas de Informação e Comunicações
>> Phone:   +351 210 464 700 (general) / VoIP: 80100
>> Helpdesk:helpdesk at net.ipl.pt  /https://www.net.ipl.pt
>> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
>>
>>
>
-- 
Best regards,

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Pedro Ribeiro
Politécnico de Lisboa, Serviços da Presidência
Departamento de Sistemas de Informação e Comunicações
Phone:   +351 210 464 700 (general) / PBX Ext.: 80100
Helpdesk: helpdesk at net.ipl.pt / https://www.net.ipl.pt
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3605 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://dovecot.org/pipermail/dovecot/attachments/20210628/427e9fa7/attachment.p7s>