Thank you!

Here is the whole problem I see:

as I saw how good it is to have messages compressed I used https://github.com/George-NG/dovecot-maildir-compress script to compress whole mailboxes and thus save more space on disk. Looks like lock won't worked well so now I can see lines like that in Dovecot logs:

Error: Mailbox ... : Deleting corrupted cache record uid=61: UID 61: Broken virtual size in mailbox ... : read(compress(/data/vmail/...)): FETCH BODY[] got too little data: 38910 vs 70020"

I tried to recover these messages by uncompress files with your command, and I can say it was possible to uncompress some files while others (the one that were reffered by these errors lines) seems can't be recovered.

Seems like I have data lost in some messages and this is not something I got used with dovecot which I use since pre-1.0 version.

P.S. I also tried gzip-ing messages so how have to ask you for the command to un-gzip messages, as "doveadm fs get compress gzip:6:posix:prefix=$PWD/ filename" seems not working for me.

...

On 17/01/2024 18:08 EETachekalin@lazurit.com wrote:

Some time ago I enabled compression on incoming messages, so LMTP delivered messages were compressed. Good idea in terms of disk space, but now I cannot uncompress any of these messages even when I need to!

I was using this setting:

plugins { zlib_save = lz4 }

and now I can see this string at the beginning of each message file - "Dovecot-LZ4", so no other software can uncompress them for me.

Please advise, how can I uncompress message file in Maildir mailbox when I need to access its contents directly?

One simple way:

doveadm fs get compress lz4:6:posix:prefix=$PWD/ filename

Aki

Thank you! Here is the whole problem I see: as I saw how good it is to have messages compressed I used https://github.com/ George-NG/dovecot-maildir-compress script to compress whole mailboxes and thus save more space on disk. Looks like lock won't worked well so now I can see lines like that in Dovecot logs: Error: Mailbox ... : Deleting corrupted cache record uid=61: UID 61: Broken virtual size in mailbox ... : read(compress(/data/vmail/...)): FETCH BODY[] got too little data: 38910 vs 70020" I tried to recover these messages by uncompress files with your command, and I can say it was possible to uncompress some files while others (the one that were reffered by these errors lines) seems can't be recovered. Seems like I have data lost in some messages and this is not something I got used with dovecot which I use since pre-1.0 version. P.S. I also tried gzip-ing messages so how have to ask you for the command to un-gzip messages, as "doveadm fs get compress gzip:6:posix:prefix=$PWD/ filename" seems not working for me.

      On 17/01/2024 18:08 EET achekalin@lazurit.com wrote:


      Some time ago I enabled compression on incoming messages,
      so LMTP delivered messages were compressed. Good idea in
      terms of disk space, but now I cannot uncompress any of
      these messages even when I need to!

      I was using this setting:

      plugins {
        zlib_save = lz4
      }

      and now I can see this string at the beginning of each
      message file - "Dovecot-LZ4", so no other software can
      uncompress them for me.

      Please advise, how can I uncompress message file in Maildir
      mailbox when I need to access its contents directly?

 One simple way:

 doveadm fs get compress lz4:6:posix:prefix=$PWD/ filename

 Aki

Is it possible that those mails are missing the S= and/or W= size
flags in the filename? I remember that being a requirement (the S= part, not sure if W= was
required).

I have stopped using maildir many years ago, and my script made sure
S= was added to every email, where the script you linked to does not,
it assumes it already exists.

Quoting Alexander Chekalin achekalin@lazurit.com:

Thank you!

Here is the whole problem I see:

as I saw how good it is to have messages compressed I used
https://github.com/George-NG/dovecot-maildir-compress script to
compress whole mailboxes and thus save more space on disk. Looks
like lock won't worked well so now I can see lines like that in
Dovecot logs:

Error: Mailbox ... : Deleting corrupted cache record uid=61: UID 61:
Broken virtual size in mailbox ... :
read(compress(/data/vmail/...)): FETCH BODY[] got too little data:
38910 vs 70020"

I tried to recover these messages by uncompress files with your
command, and I can say it was possible to uncompress some files
while others (the one that were reffered by these errors lines)
seems can't be recovered.

Seems like I have data lost in some messages and this is not
something I got used with dovecot which I use since pre-1.0 version.

P.S. I also tried gzip-ing messages so how have to ask you for the
command to un-gzip messages, as "doveadm fs get compress
gzip:6:posix:prefix=$PWD/ filename" seems not working for me.

...

...

On 17/01/2024 18:08 EETachekalin@lazurit.com wrote:

Some time ago I enabled compression on incoming messages, so LMTP
delivered messages were compressed. Good idea in terms of disk
space, but now I cannot uncompress any of these messages even when
I need to!

I was using this setting:

plugins { zlib_save = lz4 }

and now I can see this string at the beginning of each message
file - "Dovecot-LZ4", so no other software can uncompress them for
me.

Please advise, how can I uncompress message file in Maildir
mailbox when I need to access its contents directly?

One simple way:

doveadm fs get compress lz4:6:posix:prefix=$PWD/ filename

Aki