Uncompress compressed messages in Maildir mailbox?
Some time ago I enabled compression on incoming messages, so LMTP delivered messages were compressed. Good idea in terms of disk space, but now I cannot uncompress any of these messages even when I need to!
I was using this setting:
plugins { zlib_save = lz4 }
and now I can see this string at the beginning of each message file - "Dovecot-LZ4", so no other software can uncompress them for me.
Please advise, how can I uncompress message file in Maildir mailbox when I need to access its contents directly?
Not really recommended editing the mailstorage directly once you start
using dovecot custom stuff.
If you need to fetch a specific email, I use: doveadm fetch -u user@example.com text mailbox-guid $guid uid $uid
Quoting achekalin@lazurit.com:
Some time ago I enabled compression on incoming messages, so LMTP
delivered messages were compressed. Good idea in terms of disk
space, but now I cannot uncompress any of these messages even when I
need to!I was using this setting:
plugins { zlib_save = lz4 }
and now I can see this string at the beginning of each message file
- "Dovecot-LZ4", so no other software can uncompress them for me.
Please advise, how can I uncompress message file in Maildir mailbox
when I need to access its contents directly?
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
On 17/01/2024 18:08 EET achekalin@lazurit.com wrote:
Some time ago I enabled compression on incoming messages, so LMTP delivered messages were compressed. Good idea in terms of disk space, but now I cannot uncompress any of these messages even when I need to!
I was using this setting:
plugins { zlib_save = lz4 }
and now I can see this string at the beginning of each message file - "Dovecot-LZ4", so no other software can uncompress them for me.
Please advise, how can I uncompress message file in Maildir mailbox when I need to access its contents directly?
One simple way:
doveadm fs get compress lz4:6:posix:prefix=$PWD/ filename
Aki
Thank you!
Here is the whole problem I see:
as I saw how good it is to have messages compressed I used https://github.com/George-NG/dovecot-maildir-compress script to compress whole mailboxes and thus save more space on disk. Looks like lock won't worked well so now I can see lines like that in Dovecot logs:
Error: Mailbox ... : Deleting corrupted cache record uid=61: UID 61: Broken virtual size in mailbox ... : read(compress(/data/vmail/...)): FETCH BODY[] got too little data: 38910 vs 70020"
I tried to recover these messages by uncompress files with your command, and I can say it was possible to uncompress some files while others (the one that were reffered by these errors lines) seems can't be recovered.
Seems like I have data lost in some messages and this is not something I got used with dovecot which I use since pre-1.0 version.
P.S. I also tried gzip-ing messages so how have to ask you for the command to un-gzip messages, as "doveadm fs get compress gzip:6:posix:prefix=$PWD/ filename" seems not working for me.
On 17/01/2024 18:08 EETachekalin@lazurit.com wrote:
Some time ago I enabled compression on incoming messages, so LMTP delivered messages were compressed. Good idea in terms of disk space, but now I cannot uncompress any of these messages even when I need to!
I was using this setting:
plugins { zlib_save = lz4 }
and now I can see this string at the beginning of each message file - "Dovecot-LZ4", so no other software can uncompress them for me.
Please advise, how can I uncompress message file in Maildir mailbox when I need to access its contents directly?
One simple way:
doveadm fs get compress lz4:6:posix:prefix=$PWD/ filename
Aki
Thank you! Here is the whole problem I see: as I saw how good it is to have messages compressed I used https://github.com/ George-NG/dovecot-maildir-compress script to compress whole mailboxes and thus save more space on disk. Looks like lock won't worked well so now I can see lines like that in Dovecot logs: Error: Mailbox ... : Deleting corrupted cache record uid=61: UID 61: Broken virtual size in mailbox ... : read(compress(/data/vmail/...)): FETCH BODY[] got too little data: 38910 vs 70020" I tried to recover these messages by uncompress files with your command, and I can say it was possible to uncompress some files while others (the one that were reffered by these errors lines) seems can't be recovered. Seems like I have data lost in some messages and this is not something I got used with dovecot which I use since pre-1.0 version. P.S. I also tried gzip-ing messages so how have to ask you for the command to un-gzip messages, as "doveadm fs get compress gzip:6:posix:prefix=$PWD/ filename" seems not working for me.
On 17/01/2024 18:08 EET achekalin@lazurit.com wrote:
Some time ago I enabled compression on incoming messages,
so LMTP delivered messages were compressed. Good idea in
terms of disk space, but now I cannot uncompress any of
these messages even when I need to!
I was using this setting:
plugins {
zlib_save = lz4
}
and now I can see this string at the beginning of each
message file - "Dovecot-LZ4", so no other software can
uncompress them for me.
Please advise, how can I uncompress message file in Maildir
mailbox when I need to access its contents directly?
One simple way:
doveadm fs get compress lz4:6:posix:prefix=$PWD/ filename
Aki
Did you try gunzip?
Aki
On 17/01/2024 20:32 EET Alexander Chekalin achekalin@lazurit.com wrote:
Thank you!
Here is the whole problem I see:
as I saw how good it is to have messages compressed I used https://github.com/George-NG/dovecot-maildir-compress script to compress whole mailboxes and thus save more space on disk. Looks like lock won't worked well so now I can see lines like that in Dovecot logs:
Error: Mailbox ... : Deleting corrupted cache record uid=61: UID 61: Broken virtual size in mailbox ... : read(compress(/data/vmail/...)): FETCH BODY[] got too little data: 38910 vs 70020"
I tried to recover these messages by uncompress files with your command, and I can say it was possible to uncompress some files while others (the one that were reffered by these errors lines) seems can't be recovered.
Seems like I have data lost in some messages and this is not something I got used with dovecot which I use since pre-1.0 version.
P.S. I also tried gzip-ing messages so how have to ask you for the command to un-gzip messages, as "doveadm fs get compress gzip:6:posix:prefix=$PWD/ filename" seems not working for me.
On 17/01/2024 18:08 EETachekalin@lazurit.com wrote:
Some time ago I enabled compression on incoming messages, so LMTP delivered messages were compressed. Good idea in terms of disk space, but now I cannot uncompress any of these messages even when I need to!
I was using this setting:
plugins { zlib_save = lz4 }
and now I can see this string at the beginning of each message file - "Dovecot-LZ4", so no other software can uncompress them for me.
Please advise, how can I uncompress message file in Maildir mailbox when I need to access its contents directly?
One simple way:
doveadm fs get compress lz4:6:posix:prefix=$PWD/ filename
AkiThank you! Here is the whole problem I see: as I saw how good it is to have messages compressed I used https://github.com/ George-NG/dovecot-maildir-compress script to compress whole mailboxes and thus save more space on disk. Looks like lock won't worked well so now I can see lines like that in Dovecot logs: Error: Mailbox ... : Deleting corrupted cache record uid=61: UID 61: Broken virtual size in mailbox ... : read(compress(/data/vmail/...)): FETCH BODY[] got too little data: 38910 vs 70020" I tried to recover these messages by uncompress files with your command, and I can say it was possible to uncompress some files while others (the one that were reffered by these errors lines) seems can't be recovered. Seems like I have data lost in some messages and this is not something I got used with dovecot which I use since pre-1.0 version. P.S. I also tried gzip-ing messages so how have to ask you for the command to un-gzip messages, as "doveadm fs get compress gzip:6:posix:prefix=$PWD/ filename" seems not working for me.
On 17/01/2024 18:08 EET achekalin@lazurit.com wrote: Some time ago I enabled compression on incoming messages, so LMTP delivered messages were compressed. Good idea in terms of disk space, but now I cannot uncompress any of these messages even when I need to! I was using this setting: plugins { zlib_save = lz4 } and now I can see this string at the beginning of each message file - "Dovecot-LZ4", so no other software can uncompress them for me. Please advise, how can I uncompress message file in Maildir mailbox when I need to access its contents directly? One simple way: doveadm fs get compress lz4:6:posix:prefix=$PWD/ filename Aki
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
Is it possible that those mails are missing the S= and/or W= size
flags in the filename?
I remember that being a requirement (the S= part, not sure if W= was
required).
I have stopped using maildir many years ago, and my script made sure
S= was added to every email, where the script you linked to does not,
it assumes it already exists.
Quoting Alexander Chekalin achekalin@lazurit.com:
Thank you!
Here is the whole problem I see:
as I saw how good it is to have messages compressed I used
https://github.com/George-NG/dovecot-maildir-compress script to
compress whole mailboxes and thus save more space on disk. Looks
like lock won't worked well so now I can see lines like that in
Dovecot logs:Error: Mailbox ... : Deleting corrupted cache record uid=61: UID 61:
Broken virtual size in mailbox ... :
read(compress(/data/vmail/...)): FETCH BODY[] got too little data:
38910 vs 70020"I tried to recover these messages by uncompress files with your
command, and I can say it was possible to uncompress some files
while others (the one that were reffered by these errors lines)
seems can't be recovered.Seems like I have data lost in some messages and this is not
something I got used with dovecot which I use since pre-1.0 version.P.S. I also tried gzip-ing messages so how have to ask you for the
command to un-gzip messages, as "doveadm fs get compress
gzip:6:posix:prefix=$PWD/ filename" seems not working for me.On 17/01/2024 18:08 EETachekalin@lazurit.com wrote:
Some time ago I enabled compression on incoming messages, so LMTP
delivered messages were compressed. Good idea in terms of disk
space, but now I cannot uncompress any of these messages even when
I need to!I was using this setting:
plugins { zlib_save = lz4 }
and now I can see this string at the beginning of each message
file - "Dovecot-LZ4", so no other software can uncompress them for
me.Please advise, how can I uncompress message file in Maildir
mailbox when I need to access its contents directly?One simple way:
doveadm fs get compress lz4:6:posix:prefix=$PWD/ filename
Aki
participants (4)
-
achekalin@lazurit.com
-
Aki Tuomi
-
Alexander Chekalin
-
Patrick Domack