Re: Recovering expunged but not purged e-mails from mdbox with zlib compression
On 10/11/2020 20:07 Daniel Schütze dms@cwa.uk.com wrote:
Dear Aki Thank you. Unfortunately I'm struggling to get the right syntax for this as it looks like someone else was here too https://dovecot.org/pipermail/dovecot/2018-July/112441.html The location in my dovecot.conf is
mdbox:%%h/mdbox:INDEX=/indexdisk/indexes/%%n:INDEXPVT=~/mdbox/shared/%%n
This syntax is for accessing shared folders.
You probably should try
doveadm -Dv -o mail_location=mdbox_deleted:/path/to/mdbox fetch -u victim text ALL
I'm trying to fetch a message (for testing purposes now) based on it's guid as that is available from doveadm dump. So my syntax is (based on the previous person who didn't get it to work).
doveadm -o "mail_location=mdbox_deleted:%%h/mdbox:INDEX=/indexdisk/indexes/%%n:INDEXPVT=~/mdbox/shared/%%n" fetch "body" guid (msg.guid from doveadm dump)
Doveadm does not support var expand, so this will not work. See above for syntax.
But that's getting no response. I've tried putting in the hard path in case there is any trouble with the %%h etc but that doesn't help either. I've also tried the fetch with the subject of a known deleted message and also adding mailbox Drafts (as I know that's where it is or rather was) but to no avail. Given doveadm dump gives msg.uid and no subject I'd prefer to use that in any case. There are no error messages, I do with doveadm was a little more verbose to help over these bumps!
Any chance of a helping hand?
Aki
Daniel Schütze Director
CWA International Ltd 5th Floor, 42 Trinity Square London EC3N 4DJ
(t) + 44 (0)20 7242 8444 (e) dms@cwa.uk.com (w) http://www.cwa.international/ On 10/11/20 10:53, Aki Tuomi wrote:
You can use mdbox_deleted driver to access mails with refcount 0. See https://wiki2.dovecot.org/MailboxFormat/dbox
Aki
On 10.11.2020 12.42, Daniel Schütze wrote:
Yesterday I had to recover an e-mail which a user had deleted. If I understand this correctly the message was expunged but not purged (doveadm purge had not been run).
This e-mail was clearly still in the mdbox stored with zlib compression as I could tell using the doveadm dump command (doveadm dump -t dbox filename).
I could however not reveal the e-mail with the normal doveadm fetch -u username "body" guid (guid from dump output)
In the end I was able to recover the e-mail by cutting it out from the mdbox and running gunzip over it, but this method was very fiddly and would not have worked for a bulk job. I appreciate I could have just deleted the index files and gotten all the deleted messages back by having the index rebuilt but that would have been the proverbial "hammer to crack a nut".
Fortunately this e-mail had no attachment for me to worry about, as those are also detached for sis by dovecot.
Can anyone tell me of a quick and easy way to recover one or multiple e-mails marked as expunged but not purged which are stored in a mdbox with zlib compression? I'm sure I'll have to do this again the future and my method wouldn't work with a folder!
Clearly if I was not using zlib compression I could just have read the contents of the mdbox without any complication.
-- Daniel Schütze Director
CWA International Ltd 5th Floor, 42 Trinity Square London EC3N 4DJ
(t) + 44 (0)20 7242 8444 (e) dms@cwa.uk.com (w) http://www.cwa.international/
Dear Aki
Thank you again.
Running
doveadm -Dv -o maillocation=mdbox_deleted:/usr/home/vmail/mail/username/mdbox/storage/file fetch -u username text all
Does dump messages out.
However the messages do not include two messages which are present in the mdbox file and which I recovered by hand and so am using as a test to see if I can extract them effectively. These were in the Drafts mailbox so I've also run the command with mailbox Drafts at the end.
The output seems to be just dumping messages which are not expunged!
Daniel Schütze Director
CWA International Ltd 5th Floor, 42 Trinity Square London EC3N 4DJ
(t) + 44 (0)20 7242 8444 (e) dms@cwa.uk.com (w) http://www.cwa.international/ On 10/11/20 18:16, Aki Tuomi wrote:
On 10/11/2020 20:07 Daniel Schütze dms@cwa.uk.com wrote:
Dear Aki Thank you. Unfortunately I'm struggling to get the right syntax for this as it looks like someone else was here too https://dovecot.org/pipermail/dovecot/2018-July/112441.html The location in my dovecot.conf is
mdbox:%%h/mdbox:INDEX=/indexdisk/indexes/%%n:INDEXPVT=~/mdbox/shared/%%n This syntax is for accessing shared folders.
You probably should try
doveadm -Dv -o mail_location=mdbox_deleted:/path/to/mdbox fetch -u victim text ALL
I'm trying to fetch a message (for testing purposes now) based on it's guid as that is available from doveadm dump. So my syntax is (based on the previous person who didn't get it to work).
doveadm -o "mail_location=mdbox_deleted:%%h/mdbox:INDEX=/indexdisk/indexes/%%n:INDEXPVT=~/mdbox/shared/%%n" fetch "body" guid (msg.guid from doveadm dump)
Doveadm does not support var expand, so this will not work. See above for syntax.
But that's getting no response. I've tried putting in the hard path in case there is any trouble with the %%h etc but that doesn't help either. I've also tried the fetch with the subject of a known deleted message and also adding mailbox Drafts (as I know that's where it is or rather was) but to no avail. Given doveadm dump gives msg.uid and no subject I'd prefer to use that in any case. There are no error messages, I do with doveadm was a little more verbose to help over these bumps!
Any chance of a helping hand?
Aki
Daniel Schütze Director
CWA International Ltd 5th Floor, 42 Trinity Square London EC3N 4DJ
(t) + 44 (0)20 7242 8444 (e) dms@cwa.uk.com (w) http://www.cwa.international/ On 10/11/20 10:53, Aki Tuomi wrote:
You can use mdbox_deleted driver to access mails with refcount 0. See https://wiki2.dovecot.org/MailboxFormat/dbox
Aki
On 10.11.2020 12.42, Daniel Schütze wrote:
Yesterday I had to recover an e-mail which a user had deleted. If I understand this correctly the message was expunged but not purged (doveadm purge had not been run).
This e-mail was clearly still in the mdbox stored with zlib compression as I could tell using the doveadm dump command (doveadm dump -t dbox filename).
I could however not reveal the e-mail with the normal doveadm fetch -u username "body" guid (guid from dump output)
In the end I was able to recover the e-mail by cutting it out from the mdbox and running gunzip over it, but this method was very fiddly and would not have worked for a bulk job. I appreciate I could have just deleted the index files and gotten all the deleted messages back by having the index rebuilt but that would have been the proverbial "hammer to crack a nut".
Fortunately this e-mail had no attachment for me to worry about, as those are also detached for sis by dovecot.
Can anyone tell me of a quick and easy way to recover one or multiple e-mails marked as expunged but not purged which are stored in a mdbox with zlib compression? I'm sure I'll have to do this again the future and my method wouldn't work with a folder!
Clearly if I was not using zlib compression I could just have read the contents of the mdbox without any complication.
-- Daniel Schütze Director
CWA International Ltd 5th Floor, 42 Trinity Square London EC3N 4DJ
(t) + 44 (0)20 7242 8444 (e) dms@cwa.uk.com (w) http://www.cwa.international/
Could one use this "mail_location=mdbox_deleted:/path/to/mdbox" for lets say a 'recovery' namespace. Where users can copy their deleted messages from?
-----Original Message----- From: Aki Tuomi [mailto:aki.tuomi@open-xchange.com] Sent: Tuesday, November 10, 2020 7:16 PM To: Daniel Schütze; dovecot@dovecot.org Subject: Re: Recovering expunged but not purged e-mails from mdbox with zlib compression
On 10/11/2020 20:07 Daniel Schütze dms@cwa.uk.com wrote:
Dear Aki Thank you. Unfortunately I'm struggling to get the right syntax for this as it looks like someone else was here too https://dovecot.org/pipermail/dovecot/2018-July/112441.html The location in my dovecot.conf is
mdbox:%%h/mdbox:INDEX=/indexdisk/indexes/%%n:INDEXPVT=~/mdbox/shared/% %n
This syntax is for accessing shared folders.
You probably should try
doveadm -Dv -o mail_location=mdbox_deleted:/path/to/mdbox fetch -u victim text ALL
I'm trying to fetch a message (for testing purposes now) based on it's guid as that is available from doveadm dump. So my syntax is (based on the previous person who didn't get it to work).
doveadm -o "mail_location=mdbox_deleted:%%h/mdbox:INDEX=/indexdisk/indexes/%%n:IN DEXPVT=~/mdbox/shared/%%n" fetch "body" guid (msg.guid from doveadm dump)
Doveadm does not support var expand, so this will not work. See above for syntax.
But that's getting no response. I've tried putting in the hard path in case there is any trouble with the %%h etc but that doesn't help either. I've also tried the fetch with the subject of a known deleted message and also adding mailbox Drafts (as I know that's where it is or rather was) but to no avail. Given doveadm dump gives msg.uid and no subject I'd prefer to use that in any case. There are no error messages, I do with doveadm was a little more verbose to help over these bumps!
Any chance of a helping hand?
Aki
Daniel Schütze Director
CWA International Ltd 5th Floor, 42 Trinity Square London EC3N 4DJ
(t) + 44 (0)20 7242 8444 (e) dms@cwa.uk.com (w) http://www.cwa.international/ On 10/11/20 10:53, Aki Tuomi wrote:
You can use mdbox_deleted driver to access mails with refcount 0. See https://wiki2.dovecot.org/MailboxFormat/dbox
Aki
On 10.11.2020 12.42, Daniel Schütze wrote:
Yesterday I had to recover an e-mail which a user had deleted. If
I understand this correctly the message was expunged but not purged (doveadm purge had not been run).
This e-mail was clearly still in the mdbox stored with zlib compression as I could tell using the doveadm dump command (doveadm dump -t dbox filename).
I could however not reveal the e-mail with the normal doveadm fetch -u username "body" guid (guid from dump output)
In the end I was able to recover the e-mail by cutting it out from
the mdbox and running gunzip over it, but this method was very fiddly and would not have worked for a bulk job. I appreciate I could have just deleted the index files and gotten all the deleted
messages back by having the index rebuilt but that would have been
the proverbial "hammer to crack a nut".
Fortunately this e-mail had no attachment for me to worry about, as those are also detached for sis by dovecot.
Can anyone tell me of a quick and easy way to recover one or multiple e-mails marked as expunged but not purged which are stored in a mdbox with zlib compression? I'm sure I'll have to do
this again the future and my method wouldn't work with a folder!
Clearly if I was not using zlib compression I could just have read
the contents of the mdbox without any complication.
-- Daniel Schütze Director
CWA International Ltd 5th Floor, 42 Trinity Square London EC3N 4DJ
(t) + 44 (0)20 7242 8444 (e) dms@cwa.uk.com (w) http://www.cwa.international/
participants (3)
-
Aki Tuomi
-
Daniel Schütze
-
Marc Roos