Retrieving mail from read-only mdbox
This is a 'has anyone run into this and solved it' post. And yes, I've been reading and re-reading TFM but without luck. The background is that I'm working on tooling before we start a mass maildir->mdbox conversion. One of those tools is recovering mail from backups (easy as pie with maildir).
We've got all of our email on Netapp file servers. They have nice snapshotting but the snapshots are, of course, readonly.
My question: is there a doveadm command that will allow for email to be retrieved from a readonly mdbox, either directly (like manipulating the mdbox files directly) or by doveadm talking to the dovecot processes?
Ideally, there'd be something like doveadm dump, but that could dump selected message contents.
I've tried using IMAP with mail_location pointed at the snapshot, but, though I can get a listing of emails in the mailbox, the fetch fails when dovecot can't write-lock dovecot.index.log.
If anyone has gotten something similar to work, I'd love to hear about it. A working IMAP setup would be the ideal, since it's more easily automatible (but I'll take whatever I can get).
Any and all hints are most welcome!
Quoting Mark Moseley moseleymark@gmail.com:
I've tried using IMAP with mail_location pointed at the snapshot, but, though I can get a listing of emails in the mailbox, the fetch fails when dovecot can't write-lock dovecot.index.log.
I'm surprised that dovecot would even try to write-lock a write-protected file/directory, though I can appreciate the situation where a file may be in a directory that is writable by some UID other than the one dovecot is running as.
Is there an unsafe control over lock_method similar to Samba's fake oplocks setting in Dovecot?
If anyone wants some good "horror" writing, a perusal of Jeremy Allison's write-up on the schizophrenic approaches to file-locking is worthy of your time.
https://www.samba.org/samba/news/articles/low_point/tale_two_stds_os2.html
=M=
On Wed, May 31, 2017 at 3:24 PM, M. Balridge dovecot@r.paypc.com wrote:
Quoting Mark Moseley moseleymark@gmail.com:
I've tried using IMAP with mail_location pointed at the snapshot, but, though I can get a listing of emails in the mailbox, the fetch fails when dovecot can't write-lock dovecot.index.log.
I'm surprised that dovecot would even try to write-lock a write-protected file/directory, though I can appreciate the situation where a file may be in a directory that is writable by some UID other than the one dovecot is running as.
Is there an unsafe control over lock_method similar to Samba's fake oplocks setting in Dovecot?
If anyone wants some good "horror" writing, a perusal of Jeremy Allison's write-up on the schizophrenic approaches to file-locking is worthy of your time.
https://www.samba.org/samba/news/articles/low_point/tale_two_stds_os2.html
There's no fake locks from what I can tell. If I'm reading the source right, the only valid options are fcntl, flock, and dotlock. Tried 'em all :)
This is a 'has anyone run into this and solved it' post. And yes, I've been reading and re-reading TFM but without luck. The background is that I'm working on tooling before we start a mass maildir->mdbox conversion. One of those tools is recovering mail from backups (easy as pie with maildir).
We've got all of our email on Netapp file servers. They have nice snapshotting but the snapshots are, of course, readonly.
My question: is there a doveadm command that will allow for email to be retrieved from a readonly mdbox, either directly (like manipulating the mdbox files directly) or by doveadm talking to the dovecot processes?
Ideally, there'd be something like doveadm dump, but that could dump selected message contents.
I've tried using IMAP with mail_location pointed at the snapshot, but, though I can get a listing of emails in the mailbox, the fetch fails when dovecot can't write-lock dovecot.index.log.
If anyone has gotten something similar to work, I'd love to hear about it. A working IMAP setup would be the ideal, since it's more easily automatible (but I'll take whatever I can get).
Any and all hints are most welcome!
Hi Mark,
I had exactly the same problem as you. I also tried to put the INDEX to a writeable fs (also to MEMORY) but it did not work. What I did in the end is that I created a writeable version of my read-only snapshot using AuFS. This way I was able to access the snapshot over IMAP and destroy the AuFS mount when finished.
It's not a perfect solution, but it works :)
Regards,
Peter
On 01.06.2017 00:30, Mark Moseley wrote:
This is a 'has anyone run into this and solved it' post. And yes, I've been reading and re-reading TFM but without luck. The background is that I'm working on tooling before we start a mass maildir->mdbox conversion. One of those tools is recovering mail from backups (easy as pie with maildir).
We've got all of our email on Netapp file servers. They have nice snapshotting but the snapshots are, of course, readonly.
My question: is there a doveadm command that will allow for email to be retrieved from a readonly mdbox, either directly (like manipulating the mdbox files directly) or by doveadm talking to the dovecot processes? What would this tooling do exactly? Is it for restoring the users existing (writable) account from a read-only backup? I would in that case recommend looking into using "doveadm backup" or "doveadm sync". They do provide som crude selection of messages on e.g. folder level.
In case you you are aiming to provide users online access to their own backups, maybe add them as a read-only mailbox using ACLs. Though other mails in this thread indicate there still might be problems.
br, Teemu
Ideally, there'd be something like doveadm dump, but that could dump selected message contents.
I've tried using IMAP with mail_location pointed at the snapshot, but, though I can get a listing of emails in the mailbox, the fetch fails when dovecot can't write-lock dovecot.index.log.
If anyone has gotten something similar to work, I'd love to hear about it. A working IMAP setup would be the ideal, since it's more easily automatible (but I'll take whatever I can get).
Any and all hints are most welcome!
participants (4)
-
M. Balridge
-
Mark Moseley
-
Peter Benko
-
Teemu Huovila