mdbox import error from read-only filesystem

[hby]

Hi,

Thanks for the fast answer!

Let me clear the situation:
1. I want to import from a read-only filesystem where the mdbox files are
2. The indexes are necessary because of the mdbox file layout, but the
import process cannot open them
3. If I specify random writable dir for indexes the import does not do
anything, no message will be imported as obviously the indexes are
necessary for mdbox
4. The CONTROL parameter is only usable for maildir format

To make it more specific, I'm using the following command to restore the
messages right now:
doveadm -o mail_location=mdbox:/backup/restore/XXX -o
mail_attachment_dir=/mnt/_attachments backup -u XXX doveadm dsync-server -u
XXX

For this to work, I have to copy the full mdbox data to the writeable
/backup/restore/XXX. If I only change the INDEX location, the process still
tries to open the log files and fails with open() error on them.

Sami Ketola <sami.ketola at dovecot.fi> ezt írta (időpont: 2019. jan. 14., H,
10:46):

> Hi,
>
> you can use INDEX=/writable/path/%u in your mail_location setting to
> define location for the required index data when importing.
> Also possibly you would need to define writable location for CONTROL and
> VOLATILEDIR.
>
> see https://wiki.dovecot.org/MailLocation
>
> Sami
>
> On 14 Jan 2019, at 11.23, hby <hby at hby.hu> wrote:
>
> Dovecot version: 2.2.34
>
> doveadm import tries to call open() on the source indexes/logs of mdbox
> data, but even if it should work as it is just a read-related call, it
> fails on read-only filesystems.
>
> The main use case for read-only filesystem is restoring from backup: the
> massive deduplicated backup data is exposed using FUSE. For deduplication
> to work efficiently, we have to backup the whole mail storage and thus the
> only way to handle the restore to mount the whole data.
>
> I would like to ask for some proper solution for this problem.
>
> Suggestions: first, the doveadm import should never modify the indexes,
> and second: the open() calls should respect the read-only filesystem and
> should fall back to "r" flag when using open.
> The problem goes further when using SIS (which goes beyond multiple
> terrabytes).
>
> The only workaround currently is to export the whole mdbox data to a
> writeable storage, and parametrize the import command to use that and the
> mounted SIS data separately for import...which is just problematic, ugly
> and error prune.
>
> Thank you for the continous work on dovecot, I hope you get this feedback
> in a good way.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20190114/a1a4126e/attachment-0001.html>