On 9/2/19 3:03 PM, Sami Ketola wrote:

...

On 2 Sep 2019, at 15.25, Peter Mogensen via dovecot <dovecot@dovecot.org> wrote: ... Is there anyway for dsync to avoid moving Gigabytes of data for could just be "moved" by moving the mount?

Not tested but you can probably do something like this in the target server:

doveadm backup -u victim -R ssh sudouser@old-server "sudo doveadm dsync-server -o mail_location=sdbox:/location-to-your-sdbox/ -u victim"

just leave ALT storage path from the setting.

I'll have to test this... but my initial guess would be that doveadm would then think the mails has disappeared. Would it then copy the index metadata for those mails to the target host anyway?

/Peter

On 9/3/19 2:38 PM, Sami Ketola wrote:

...

On 3 Sep 2019, at 15.34, Peter Mogensen via dovecot <dovecot@dovecot.org> wrote:

On 9/2/19 3:03 PM, Sami Ketola wrote:

...

...

On 2 Sep 2019, at 15.25, Peter Mogensen via dovecot <dovecot@dovecot.org> wrote: ... Is there anyway for dsync to avoid moving Gigabytes of data for could just be "moved" by moving the mount?

Not tested but you can probably do something like this in the target server:

doveadm backup -u victim -R ssh sudouser@old-server "sudo doveadm dsync-server -o mail_location=sdbox:/location-to-your-sdbox/ -u victim"

just leave ALT storage path from the setting.

I'll have to test this... but my initial guess would be that doveadm would then think the mails has disappeared. Would it then copy the index metadata for those mails to the target host anyway?

Hmm. That is true. It will probably not work after all then.

Now I'm out of ideas how to do this efficiently.

I assume it won't even work to just premount the shared storage read-only on the target side, so the mails are already there. ... since I suppose the receiving dsync reserves the right to re-pack the m.* storage files?

/Peter

On 9/4/19 2:12 PM, Peter Mogensen wrote:

So... I've done some testing.

One method which seemed to work - at least for primitive cases - was to:

• Mount the ALT storage on the destination.
• Run "doveadm force-resync \*" on the destination. (putting all the mails in ALT storage into the dovecot.map.index)
• Run dsync from source to destination.

Of course... if there was some way to avoid step 2...

So ... I have an idea.

Assuming users mail_location is:

mdbox:~/mdbox:ALT=/alt:INDEX=~/idx

And /alt is a shard mounted storage.

Then, it suspect the following steps would make dsync avoid transfering mails on shared storage:

1. Create a rudimentary mdbox on the target side (just containing the dbox-alt-root link)

2. Mount /alt on the target host

3. Copy all dovecot.index and dovecot.map.index in ~/idx from source to target. That is: not the transaction (*.log) files or cache files. I suppose this needs to be done under appropriate read locking.

4. doveadm sync -u source doveadm dsync-server -u target ... to get the rest of the mails in primary storage and all updates sine the index files where snapshot.

It would be nice if there was a way to force dovecot*index.log files to be snapshot to index files.

If the aim is not to sync two different accounts but to simply move one account from one host to a new host where it doesn't exist in advance, are there any caveats with this?

... apart from a few missing tools.

/Peter