Migrating from external server to Dovecot (doveadm sync issues)

[Sven Strickroth]

Hi,

I'm experiencing serious issues when using doveadm sync not working at
all or do not properly sync. I wanted to use this for migrating external
mailboxes to my server (in order to overcome the time when some
servers/clients still use the old server and some already the my server).

I tried to find a few steps in order to replicate this:
For testing purposes I simulate both the remote and local server on my
Dovecot instance. I created remote at DOMAIN.TLD and local at DOMAIN.TLD both
use mdbox and are empty (i.e., the mail_location is still non-existent).

Then I send two mails to remote at DOMAIL.TLD (Subjects: Test1 and Test2).

Start backup remote IMAP account (remote at DOMAIN.TLD) to local account
(local at DOMAIN.TLD):
* doveadm -o imapc_user="remote at DOMAIN.TLD" -o imapc_password="PW" -o
imapc_host="localhost" -o mail_prefetch_count=20 -o
imapc_features="rfc822.size fetch-headers" backup -R -u local at DOMAIN.TLD
imapc:
(without this I often got errors such as "Error: Backend didn't update
mailbox Trash GUID" on sync runs, which seems to be an unrecoverable error).

Now the two mailboxes seem to be the same (can confirm this when logging
into the two mailboxes).

Now, run first sync:
* doveadm -o imapc_user="remote at DOMAIN.TLD" -o imapc_password="PW" -o
imapc_host="localhost" -o mail_prefetch_count=20 -o
imapc_features="rfc822.size fetch-headers" sync -R -u local at DOMAIN.TLD
imapc:

Seems to work and does nothing at the moment.

Deliver one message to local at DOMAIN.TLD (Subject Test3).

* doveadm -o imapc_user="remote at DOMAIN.TLD" -o imapc_password="PW" -o
imapc_host="localhost" -o mail_prefetch_count=20 -o
imapc_features="rfc822.size fetch-headers" sync -R -u local at DOMAIN.TLD
imapc:
Now Test1,2,3 are in both mailboxes (further runs do change nothing).
Everything OK up the here.

Now deliver one messages to remote at DOMAIN.TLD (Test4) and delete one old
message (Test3). Deliver two messages to local at DOMAIN.TLD (Subjects
Test5 and Test6) and delete one message (Test2).
In sum both mailboxes look like this:
remote at DOMAIN.TLD has Test1,2,4 and local at DOMAIN.TLD has 1,3,5,6.

Now sync:
* doveadm -o imapc_user="remote at DOMAIN.TLD" -o imapc_password="PW" -o
imapc_host="localhost" -o mail_prefetch_count=20 -o
imapc_features="rfc822.size fetch-headers" sync -R -u local at DOMAIN.TLD
imapc:
dsync(local at DOMAIN.TLD): Warning: Mailbox changes caused a desync. You
may want to run dsync again: 3 UIDs changed due to UID conflicts

Rerun again and see that mails start duplicating and deletings are not
synced: remote at DOMAIN.TLD: Test1,2,4,3,5,6,5,6,6,5,3,4 (12 mails) and
local at DOMAIN.TLD: 1,6,4,3,5,6,4,3,5,6,5 (11 mails)

When I re-run again I sometimes see the above error message again, mails
keep duplicating in both mailboxes and the mailboxes are not synced at all.

Am I using doveadm sync wrongly? Is there a way to use "sync" directly
w/o backup from the beginning (i.e., how to get rid of the error "Error:
Backend didn't update mailbox Trash GUID").

Context:
# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.8
imap_client_workarounds = delay-newmail

I can't upgrade to latest Dovecot right now. :(

-- 
Best regards,
 Sven Strickroth
 PGP key id F5A9D4C4 @ any key-server