doveadm sync issues (revised)

[Sven Strickroth]

Hi,

I'm experiencing serious issues using doveadm sync; mails are not
replicating and/or mails are duplicating. I wanted to use this for
migrating external mailboxes to my server (in order to overcome the time
when some servers/clients still use the old server and some already my
server, as described on <https://wiki2.dovecot.org/Tools/Doveadm/Sync>).

Finally I was able to reproduce this with the 2.3 branch (cf.
<https://dovecot.org/pipermail/dovecot/2019-April/115600.html>). I'm
sorry that I can only test the version 2.3.4.1, but according to the
changelog I think this is still an issue with the latest version.

I tried to find a few steps in order to replicate this easily:
For testing purposes I simulate both the remote and local server on my
Dovecot instance. I created remote at DOMAIN.TLD and local at DOMAIN.TLD
accounts and both use mdbox and are empty.

First, send two mails to remote at DOMAIL.TLD (Subjects: Test1 and Test2).

Start backup remote IMAP account (remote at DOMAIN.TLD) to local account
(local at DOMAIN.TLD):
$ doveadm -o imapc_user="remote at DOMAIN.TLD" -o imapc_password="PW" -o
imapc_host="localhost" backup -R -u local at DOMAIN.TLD imapc:

Both mailboxes are in sync now (containing Test1 und Test2).

Now send two messages to remote at DOMAIN.TLD (Subjects Test3 and Test4)
and two messages to local at DOMAIN.TLD (Subjects Test5 and Test6).

Run sync:
$ doveadm -o imapc_user="remote at DOMAIN.TLD" -o imapc_password="PW" -o
imapc_host="localhost" sync -R -u local at DOMAIN.TLD imapc:

See that syncing fails:
remote at DOMAIN.TLD contains: Test1,2,3,4,5,6 (as expected)
local at DOMAIN.TLD contains:  Test1,2,5,6,4 (Test3 is missing here)
When not using the "-R" parameter it seems to work.

Running sync again:
$ doveadm -o imapc_user="remote at DOMAIN.TLD" -o imapc_password="PW" -o
imapc_host="localhost" sync -R -u local at DOMAIN.TLD imapc:
causes messages to duplicate:
remote at DOMAIN.TLD contains: Test1,2,6,5,4,3,5,6,4,6,5,4,3,5,6
local at DOMAIN.TLD contains:  Test1,2,5,4,6,4,5,6 (3 is still missing)
This is independent of the "-R" parameter.

I could also experience this issue when using one-way sync, however, the
first one-way sync is still ok, but then messages start duplicating as
for the two-way sync without "-R":
$ doveadm -o imapc_user="remote at DOMAIN.TLD" -o imapc_password="PW" -o
imapc_host="localhost" sync -R -1 -u local at DOMAIN.TLD imapc:

Hope you can replicate this, too, and finally fix the issue.

-- 
Best regards,
 Sven Strickroth
 PGP key id F5A9D4C4 @ any key-server