I'm upgrading a mail server using postfix 2.9.6 and dovecot 2.0.19 to a new server running postfix 3.4.13 and dovecot 2.3.7.2 (using Maildir format on both sides).
I had trouble getting doveadm backup to work for one user, while all the others worked straight away. It would complain about problems on the old server in the Maildir/dovecot* files. I managed to get it to work by removing all the dovecot* files under Maildir for that one user (accepting that this would mean clients having to redownload emails).
From that point, a doveadm backup worked.
Using roundcube to look at the new mail server, I could see everything arrived.
At this point, some mail generated on the new mail server would go to the new mail server, but all other mail is going to the old mail server.
I then tried to use doveadm sync -1R to update the new mail server so the new server contains all the mail from the old server plus the new mail that has been locally delivered.
What then happened was that all the new emails delivered to the old server since the backup were copied to the new server, and all the emails that were locally delivered on the new mail server were still there.
(So far so good).
But I also saw that every email that was on the old server before the backup now appeared twice on the new mail server.
Running sync again would mean I had three copies of email.
The sync ran quickly - much too quickly for these extra copies to have been sent over the network. It appears that the sync made extra local copies on the new server.
I see no errors in logs that can explain things.
Looking at the email, the files stored in the cur directories are there multiple times; they have different names, but identical content.
Trying "doveadm deduplicate -F user-list ALL" ran quickly and to no effect (where user-list is a file with a list of users). Trying "doveadm deduplicate -F user-list -m ALL" takes 100% cpu and a long time to run and still running right now after 4 hours. Judging by the disk space usage freed up, this is going to take a very long time to complete.
Has anybody got any idea what might cause this and what change I should make?
This was the sync command line:
doveadm -vc /etc/dovecot/dovecot-migration.conf sync -1RF user-list imapc:
For the backup and sync, I used this as the config:
imapc_features = rfc822.size fetch-headers
imapc_host = <snipped>
imapc_ssl = imaps
imapc_port = 993
imapc_user = %n
imapc_master_user = master
imapc_password = <snipped>
mail_prefetch_count = 20
ssl_cipher_list = EECDH+AESGCM+AES128:EECDH+AESGCM+AES256:EECDH+CHACHA20:EDH+AESGCM+AES128:EDH+AESGCM+AES256:EDH+CHACHA20:EECDH+SHA256+AES128:EECDH+SHA384+AES256:EDH+SHA256+AES128:EDH+SHA256+AES256:EECDH+SHA1+AES128:EECDH+SHA1+AES256:EDH
+SHA1+AES128:EDH+SHA1+AES256:EECDH+HIGH:EDH+HIGH:AESGCM+AES128:AESGCM+AES256:CHACHA20:SHA256+AES128:SHA256+AES256:SHA1+AES128:SHA1+AES256:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK:!KRB5:!aECDH:!DH
mail_location = maildir:~/Maildir
mail_home = /var/vmail/%d/%n/
mail_fsync=never