Dsync deleting mailboxes due to duplicate UIDs

[Sami Ketola]

> On 9 Sep 2018, at 18.42, Andreas Thienemann <andreas at bawue.net> wrote:
> 
> Hi,
> 
> I am attempting to migrate a mailspool from a cyrus server to a dovecot server using the dsync backup approach as described in the wiki at <https://wiki.dovecot.org/Migration/Dsync>.
> 
> The first attempt works great. Everything copies over and a quick glance over the spool looks good.
> 
> Running doveadm backup again though results in the following error:
> 
> dsync(eggs): Warning: Deleting mailbox 'INBOX.MailRestore': UID=39 already exists locally for a different mail: Headers hashes don't match (18d567fc7e258a67e47b629c8bb16500 vs 230354b2d5cad21ebbb4a7440b977adb)
> 
> As promised, the folder MailRestore is gone after dsync finishes.
> 
> Running doveadm backup again for the third time copies the folder again.
> 
> Running it a fourth time gives the same error and the folder is gone.
> 
> Trying to figure this out I initially had a few mails where the header hashes were 68b329da9893e34099c7d8ad5cb9c940, the md5sum of an empty string.
> Turns out that the source mailspool had a few broken emails and cleaning these out fixed most my issues.
> 
> But in this case, I am stumped. UID 39 on the server is a valid mail with correct headers and everything.
> 
> Is there a good way of identifying the two mails that seem to clash? I had a quick look at the dovecot code but did not see how the header_stream gets hashed into the hdr_hash used for comparing mails.


Is it possible to get imapc rawlogs to analyze? create a directory that is writeable and add -o imapc_rawlog_dir=/path/to/directory to command line.

Also with recent dovecot release you can tell dsync which header fields to hash when matching mails. Add dsync_hashed_headers=Message-ID to config and dovecot will only mach mails using Message-ID header fileld.

dsync_hashed_headers setting is supported since dovecot 2.2.33.

Sami