[Dovecot] dsync 2.0.x versus 2.1.x field report

[Michael Grimm]

Hi --

I used to run two mailservers with identical MX priorities (poor man's fail-over approach) with Maildir mail folders accessed by Dovecot. Synchronization has been achieved using unison, mirroring all files except dovecot index files. 

Once in a while some oddities could be observed, like reappearing of deleted mails, duplicates and such alike. But the all-over performance was quite good.

Then I wanted to migrate to mdbox, and I became interested in dsync. I did start with 2.0.16 a couple of month ago, and recently upgraded to 2.0.18.

My experience with the 2.0.x dsync version is as follows (synchronization time: every minute):

-- As long as mail is not stored in corresponding mailboxes at both servers at the same time, all mailboxes become synchronized perfectly well, and no oddites like reappearing of deleted mails or generation of duplicates can be observed.

-- If mail is stored into corresponding mailboxes at both servers simultaneously in between two synching runs, it gets worrying. 

-- If I do run dsync without '-f' those two mailboxes end up in becoming asynchronous. 

-- If I do run dsync with '-f' both mailboxes will become synchronized let's say in 95% of all synchronization runs. BUT in 5%, I do experience asynchronicity. Here a 'doveadm force-resync' sometimes works, but not always, what worries me a lot. 

-- I can see the missing mail in the mdbox storage file using UNIX' little helpers but not with the help of dovecot tools. (Or I was unable to use them properly.)

-- Modifying MX priorities in a way, that I do now have a primary and backup mailserver scenario, very much reduced the risk of losing legitimate mail. 

-- But sometimes I do lose some syslog mail being produced at the same time. Therefore I modified runtimes in way to omit synchronicity.


Ok, I could have lived with that, but I was anxious to learn, if 2.1.x dsync would perform better ;-) 

Here my observations sofar (synchronization time: every minute):

-- No losses sofar using 'doveadm sync' with '-f'

-- A *lot* of reappearing deleted mails or mails moved from one mailbox to the other.

-- Quite often I am unable to delete those mails using client functionality (Mail.app, Roundcube).

-- In those cases one needs to stop synchronization, run 'doveadm force-resync', delete all reappearing mail (quite a lot sometimes), empty 'Deleted' folders, and finally run 'doveadm purge'.

-- Running of 'doveadm sync' without '-f' doesn't improve the situation. Still I do have to struggle with reappearing mails, and sometimes with appearing duplicates.

-- I do have a test setup running for 24 hours now, where I do generate test mails in a 6 minutes interval at both servers. Mails become synchronized perfectly well, irrespectively if '-f' is set or not. This is a major improvement to 2.0x. dsync (sofar).

-- But running the synchronization without '-f' still shows a *lot* of reappering mail, though.


I do have the feeling that dsync became very much improved with regard to missing mail in synchronization, good. But those reappearing mails make me think about switching back to 2.0.x dsync and live with the drawback of preventing mail to arrive synchronously at both servers.

Is someone out there experiencing a similar behaviour with dsync 2.1.x regarding reappearing, thus indelible mail?

Regards,
Michael