Replication Troubles

[Reuben Farrelly]

On 20/02/2017 9:09 PM, Timo Sirainen wrote:
> On 17 Feb 2017, at 21.37, Wolfgang Hennerbichler <wogri at wogri.com>
> wrote:
>>
>> Hi Dovecot Users,
>>
>> I’ve configured dovecot dsync replication and I see troubles in the
>> logs and get user complaints which I can’t explain. I found similar
>> threads on this mailinglist, but I couldn’t find a solution
>> anywhere. Does anybody have dsync running without problems on a
>> high volume mailserver?
>>
>> I see the following logs, examples given:
>>
>> Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning:
>> /var/mail/zoechi/dovecot-uidlist: Duplicate file entry at line
>> 10395: 1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377
>> (uid 41092 -> 41093) - retrying by re-reading from beginning
>>
>> with this one I’m not sure - it might be that this is completely OK
>> because due to replication UIDs clash. Maybe that’s OK, but I
>> couldn’t find a confirmation.
>>
>> Feb 17 18:16:49 dovecot dovecot: imap(zoechi): Warning: Maildir
>> /var/mail/zoechi: Expunged message reappeared, giving a new UID
>> (old uid=41092,
>> file=1487350019.M138380P28563.dovecot.wogri.at,S=18930,W=19377)
>
> There seems to be something weird with using Maildir and replication.
> Haven't had time to debug it and it's likely not an easy bug to fix,
> so for now the solution would be to use only sdbox/mdbox with
> replication.


I have experienced this quite a few times.  It is tricky to reliably 
reproduce.  I am also using replication with Maildir.

This appears to be a timing/locking issue of some sort.  One way to 
trigger this seems to be to have a new mail come in and be delivered, 
and then as soon as it appears in Thunderbird, delete it straight away.

Thunderbird is set to move mails to Trash when they are deleted and 
expunge the Inbox on exit.

My replication partner sees the new mail has come in, and then it pushes 
it back to the original.  Meanwhile the one on the original has already 
been deleted by the client so the mail re-appears back on the original 
again.

The latency to my replication partner is 150ms or so but this problem 
might become easier to reproduce if the delay is increased.

I can't say that this is definitely how to reproduce it all of the time 
but when I have experience this, that's roughly the sequence of events.

Reuben