flags not synced correctly with dovecot sync (dsync)

[Marc Weustink]

Dan Christensen via dovecot wrote:
> I should add that these problems arise frequently in practice, even if
> changes are only being made on a single machine, e.g. laptop1.  Flag
> changes on laptop1 propagate to server, but then are reverted on server
> when laptop2 syncs with server, and then are reverted on laptop1 when it
> syncs again with server.  If I redo the flag changes on laptop1, the
> process just repeats and they are removed again.
> 
> Suggestions for debugging are very welcome, as currently syncing is
> much too unreliable for me to continue using it.
> 
> Dan

I (and others) have the same issue.

A quote from Stephan Bosch on 20-1-2019:

>> We're working on it. We have reproduced the problem without difficulty and isolated the commit that caused it.
> 
> Tracking internally as DOP-842.
> 
> Regards,
> 
> Stephan. 


Marc


> 
> On Feb 16, 2019, Dan Christensen via dovecot <dovecot at dovecot.org> wrote:
> 
>> I'm running dovecot 2.3.4.1 from https://repo.dovecot.org/ on Ubuntu
>> 18.04 on three machines that I'll call server, laptop1 and laptop2.
>>
>> Both laptop1 and laptop2 run dovecot sync against server to keep local
>> copies of my imap folders.  Even when I initially had only two machines,
>> laptop1 and server, I occasionally noticed that flags were lost, usually
>> custom flags used by Gnus, but I couldn't reliably reproduce the
>> problem.
>>
>> Now that I have two laptops syncing against the server, the problem has
>> gotten worse and I figured out a way to reproduce it:
>>
>> - on server: create new IMAP folder test, and put two read messages in it
>> - on laptop1:  doveadm sync -u user -l 10 -m test -f user at server
>> - on laptop2:  doveadm sync -u user -l 10 -m test -f user at server
>>
>> At this point, all three machines show the two messages M1 and M2
>> as being read.
>>
>> - on laptop1: mark message M1 unread
>> - on laptop2: mark message M2 unread
>> - on laptop1:  doveadm sync -u user -l 10 -m test -f user at server
>>    Both laptop1 and server have M1 unread, M2 read, as expected.
>> - on laptop2:  doveadm sync -u user -l 10 -m test -f user at server
>>    Now laptop2 and server have M1 *read*, M2 unread.
>> - on laptop1:  doveadm sync -u user -l 10 -m test -f user at server
>>    Now laptop1 and the server have both M1 and M2 *read*.
>> - on laptop2:  doveadm sync -u user -l 10 -m test -f user at server
>>    Now laptop2 has both read as well.
>>
>> The two lines that say "*read*" are wrong in my opinion.  dsync
>> propagated a read mark to an unread message, even though that message
>> was marked unread more recently than it was marked read.
>>
>> I usually use stateful sync, and get many related problems.
>> I just did a test in which M1 and M2 started out read, and I
>> started with empty files named dstate.test on laptop1 and laptop2.
>> Then I did the above procedure, using the command
>>
>> doveadm sync -u user -l 10 -m test -s "`cat dstate.test`" user at server > dstate.test
>>
>> At the end, laptop2 and server had both messages unread (which is good),
>> but laptop1 had only M1 unread, and repeated runs of the sync command
>> did not correct this.  So the stateful sync failed to detect a change.
>>
>> Are these bugs in dovecot?  Is there more information that I can
>> provide?  The output of doveconf -n on one machine is below, and
>> the others are almost identical.
>>
>> Thanks for any help!
>>
>> Dan
>>
>> # 2.3.4.1 (3c0b8769e): /etc/dovecot/dovecot.conf
>> # OS: Linux 4.15.0-45-generic x86_64 Ubuntu 18.04.1 LTS
>> # Hostname: laptop2
>> auth_mechanisms = plain login
>> listen = 127.0.0.1
>> mail_index_log2_max_age = 10 days
>> mail_index_log_rotate_min_age = 1 days
>> mail_index_log_rotate_min_size = 300 k
>> mail_location = maildir:~/Maildir
>> namespace inbox {
>>    inbox = yes
>>    location =
>>    mailbox Drafts {
>>      special_use = \Drafts
>>    }
>>    mailbox Junk {
>>      special_use = \Junk
>>    }
>>    mailbox Sent {
>>      special_use = \Sent
>>    }
>>    mailbox "Sent Messages" {
>>      special_use = \Sent
>>    }
>>    mailbox Trash {
>>      special_use = \Trash
>>    }
>>    prefix =
>> }
>> passdb {
>>    args = scheme=CRYPT username_format=%u /etc/dovecot/users
>>    driver = passwd-file
>> }
>> protocols = imap
>> service imap-login {
>>    inet_listener imap {
>>      address = *
>>      port = 143
>>    }
>>    inet_listener imaps {
>>      address = *
>>      port = 943
>>      ssl = yes
>>    }
>> }
>> service imap {
>>    process_limit = 25
>> }
>> ssl_cert = </etc/dovecot/private/dovecot.pem
>> ssl_client_ca_dir = /etc/ssl/certs
>> ssl_dh = # hidden, use -P to show it
>> ssl_key = # hidden, use -P to show it
>> userdb {
>>    args = username_format=%u /etc/dovecot/users
>>    driver = passwd-file
>> }
>> protocol lda {
>>    postmaster_address = [elided]
>> }
>> protocol imap {
>>    mail_max_userip_connections = 20
>> }
>