Hi,

Hope you are well.

We have a cluster of 2 Dovecot servers, both on v2.3.13 (89f716dc2), using dsync to sync the messages between them.

Previously, we used TCPS to sync the messages, but after some testing, we concluded that syncing over SSH resulted in a lot less failed syncs, so we started using SSH.

The change has been a success, but I now notice a issue when a email client immediately moves a messages to a separate folder while Dovecot is syncing, where the message is seen twice in the folder (with the exact same headers/content).

For example, I have set a filter in my email client, Thunderbird, to immediately move all emails coming from this list to a separate folder. When opening that folder, I do not see one, but two unread messages, both identical to each other.

When checking the directories on the server, I see this:
mx1:
/var/vmail/example.com/joe/Maildir/.Subdir.Subdir/cur/1652615808.M190190P990486.mx2,S=19089,W=19384:2,S
/var/vmail/example.com/joe/Maildir/.Subdir.Subdir/cur/1652615811.M180050P1376677.mx1,S=19089,W=19384:2,S

mx2:
/var/vmail/example.com/joe/Maildir/.Subdir.Subdir/cur/1652615811.M981426P990530.mx2,S=19089,W=19384:2,S
/var/vmail/example.com/joe/Maildir/.Subdir.Subdir/cur/1652615808.M190190P990486.mx2,S=19089,W=19384:2,S

Note the difference: on mx1, one indicates mx1, and one mx2, while on the other server, both indicate mx2/

Any idea (other then telling end-users not to use such filters) how we could prevent these duplicate messages?

This is our config:
# Pigeonhole version 0.5.13 (cdd19fe3)
# OS: Linux 5.10.0-13-cloud-amd64 x86_64 Debian 11.3
# Hostname: mx1.example.com
auth_mechanisms = plain login
disable_plaintext_auth = no
dsync_remote_cmd = ssh -p 222 -l%{login} %{host} doveadm dsync-server -u%u
imap_capability = +SPECIAL-USE XLIST
listen = *,[::]
lmtp_rcpt_check_quota = yes
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_max_userip_connections = 100
mail_plugins = quota
mail_privileged_group = vmail
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
  separator = .
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
plugin {
  mail_replica = remote:root@mx2.example.com
  quota = dict:user::file:/var/vmail/%d/%n/.quotausage
  quota_status_nouser = DUNNO
  quota_status_overquota = 552 5.2.2 Mailbox is full
  quota_status_success = DUNNO
  sieve = /var/vmail/%d/%n/.sieve
  sieve_after = /var/vmail/%d/%n/.ispconfig.sieve
  sieve_before = /var/vmail/%d/%n/.ispconfig-before.sieve
  sieve_max_actions = 100
  sieve_max_redirects = 25
  sieve_max_script_size = 2M
}
protocols = imap pop3 lmtp
replication_max_conns = 50
service aggregator {
  fifo_listener replication-notify-fifo {
    mode = 0666
    user = vmail
  }
  unix_listener replication-notify {
    mode = 0666
    user = vmail
  }
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
  unix_listener auth-userdb {
    group = vmail
    mode = 0600
    user = vmail
  }
  user = root
}
service imap-login {
  client_limit = 1000
  process_limit = 512
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0600
    user = postfix
  }
}
service quota-status {
  client_limit = 1
  executable = quota-status -p postfix
  unix_listener /var/spool/postfix/private/quota-status {
    group = postfix
    mode = 0660
    user = postfix
  }
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
    mode = 0666
    user = vmail
  }
}
service stats {
  unix_listener stats-reader {
    group = vmail
    mode = 0660
    user = vmail
  }
  unix_listener stats-writer {
    group = vmail
    mode = 0660
    user = vmail
  }
}
ssl_cert = </etc/postfix/smtpd.cert
ssl_cipher_list = kEECDH:+kEECDH+SHA:kEDH:+kEDH+SHA:+kEDH+CAMELLIA:kECDH:+kECDH+SHA:kRSA:+kRSA+SHA:+kRSA+CAMELLIA:!aNULL:!eNULL:!SSLv2:!RC4:!MD5:!DES:!EXP:!SEED:!IDEA:!3DES
ssl_client_ca_dir = /etc/ssl/certs
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
ssl_min_protocol = TLSv1.2
ssl_prefer_server_ciphers = yes
userdb {
  driver = prefetch
}
userdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
protocol imap {
  auth_verbose = yes
  mail_plugins = quota quota imap_quota quota notify replication
}
protocol pop3 {
  auth_verbose = yes
  mail_plugins = quota quota quota notify replication
  pop3_uidl_format = %08Xu%08Xv
}
protocol lda {
  mail_plugins = quota sieve quota sieve quota notify replication
  postmaster_address = postmaster@mx1.example.com
}
protocol lmtp {
  mail_plugins = quota quota sieve sieve quota notify replication
  postmaster_address = postmaster@mx1.example.com
}

Thanks in advance for your thoughts.

Kind regards,

Thom Pol