dsync not deleting mail on remote side

Tue Nov 23 13:12:48 UTC 2021

Hello

Short version:

Dovecot's dsync from release 2.3.7.2 (default install from focal repository)
running under Ubuntu 20.04.6 LTS server is not deleting mails on the
identical remote server. 

Background:

I have installed this configuration using automated scripts several times
for customers. Functionally identical installations done on Ubuntu LTS
Servers 14.04, 16.04 and 18.04 are running for years without problem. Now I
have tried several times to do the same with Ubuntu 20.04 and can't get
dsync to work properly. Last year I could fall back onto Ubuntu 18.04. This
time I can't and must install the latest versions.

Setup:

-          Two VM's on ESXI 6.7U3 with 36 GB of disk space and 8 GB ram, 10k
spinning disks or SSD with HW RAID1

-          Mariadb for mail users (master - master synched)

-          Postfixadmin to manage the users

-          Postfix

-          Dovecot

-          Some with Roundcube 

-          The clients can connect to either one server and connect SMTPS
(465 or 587) or POPS (995) to the servers.

-          The two servers connect on a dedicated network without routing
(SSL, port > 49000). The synchronization network has no mail traffic except
the one that synchronizes the servers.

-          Additionally the two servers are ntp peers within a few ms - but
that is only relevant to know when we start comparing logs from the machines

Problem description:

-          When I send one mail to one account, it is immediately synched to
the other server. This is also seen in the dovecot log and I can connect to
any one of the two servers to download it.

-          When I read and delete the mail on one server, it is NOT deleted
on the other server. I do not see a trigger in the dovecot log that kicks on
the synchronization (like I do when sending mails). As a result, the mail
can be downloaded and deleted from both servers.

-          When I send a new mail to one of the two servers, the
synchronization takes place and the mail that I read and deleted is also
gone on the remote (backup) server.

-          Changing users, rights in dsynch.conf , SSL on or off, or
connecting to PLAIN ports 25, 110 or 143 is never improved it. Iptables or
ufw are not in use.

-           In my case I have systems using the server that' don't like
duplicate mails and the servers fail the acceptance test criteria.

-          This causes duplicate mail if one client is connecting to the
other server. Since a new mail fixes that duplicate mail problem, it is
likely that up to now nobody detects it and complains about this. Especially
if the servers would run in a primary - backup mode. I assume that's why I
could not find reportings on the web until now.

Dovecot -n:

See attachments

Log from my automated test system using Ubuntu 20.01 and focal default
dovecot package from repository (dovecot 2.3.7.2):

2021-11-22 18:20:06.30 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-22 18:20:06.81 - SMTP Server: mail-pri Port 465 TLS      - User:
user at test.ch - To:   user at test.ch - Subject: Test 2021-11-22 18:20:06, Msg
00001

2021-11-22 18:20:14.90 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-22 18:20:15.37 - POP3 Server: mail-pri Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-22 18:20:06, Msg
00001 (retr no delete)

2021-11-22 18:20:17.32 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-22 18:20:17.77 - POP3 Server: mail-sec Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-22 18:20:06, Msg
00001 (retr no delete)     -> synchronization worked when sending mails

2021-11-22 18:20:27.52 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-22 18:20:27.96 - POP3 Server: mail-pri Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-22 18:20:06, Msg
00001 (retr and delete)

2021-11-22 18:20:30.72 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-22 18:20:31.17 - POP3 Server: mail-sec Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-22 18:20:06, Msg
00001 (retr and delete)   -> synchronization failed after POP3 connection
with delete

2021-11-22 18:20:53.06 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-22 18:20:53.66 - SMTP Server: mail-sec Port 465 TLS      - User:
user at test.ch - To:   user at test.ch - Subject: Test 2021-11-22 18:20:53, Msg
00002

2021-11-22 18:20:57.91 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-22 18:20:58.33 - POP3 Server: mail-pri Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-22 18:20:53, Msg
00002 (retr no delete)

2021-11-22 18:21:00.82 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-22 18:21:01.27 - POP3 Server: mail-sec Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-22 18:20:53, Msg
00002 (retr no delete)

2021-11-22 18:21:04.65 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-22 18:21:05.11 - POP3 Server: mail-pri Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-22 18:20:53, Msg
00002 (retr and delete)

2021-11-22 18:21:08.95 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-22 18:21:09.40 - POP3 Server: mail-sec Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-22 18:20:53, Msg
00002 (retr and delete)

2021-11-22 18:21:39.50 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

Reference log with the same installation using Ubuntu 18.04 and bionic
default package form repository (dovecot 1:2.2.33.2-1ubuntu4.7):

2021-11-23 13:55:06.93 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-23 13:54:49.86 - SMTP Server: mail-pri Port 465 TLS      - User:
user at test.ch - To:   user at test.ch - Subject: Test 2021-11-23 13:54:49, Msg
00001

2021-11-23 13:55:06.93 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-23 13:55:07.47 - POP3 Server: mail-pri Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-23 13:54:49, Msg
00001 (retr no delete)

2021-11-23 13:55:13.10 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-23 13:55:13.66 - POP3 Server: mail-sec Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-23 13:54:49, Msg
00001 (retr no delete)   -> synchronization OK for sending mails

2021-11-23 13:55:22.32 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-23 13:55:22.87 - POP3 Server: mail-pri Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-23 13:54:49, Msg
00001 (retr and delete)

2021-11-23 13:55:27.91 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-23 13:55:28.35 - POP3 Server: mail-sec Port 995 TLS      - User:
user at test.ch                      - no mails found
-> synchronization OK for deleting mails

2021-11-23 13:55:36.48 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-23 13:55:37.00 - SMTP Server: mail-sec Port 465 TLS      - User:
user at test.ch - To:   user at test.ch - Subject: Test 2021-11-23 13:55:36, Msg
00002

2021-11-23 13:55:45.94 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-23 13:55:46.48 - POP3 Server: mail-pri Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-23 13:55:36, Msg
00002 (retr no delete)

2021-11-23 13:55:55.59 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-23 13:55:56.14 - POP3 Server: mail-sec Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-23 13:55:36, Msg
00002 (retr no delete)

2021-11-23 13:56:05.70 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-23 13:56:06.26 - POP3 Server: mail-pri Port 995 TLS      - User:
user at test.ch - From: user at test.ch - Subject: Test 2021-11-23 13:55:36, Msg
00002 (retr and delete)

2021-11-23 13:56:38.70 -
----------------------------------------------------------------------------
----------------------------------------------------------------------

2021-11-23 13:56:39.12 - POP3 Server: mail-sec Port 995 TLS      - User:
user at test.ch                      - no mails found

P.S. I compared the dovecot and postfix config files between these two
versions and both servers and they are functionally identical except where
they need to be different.

I would appreciate any help here. If needed I can create log files that show
that nothing is triggered as well as produce wireshark captures (port 25 and
110) or the link for the synch (currently with ssl but I could change that
if needed). 

Thanks in advance 

Otto

-- 
This email has been checked for viruses by AVG.
https://www.avg.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20211123/a214d380/attachment-0001.htm>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dovecot-n-config_mdt-sec_2021-11-21-1800.txt
URL: <https://dovecot.org/pipermail/dovecot/attachments/20211123/a214d380/attachment-0002.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dovecot-n-config_mdt-pri_2021-11-21-1800.txt
URL: <https://dovecot.org/pipermail/dovecot/attachments/20211123/a214d380/attachment-0003.txt>