Message duplication in Sent folder after upgrade
Hi All,
I have migrated my mailserver (from dovecot 2.2 on debian 10 to dovecot 2.3 provided by the mailcow-dockerized project) by coping over the mailstorage directory "as is".
There is now a single mailbox where I have an issue with duplicated mails - the "Sent" folder had approx 400 mails upon migration, it now has 4500 mails with each mail duplicated several times. Below are the header lines of one message, besides the Message-Id the header is exactly the same on all copies.
I tried "doveadm force-resync" on the mailbox and the mailbox was removed and recreated from scratch on the mail app but we still get additional copies added - anybody has an idea what is going on here and how to fix this?
Oliver
From: "presse@example.com" <presse@example.com>
Content-Type: multipart/alternative;
boundary="Apple-Mail=_451F3D0B-155E-46EF-9BFB-B8034A7E2AE1"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.300.87.4.3\))
Subject: Re: EXTERN: Re: Example Mail
X-Universally-Unique-Identifier: 6CD82CDA-138B-412E-8D4D-74C8C82B72ED
Date: Mon, 13 Jan 2025 11:17:09 +0100
References:
<DB3PR05MB100724231A87B9AE130C0EE0C853A2@DB3PR05MB10072.eurprd05.prod.outlook.com>
<DB3PR05MB100725A1F1367C304AC417A9585002@DB3PR05MB10072.eurprd05.prod.outlook.com>
<DB3PR05MB1007220945CC70CDACDDBA5D685022@DB3PR05MB10072.eurprd05.prod.outlook.com>
<DB3PR05MB100724A5040FA029A29E625D1850D2@DB3PR05MB10072.eurprd05.prod.outlook.com>
<09B63703-8BC4-484F-905A-899BA9A72139@example.com>
<DB3PR05MB10072F836F9FCE2F9F626392D85142@DB3PR05MB10072.eurprd05.prod.outlook.com>
<BA8A1660-FC7D-4478-9F5F-E8B062B39115@example.com>
<005d01db659c$cf2a48c0$6d7eda40$@John.Doe@t-online.de>
To: John Doe <John.Doe@t-online.de>
In-Reply-To: <005d01db659c$cf2a48c0$6d7eda40$@Jon.Doe@t-online.de>
Message-Id: <A03E3B7A-638A-4482-83D8-C5B297D9953E@example.com>-- Protect your environment - close windows and adopt a penguin!
Hi All,
I have migrated my mailserver (from dovecot 2.2 on debian 10 to dovecot 2.3 provided by the mailcow-dockerized project) by coping over the mailstorage directory "as is".
There is now a single mailbox where I have an issue with duplicated mails
- the "Sent" folder had approx 400 mails upon migration, it now has 4500 mails with each mail duplicated several times. Below are the header lines of one message, besides the Message-Id the header is exactly the same on all copies.
I tried "doveadm force-resync" on the mailbox and the mailbox was removed and recreated from scratch on the mail app but we still get additional copies added - anybody has an idea what is going on here and how to fix this?
Oliver
From: [1]"presse@example.com" [2]<presse@example.com>
Content-Type: multipart/alternative;
boundary="Apple-Mail=_451F3D0B-155E-46EF-9BFB-B8034A7E2AE1"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.300.87.4.3\))
Subject: Re: EXTERN: Re: Example Mail
X-Universally-Unique-Identifier: 6CD82CDA-138B-412E-8D4D-74C8C82B72ED
Date: Mon, 13 Jan 2025 11:17:09 +0100
References:
[3]<DB3PR05MB100724231A87B9AE130C0EE0C853A2@DB3PR05MB10072.eurprd05.prod.outlook.com>
[4]<DB3PR05MB100725A1F1367C304AC417A9585002@DB3PR05MB10072.eurprd05.prod.outlook.com>
[5]<DB3PR05MB1007220945CC70CDACDDBA5D685022@DB3PR05MB10072.eurprd05.prod.outlook.com>
[6]<DB3PR05MB100724A5040FA029A29E625D1850D2@DB3PR05MB10072.eurprd05.prod.outlook.com>
[7]<09B63703-8BC4-484F-905A-899BA9A72139@example.com>
[8]<DB3PR05MB10072F836F9FCE2F9F626392D85142@DB3PR05MB10072.eurprd05.prod.outlook.com>
[9]<BA8A1660-FC7D-4478-9F5F-E8B062B39115@example.com>
[10]<005d01db659c$cf2a48c0$6d7eda40$@John.Doe@t-online.de>
To: John Doe [11]<John.Doe@t-online.de>
In-Reply-To: [12]<005d01db659c$cf2a48c0$6d7eda40$@Jon.Doe@t-online.de>
Message-Id: [13]<A03E3B7A-638A-4482-83D8-C5B297D9953E@example.com>-- Protect your environment - close windows and adopt a penguin!
References
Visible links
- mailto:presse@example.com
- mailto:presse@example.com
- mailto:DB3PR05MB100724231A87B9AE130C0EE0C853A2@db3pr05mb10072.eurprd05.prod.outlook.com
- mailto:DB3PR05MB100725A1F1367C304AC417A9585002@db3pr05mb10072.eurprd05.prod.outlook.com
- mailto:DB3PR05MB1007220945CC70CDACDDBA5D685022@db3pr05mb10072.eurprd05.prod.outlook.com
- mailto:DB3PR05MB100724A5040FA029A29E625D1850D2@db3pr05mb10072.eurprd05.prod.outlook.com
- mailto:09B63703-8BC4-484F-905A-899BA9A72139@example.com
- mailto:DB3PR05MB10072F836F9FCE2F9F626392D85142@db3pr05mb10072.eurprd05.prod.outlook.com
- mailto:BA8A1660-FC7D-4478-9F5F-E8B062B39115@example.com
- mailto:005d01db659c$cf2a48c0$6d7eda40$@John.Doe@t-online.de
- mailto:John.Doe@t-online.de
- mailto:005d01db659c$cf2a48c0$6d7eda40$@Jon.Doe@t-online.de
- mailto:A03E3B7A-638A-4482-83D8-C5B297D9953E@example.com
On Fri, 3 Oct 2025 at 13:55, Oliver Welter via dovecot <dovecot@dovecot.org> wrote:
There is now a single mailbox where I have an issue with duplicated mails
- the "Sent" folder had approx 400 mails upon migration, it now has 4500 mails with each mail duplicated several times. Below are the header lines of one message, besides the Message-Id the header is exactly the same on all copies.
Not sure if it is related to the upgrade, but we had a similar issue. In general, if Message-id is different, it means it is a different message, not the same message duplicated/copied. In our case, it was some antivirus plugin on client machine trying to move message into Junk/Spam folder, but changing it in the process, which caused it detected the moved message as a new one and a loop started. You can try to check your mailserver (SMTP) logs if you will find the different message ids there.
-- bye, Marki
Hi Marek
On 03.10.25 14:05, Marek Podmaka via dovecot wrote:
On Fri, 3 Oct 2025 at 13:55, Oliver Welter via dovecot <dovecot@dovecot.org> wrote:
There is now a single mailbox where I have an issue with duplicated mails - the "Sent" folder had approx 400 mails upon migration, it now has 4500 mails with each mail duplicated several times. Below are the header lines of one message, besides the Message-Id the header is exactly the same on all copies.Not sure if it is related to the upgrade, but we had a similar issue. In general, if Message-id is different, it means it is a different message, not the same message duplicated/copied. In our case, it was some antivirus plugin on client machine trying to move message into Junk/Spam folder, but changing it in the process, which caused it detected the moved message as a new one and a loop started. You can try to check your mailserver (SMTP) logs if you will find the different message ids there.
the messages do NOT have any SMTP related headers so I am quite sure they are put on the server via IMAP. As it "worked before" the upgrade for ages something must be wrong / have changedwith the migration but as said I am out of ideas...
Oliver
-- Protect your environment - close windows and adopt a penguin!
Hi Marek
On 03.10.25 14:05, Marek Podmaka via dovecot wrote:
On Fri, 3 Oct 2025 at 13:55, Oliver Welter via dovecot [1]<dovecot@dovecot.org> wrote:
There is now a single mailbox where I have an issue with duplicated mails
- the "Sent" folder had approx 400 mails upon migration, it now has 4500
mails with each mail duplicated several times. Below are the header lines
of one message, besides the Message-Id the header is exactly the same on
all copies.Not sure if it is related to the upgrade, but we had a similar issue. In general, if Message-id is different, it means it is a different message, not the same message duplicated/copied. In our case, it was some antivirus plugin on client machine trying to move message into Junk/Spam folder, but changing it in the process, which caused it detected the moved message as a new one and a loop started. You can try to check your mailserver (SMTP) logs if you will find the different message ids there.
the messages do NOT have any SMTP related headers so I am quite sure they are put on the server via IMAP. As it "worked before" the upgrade for ages something must be wrong / have changed with the migration but as said I am out of ideas...
Oliver
-- Protect your environment - close windows and adopt a penguin!
References
Visible links
- mailto:dovecot@dovecot.org
On 04/10/2025 00:49, Oliver Welter via dovecot wrote:
Hi All, I have migrated my mailserver (from dovecot 2.2 on debian 10 to dovecot 2.3 provided by the mailcow-dockerized project) by coping over the mailstorage directory "as is". There is now a single mailbox where I have an issue with duplicated mails - the "Sent" folder had approx 400 mails upon migration, it now has 4500 mails with each mail duplicated several times. Below are the header lines of one message, besides the Message-Id the header is exactly the same on all copies. I tried "doveadm force-resync" on the mailbox and the mailbox was removed and recreated from scratch on the mail app but we still get additional copies added - anybody has an idea what is going on here and how to fix this? Oliver
Hi Oliver.
Just wanted to let you know that I received 2 copies of your email above. They had different dates and message IDs, and the multipart boundary IDs were different (which is probably due to the way mailman forwards the message) and a few other minor things (also related to mailman?) but the content was otherwise the same.
In case it helps the dates and message IDs were:
Message-ID:<4f2d0415-a239-4af7-b636-09c47bc510b1@oliwel.de> Date: Fri, 3 Oct 2025 13:21:30 +0200
and
Message-ID:<f0f3e00c-ff56-435b-b861-c5c555d2c6ee@oliwel.de> Date: Fri, 3 Oct 2025 13:49:55 +0200
It would be interesting to know what your mail logs show when you sent your email?
Nick.
On 04/10/2025 00:49, Oliver Welter via dovecot wrote:
Hi All,
I have migrated my mailserver (from dovecot 2.2 on debian 10 to dovecot
2.3 provided by the mailcow-dockerized project) by coping over the
mailstorage directory "as is".
There is now a single mailbox where I have an issue with duplicated mails
- the "Sent" folder had approx 400 mails upon migration, it now has 4500
mails with each mail duplicated several times. Below are the header lines
of one message, besides the Message-Id the header is exactly the same on
all copies.
I tried "doveadm force-resync" on the mailbox and the mailbox was removed
and recreated from scratch on the mail app but we still get additional
copies added - anybody has an idea what is going on here and how to fix
this?
OliverHi Oliver.
Just wanted to let you know that I received 2 copies of your email above. They had different dates and message IDs, and the multipart boundary IDs were different (which is probably due to the way mailman forwards the message) and a few other minor things (also related to mailman?) but the content was otherwise the same.
In case it helps the dates and message IDs were:
Message-ID: [1]<4f2d0415-a239-4af7-b636-09c47bc510b1@oliwel.de> Date: Fri, 3 Oct 2025 13:21:30 +0200
and
Message-ID: [2]<f0f3e00c-ff56-435b-b861-c5c555d2c6ee@oliwel.de> Date: Fri, 3 Oct 2025 13:49:55 +0200
It would be interesting to know what your mail logs show when you sent your email?
Nick.
References
Visible links
- mailto:4f2d0415-a239-4af7-b636-09c47bc510b1@oliwel.de
- mailto:f0f3e00c-ff56-435b-b861-c5c555d2c6ee@oliwel.de
Hi Nick,
the Double Post was "intenional" - I was not subscribed with this emailadress so the first mail went into "moderator approval" and I resent it after subscribing myself to the list....the mods were even faster so it was doubled :)
Oliver
On 03.10.25 22:58, Nick Tait via dovecot wrote:
On 04/10/2025 00:49, Oliver Welter via dovecot wrote: Hi All, I have migrated my mailserver (from dovecot 2.2 on debian 10 to dovecot 2.3 provided by the mailcow-dockerized project) by coping over the mailstorage directory "as is". There is now a single mailbox where I have an issue with duplicated mails - the "Sent" folder had approx 400 mails upon migration, it now has 4500 mails with each mail duplicated several times. Below are the header lines of one message, besides the Message-Id the header is exactly the same on all copies. I tried "doveadm force-resync" on the mailbox and the mailbox was removed and recreated from scratch on the mail app but we still get additional copies added - anybody has an idea what is going on here and how to fix this? Oliver Hi Oliver. Just wanted to let you know that I received 2 copies of your email above. They had different dates and message IDs, and the multipart boundary IDs were different (which is probably due to the way mailman forwards the message) and a few other minor things (also related to mailman?) but the content was otherwise the same. In case it helps the dates and message IDs were:Message-ID: [1]<4f2d0415-a239-4af7-b636-09c47bc510b1@oliwel.de> Date: Fri, 3 Oct 2025 13:21:30 +0200
andMessage-ID: [2]<f0f3e00c-ff56-435b-b861-c5c555d2c6ee@oliwel.de> Date: Fri, 3 Oct 2025 13:49:55 +0200
It would be interesting to know what your mail logs show when you sent your email? Nick.References
Visible links 1. mailto:4f2d0415-a239-4af7-b636-09c47bc510b1@oliwel.de 2. mailto:f0f3e00c-ff56-435b-b861-c5c555d2c6ee@oliwel.de
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
-- Protect your environment - close windows and adopt a penguin!
participants (3)
-
Marek Podmaka
-
Nick Tait
-
Oliver Welter