Announcement: wormhole plugin, replication for Dovecot 2.4
Hello List,
I am proud to announce my project "wormhole". This is an extension for Dovecot 2.4.1. It allows to continue the use of the replication feature of Dovecot that was removed with the 2.4 release.
History: I started this project by forking the replication code of Dovecot 2.3.21.1 and porting it to 2.4. The first result was a patch for Dovecot 2.4.1. While working with the code, I realized that it is possible to provide the feature as a plugin / extension. In the first release of the plugin, I tried to integrate it with the existing dsync commands for doveadm. This required an API for access to the dsync commands, which I proposed to the Dovecot developers in https://github.com/dovecot/core/pull/249 . Unfortunately, my pull request was rejected, though I can understand the reasons for that. Without the required API, I had to switch to another approach: I forked the dsync code from doveadm to provide a new set of commands: "repl" and "repl-server". These commands work similar to "sync" and "dsync-server" but use additional functions required by the replication feature. This approach was released as version 0.3 of the plugin. With version 0.4 I renamed the project to wormhole. Before that it was just called dovecot-replication (see https://codeberg.org/errror/dovecot-replication). I thought that the project should have a more unique name. Currently I'm working on designing a test suite for wormhole. Doing unit tests of the components is not sufficient IMHO. The test suite should check the whole replication feature using two running dovecot instances that replicate the users' mailboxes. Having a sophisticated test suite will help in future development and bug tracking.
The code can be found on my codeberg site at: https://codeberg.org/errror/wormhole
There is also a web page on my personal homepage at: https://people.mpi-klsb.mpg.de/~pcernko/wormhole.shtml
The project is still missing documentation. For the moment, configuration is similar to the existing Dovecot 2.3 documentation.
Upgrading from 2.3: When using the included patch for Dovecot, the plugin should be 100% compatible with existing Dovecot 2.3 servers. This compatibility should allow uninterrupted upgrades of the service by replacing Dovecot on one server and then the other one. After the upgrade, the plugin can be reconfigured to work without the patch for Dovecot. Upgrading without the patch is also possible. During the upgrade (aka "while one server still runs 2.3") some aspects of the replication will not work or lack performance. Upgrading is not yet tested very well and missing documentation also. I will work on that soon.
I hope to get some feedback, pull requests or bug reports via mail or at codeberg (preferred).
Patrick Cernko <pcernko@mpi-klsb.mpg.de> +49 681 9325 5815 Joint Scientific IT and Technical Service Max-Planck-Institute für Informatik & Softwaresysteme
On Tue, Oct 7, 2025 at 12:45 PM Patrick Cernko via dovecot < dovecot@dovecot.org> wrote:
Hello List,
I am proud to announce my project "wormhole". This is an extension for Dovecot 2.4.1. It allows to continue the use of the replication feature of Dovecot that was removed with the 2.4 release.
History: I started this project by forking the replication code of Dovecot 2.3.21.1 and porting it to 2.4. The first result was a patch for Dovecot 2.4.1. While working with the code, I realized that it is possible to provide the feature as a plugin / extension. In the first release of the plugin, I tried to integrate it with the existing dsync commands for doveadm. This required an API for access to the dsync commands, which I proposed to the Dovecot developers in https://github.com/dovecot/core/pull/249 . Unfortunately, my pull request was rejected, though I can understand the reasons for that. Without the required API, I had to switch to another approach: I forked the dsync code from doveadm to provide a new set of commands: "repl" and "repl-server". These commands work similar to "sync" and "dsync-server" but use additional functions required by the replication feature. This approach was released as version 0.3 of the plugin. With version 0.4 I renamed the project to wormhole. Before that it was just called dovecot-replication (see https://codeberg.org/errror/dovecot-replication). I thought that the project should have a more unique name. Currently I'm working on designing a test suite for wormhole. Doing unit tests of the components is not sufficient IMHO. The test suite should check the whole replication feature using two running dovecot instances that replicate the users' mailboxes. Having a sophisticated test suite will help in future development and bug tracking.
The code can be found on my codeberg site at: https://codeberg.org/errror/wormhole
There is also a web page on my personal homepage at: https://people.mpi-klsb.mpg.de/~pcernko/wormhole.shtml
The project is still missing documentation. For the moment, configuration is similar to the existing Dovecot 2.3 documentation.
Upgrading from 2.3: When using the included patch for Dovecot, the plugin should be 100% compatible with existing Dovecot 2.3 servers. This compatibility should allow uninterrupted upgrades of the service by replacing Dovecot on one server and then the other one. After the upgrade, the plugin can be reconfigured to work without the patch for Dovecot. Upgrading without the patch is also possible. During the upgrade (aka "while one server still runs 2.3") some aspects of the replication will not work or lack performance. Upgrading is not yet tested very well and missing documentation also. I will work on that soon.
I hope to get some feedback, pull requests or bug reports via mail or at codeberg (preferred).
Patrick Cernko <pcernko@mpi-klsb.mpg.de> +49 681 9325 5815 Joint Scientific IT and Technical Service Max-Planck-Institute für Informatik & Softwaresysteme
While I have never needed replication, this is a very welcome development and I believe the community will greatly benefit from it. Some people who were stuck on 2.3.x due to the removal of the replication feature will find this a welcome move. I thank you on their behalf :)
-- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' ¯\_(ツ)_/¯ :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html]
On Tue, Oct 7, 2025 at 12:45PM Patrick Cernko via dovecot <[1]dovecot@dovecot.org> wrote:
Hello List,
I am proud to announce my project "wormhole". This is an extension for
Dovecot 2.4.1.
It allows to continue the use of the replication feature of Dovecot that
was removed
with the 2.4 release.
History:
I started this project by forking the replication code of Dovecot
2.3.21.1 and
porting it to 2.4. The first result was a patch for Dovecot 2.4.1.
While working with the code, I realized that it is possible to provide
the feature as
a plugin / extension. In the first release of the plugin, I tried to
integrate it
with the existing dsync commands for doveadm. This required an API for
access to the
dsync commands, which I proposed to the Dovecot developers in
[2]https://github.com/dovecot/core/pull/249 . Unfortunately, my pull
request was
rejected, though I can understand the reasons for that.
Without the required API, I had to switch to another approach: I forked
the dsync
code from doveadm to provide a new set of commands: "repl" and
"repl-server". These
commands work similar to "sync" and "dsync-server" but use additional
functions
required by the replication feature. This approach was released as
version 0.3 of the
plugin.
With version 0.4 I renamed the project to wormhole. Before that it was
just called
dovecot-replication (see
[3]https://codeberg.org/errror/dovecot-replication). I thought
that the project should have a more unique name.
Currently I'm working on designing a test suite for wormhole. Doing unit
tests of the
components is not sufficient IMHO. The test suite should check the whole
replication
feature using two running dovecot instances that replicate the users'
mailboxes.
Having a sophisticated test suite will help in future development and
bug tracking.
The code can be found on my codeberg site at:
[4]https://codeberg.org/errror/wormhole
There is also a web page on my personal homepage at:
[5]https://people.mpi-klsb.mpg.de/~pcernko/wormhole.shtml
The project is still missing documentation. For the moment,
configuration is similar
to the existing Dovecot 2.3 documentation.
Upgrading from 2.3:
When using the included patch for Dovecot, the plugin should be 100%
compatible with
existing Dovecot 2.3 servers. This compatibility should allow
uninterrupted upgrades
of the service by replacing Dovecot on one server and then the other
one. After the
upgrade, the plugin can be reconfigured to work without the patch for
Dovecot.
Upgrading without the patch is also possible. During the upgrade (aka
"while one
server still runs 2.3") some aspects of the replication will not work or
lack
performance.
Upgrading is not yet tested very well and missing documentation also. I
will work on
that soon.
I hope to get some feedback, pull requests or bug reports via mail or at
codeberg
(preferred).
--
Patrick Cernko <[6]pcernko@mpi-klsb.mpg.de> +49 681 9325 5815
Joint Scientific IT and Technical Service
Max-Planck-Institute fuer Informatik & SoftwaresystemeWhile I have never needed replication, this is a very welcome development and I believe the community will greatly benefit from it. Some people who were stuck on 2.3.x due to the removal of the replication feature will find this a welcome move. I thank you on their behalf :)
Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' -\_(Tu)_/- :-) [How to ask smart questions: [7]http://www.catb.org/~esr/faqs/smart-questions.html]
References
Visible links
- mailto:dovecot@dovecot.org
- https://github.com/dovecot/core/pull/249
- https://codeberg.org/errror/dovecot-replication
- https://codeberg.org/errror/wormhole
- https://people.mpi-klsb.mpg.de/~pcernko/wormhole.shtml
- mailto:pcernko@mpi-klsb.mpg.de
- http://www.catb.org/~esr/faqs/smart-questions.html
Patrick Cernko via dovecot <dovecot@dovecot.org> wrote:
I am proud to announce my project "wormhole". This is an extension for Dovecot 2.4.1. It allows to continue the use of the replication feature of Dovecot that was removed with the 2.4 release.
Thanks, I've been waiting for such an announcement ;-)
Right now, I do have no needs to upgrade to 2.4, but I am sure that time will come sometimes. And it is good to know that there will be solutions to bring back replication to 2.4+.
I am still pissed off by having replication removed from CE, thus: thanks!
Michael
On 7/10/25 11:41, Patrick Cernko via dovecot wrote:
I am proud to announce my project "wormhole". This is an extension for Dovecot 2.4.1. It allows to continue the use of the replication feature of Dovecot that was removed with the 2.4 release.
Hi, Patrick. I am checking dovecot 2.4 and I am very disappointed that internal replication (and SIS) are not supported anymore. That is (almost) a showstopper.
Reading about your wormhole plugin, I wonder why is that better that just using "doveadm sync" launched from a 50 lines python script connecting via IMAP4 and using "notify" imap extension to decide when and what to replicate. That is, a Python script connects to your mailbox using regular IMAP4, it uses "notify" to know about any change in the account, and launches "doveadm sync" when necessary.
I am not questioning your approach, I am just wondering.
Thinking about SIS too. In a company dovecot server, having big attachments duplicated in tons of users is quite frequent. Your example (of recovering an eliminated feature) is inspiring. Thanks.
-- Jesús Cea Avión _/_/ _/_/_/ _/_/_/ jcea@jcea.es - https://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ Twitter: @jcea _/_/ _/_/ _/_/_/_/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
Hi Jesus,
On 2026-06-10 15:43:11, Jesus Cea via dovecot wrote:
Reading about your wormhole plugin, I wonder why is that better that just using "doveadm sync" launched from a 50 lines python script connecting via IMAP4 and using "notify" imap extension to decide when and what to replicate. That is, a Python script connects to your mailbox using regular IMAP4, it uses "notify" to know about any change in the account, and launches "doveadm sync" when necessary.
Using your approach is totally different to wormhole. It requires authenticated IMAP connections for every user on your server. This will consume a lot of resources (compute, mem, network sockets, ...). You also have to have the credentials of every user or you have to enable master user access, which seems not a good idea for a secure setup. Last but not least, the IMAP service must be available for your replication to work. With wormhole you can have replication although you temp. disable IMAP service (e.g. during a maintenance).
I am not questioning your approach, I am just wondering.
Thinking about SIS too. In a company dovecot server, having big attachments duplicated in tons of users is quite frequent. Your example (of recovering an eliminated feature) is inspiring. Thanks.
We have been using SIS for this exact reason (e.g. summer party invite PDF in all users' mailboxes). However managing the integrity of the SIS storage has generated a rather large workload for us admins and we decided to exchange this workload with larger storage requirements, because storage is much cheaper than HR. ;-)
Best,
Patrick Cernko <pcernko@mpi-klsb.mpg.de> +49 681 9325 5815 Joint Scientific IT and Technical Service Max-Planck-Institute für Informatik & Softwaresysteme
participants (4)
-
Jesus Cea
-
Michael Grimm
-
Odhiambo Washington
-
Patrick Cernko