dsync between 2 instances on same machine (but different versions and different DNS hostnames) fails with "doveadm server sent invalid handshake"
I'm trying to use dsync to upgrade from an existing 2.3 server running native on MacOS to a 2.4.1 server running inside a Docker container on the same machine, using the official Dovecot 2.4.1 Docker image from DockerHub.
For testing, I decided to set up a new hostname (CNAME) record in DNS pointing to the same IP, since both Dovecot instances are running on the same host.
The new 2.4.1 Docker-containerized instance is working fine so far, which is to say I can connect to it from a mail client (the Mac Mail app) using TLS/SSL, and I'm able to move mail messages in, delete them, create folders, etc.
For the next phase of testing, I want to migrate a single folder of a single user's email from the 2.3 instance to the new 2.4.1 Docker instance. I recently found out about dsync and thought I'd give it a try.
The test Dockerized instance exposes its IMAPS port to a different port than the legacy instance (of course).
I found that dsync seems to behave differently than the IMAP service wrt the certficate chain: Whereas the Mail client was able to negotiate the encrypted connection with the LetsEncrypt server cert alone in the dovecot cert file, with dsync, the 2.3 server seemed to require the entire cert chain to reside in the file. (?)
After resolving that, the source dsync could at least get past the certificate validation phase, but then complained thusly:
Oct 19 21:52:11 doveadm(myuser@mydomain.net): Error: doveadm server sent invalid handshake: * OK [CAPABILITY IMAP4rev1 LOGIN-REFERRALS ID ENABLE IDLE SASL-IR LITERAL+ AUTH=PLAIN AUTH=LOGIN] Dovecot ready. Oct 19 21:52:11 doveadm(myuser@mydomain.net): Error: Disconnected from remote: EOF Oct 19 21:52:11 doveadm(myuser@mydomain.net): Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb (uid=0): Disconnected: Connection closed (fd=8)
I can't tell from that error message what the source dsync doesn't like about the destination dsync, or for that matter, whether it might be the destination dsync that doesn't like what the source dsync is saying??
Anyway, I'm invoking dsync like this, to sync just a single folder:
$ sudo doveadm -D -v backup -u myuser@mydomain.net -m FolderName tcps:testserver.mydomain.net:31993
ALSO: I confess that I also don't understand how dsync can work without asking for user account passwords, if it's indeed running over the IMAPS service. How can the destination service trust the incoming dsync connection? I assume it's running over a normal IMAPS connection, since that's the only port that I've given to the source dsync command, but maybe that's not the case?
The docs are pretty unclear about this aspect, but I'm no expert, so maybe this is just obvious to others. (?)
On 21/10/2025 04:29 EEST rmfuhrer--- via dovecot <dovecot@dovecot.org> wrote:
I'm trying to use dsync to upgrade from an existing 2.3 server running native on MacOS to a 2.4.1 server running inside a Docker container on the same machine, using the official Dovecot 2.4.1 Docker image from DockerHub.
For testing, I decided to set up a new hostname (CNAME) record in DNS pointing to the same IP, since both Dovecot instances are running on the same host.
The new 2.4.1 Docker-containerized instance is working fine so far, which is to say I can connect to it from a mail client (the Mac Mail app) using TLS/SSL, and I'm able to move mail messages in, delete them, create folders, etc.
For the next phase of testing, I want to migrate a single folder of a single user's email from the 2.3 instance to the new 2.4.1 Docker instance. I recently found out about dsync and thought I'd give it a try.
The test Dockerized instance exposes its IMAPS port to a different port than the legacy instance (of course).
I found that dsync seems to behave differently than the IMAP service wrt the certficate chain: Whereas the Mail client was able to negotiate the encrypted connection with the LetsEncrypt server cert alone in the dovecot cert file, with dsync, the 2.3 server seemed to require the entire cert chain to reside in the file. (?)
After resolving that, the source dsync could at least get past the certificate validation phase, but then complained thusly:
Oct 19 21:52:11 doveadm(myuser@mydomain.net): Error: doveadm server sent invalid handshake: * OK [CAPABILITY IMAP4rev1 LOGIN-REFERRALS ID ENABLE IDLE SASL-IR LITERAL+ AUTH=PLAIN AUTH=LOGIN] Dovecot ready. Oct 19 21:52:11 doveadm(myuser@mydomain.net): Error: Disconnected from remote: EOF Oct 19 21:52:11 doveadm(myuser@mydomain.net): Debug: auth-master: conn unix:/usr/local/var/run/dovecot/auth-userdb (uid=0): Disconnected: Connection closed (fd=8)
I can't tell from that error message what the source dsync doesn't like about the destination dsync, or for that matter, whether it might be the destination dsync that doesn't like what the source dsync is saying??
Anyway, I'm invoking dsync like this, to sync just a single folder:
$ sudo doveadm -D -v backup -u myuser@mydomain.net -m FolderName tcps:testserver.mydomain.net:31993
ALSO: I confess that I also don't understand how dsync can work without asking for user account passwords, if it's indeed running over the IMAPS service. How can the destination service trust the incoming dsync connection? I assume it's running over a normal IMAPS connection, since that's the only port that I've given to the source dsync command, but maybe that's not the case?
The docs are pretty unclear about this aspect, but I'm no expert, so maybe this is just obvious to others. (?)
Hi!
Please look at migration guide at https://doc.dovecot.org/2.4.1/core/admin/migration.html
Aki
Thanks!
Obviously the documentation accessible via "doveadm help sync" is sorely incomplete, and in particular doesn't even mention this migration guide, which would probably have answered most/all of my questions.
Perhaps it makes sense to file a doc bug to that effect?
Oh, also, the migration guide recommends a config stanza like this:
service doveadm { inet_listener { port = 12354 } }
but then 2.4.1 complains that "inet_listener { } is missing section name" .
You can use e.g. inet_listener doveadm {
Aki
On 23/10/2025 20:48 EEST rmfuhrer--- via dovecot
<[1]dovecot@dovecot.org> wrote:
Oh, also, the migration guide recommends a config stanza like this:
service doveadm {
inet_listener {
port = 12354
}
}
but then 2.4.1 complains that "inet_listener { } is missing section
name" .
_______________________________________________
dovecot mailing list -- [2]dovecot@dovecot.org
To unsubscribe send an email to [3]dovecot-leave@dovecot.orgReferences
Visible links
- mailto:dovecot@dovecot.org
- mailto:dovecot@dovecot.org
- mailto:dovecot-leave@dovecot.org
That's right, but my (subtle) point was that that part of the migration guide should be fixed.
I changed the doveadm config per the migration guide:
service doveadm { inet_listener http { port = 12354 } } doveadm_password = redacted
Of course, I also added the doveadm service port to the exposed ports on the Docker container, and added the doveadm_password to that dovecot's config.
I then ran the "doveadm backup" command on the host, where the old dovecot instance is running, connecting to the doveadm service in the Docker container:
$ sudo doveadm -D -v backup -u myuser@mydomain.net -m MyFolder tcp:127.0.0.1:12354
which produces the following error:
Oct 23 21:27:58 doveadm(myuser@mydomain.net): Error: doveadm server sent invalid handshake: HTTP/1.1 400 Bad Request Oct 23 21:27:58 doveadm(myuser@mydomain.net): Error: Disconnected from remote: EOF
Meanwhile, the logs from the dovecot instance running in the Docker container show this error:
Oct 24 01:27:58 doveadm(192.168.65.1): Info: http-server: conn 192.168.65.1:26654 [1]: Client sent invalid request: Unexpected character <0x09> in request method
Any ideas as to what's wrong?
On 24/10/2025 04:34 EEST rmfuhrer--- via dovecot <dovecot@dovecot.org> wrote:
I changed the doveadm config per the migration guide:
service doveadm { inet_listener http { port = 12354 } } doveadm_password = redacted
No where in the migration guide does it suggest using http there. I suggested
inet_listener doveadm {
Using http will indeed make it talk HTTP, so don't use http there.
Aki
True, the migration guide recommendation has no “section name” in its snippet, which is an syntax error.
The only section I found in the docs that describes inet_listener says this:
inet_listeners name Section name of this listener. It is meant to be descriptive for humans (e.g. imap, imaps).
That sounds like the “descriptive name” could be anything I choose, but in fact the name actually has to be one of a specific set of values, and sadly, that set is never enumerated anywhere. In this particular context (the doveadm service), it has to be the specific value you used, or the service won’t work.
IMHO, yet another doc bug.
Anyway, thanks, I’ll give that a try.
For posterity, changing the service specification to use "inet_listener doveadm" worked.
I'm happily synching now. :-)
I really do think the docs could use a *lot* of improvement, both in general and for this topic in particular.
I'd be happy to file some doc bugs, if such things get any attention.
I'd also be happy to work up some suggested doc fixes, but I'd need help with some of the technical details, and it doesn't sound like anyone knowledgeable enough cares enough to give me the time. :-(
LMK if someone out there is willing to work a bit with me to make the docs better...
On 10/24/2025 5:59 PM MDT rmfuhrer--- via dovecot <dovecot@dovecot.org> wrote:
I really do think the docs could use a *lot* of improvement, both in general and for this topic in particular.
I'd be happy to file some doc bugs, if such things get any attention.
Documentation contribution guide (not merged to site yet); hopefully it is useful for this or other future submissions:
https://github.com/dovecot/documentation/pull/1321
I'd also be happy to work up some suggested doc fixes, but I'd need help with some of the technical details, and it doesn't sound like anyone knowledgeable enough cares enough to give me the time. :-(
A quick review of this thread indicates that you were/are asking for support for behavior that is not supported in CE: syncing between multiple Dovecot nodes is out-of-scope for Dovecot CE, as it is intended for single node use only.
We welcome all suggestions on how to improve core features (e.g. we know that the new configuration syntax may have some documentation holes and limitations that can be improved). However, if your frustration stems from a lack of information on something like using dsync between different nodes, this is intentional.
dsync is maintained for use in one-time import into Dovecot or for backup to a static location. Our documentation is designed to explain only what the software supports.
michael
Actually, I'm trying to do a one-shot migration from a legacy single-node Dovecot v2.3 instance to a new Docker container-hosted Dovecot 2.4 instance.
participants (3)
-
Aki Tuomi
-
Michael Slusarz
-
rmfuhrer＠optonline.net