newbie dsync problems
I have a very basic postfix/dovecot server, with just a few users, and it works. But in the past I have had hardware failures…so I would like use dsync to a nearly matching server for some extra safety, and I have not yet been successful.
I am trying to do this as root over ssh, with a private key for authentication.
I do not managed to get the whole replicator thing working, so as part of simplifying things I am trying to run dsync manually, just on one user (myself). I hope this isn't a silly approach.
What would the manual command look like? The closest example on the man page is:
$ doveadm sync -u username@example.com ssh -i id_dsa.dovecot
mailuser@example.com doveadm dsync-server -u username@example.com
I try to change that to match my case and it has never heard of me:
root@la:/etc/dovecot# doveadm sync -1 -u kentborg@mail.borg.org ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync kentborg@mail.borg.org doveadm dsync-server -u kentborg@mail.borg.org doveadm(kentborg@mail.borg.org): Error: User doesn't exist root@la:/etc/dovecot#
I drop the machine part of the name and it knows who I am on this end but can't get to the other end:
root@la:/etc/dovecot# doveadm sync -1 -u kentborg ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync kentborg@mail.borg.org doveadm dsync-server -u kent
borg@mail.borg.org kentborg@mail.borg.org's password:
I think I have root's ssh keys set up correctly, I can run this:
root@la:/etc/dovecot# ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org PTY allocation request failed on channel 0 C-c C-croot@la:/etc/dovecot# …and on the remote end I see some debugging output I put in the remote script, outputting an empty username. Makes sense.
If I try to put that in this (incomplete) dsync command:
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org Error: Extraneous arguments found: 3 5 doveadm(kentborg)<1029674><QvOnDPmKkmcqtg8AA8+vgQ>: Error: read(remote) failed: EOF (version not received) doveadm(kentborg)<1029674><QvOnDPmKkmcqtg8AA8+vgQ>: Error: Remote command returned error 64: ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg
.org dsync-server root@la:/etc/dovecot# And the remote end outputs the username as "VERSION dsync 3 5", okay, maybe progress.
But when I try to make the command more complete and send a username to the remote end, and now I am no longer talking to the remote end:
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix@mail.borg.org remoteprefix@mail.borg.org's password:
Suggestions?
Thanks,
-kb
I had a typo (I said I'm a newbie).
On 1/23/25 10:50 AM, Kent Borg via dovecot wrote:
But when I try to make the command more complete and send a username to the remote end, and now I am no longer talking to the remote end:
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix@mail.borg.org remoteprefix@mail.borg.org's password:
This better version also doesn't work:
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix:root@mail.borg.org remoteprefix:root@mail.borg.org's password
Sorry for the error,
-kb//
Kent,
You are being prompted for a password, so it isn't using private key authentication. I recommend you get ssh working first, prove you are indeed connecting to your secondary server, and only then introduce doveadm.
Is mail.borg.org the name of your "matching server" or is that the name of your primary server? If so, it looks to me like you are using ssh to connect back to yourself. You should have something like mail.borg.org as primary, mail2.borg.org as backup, and if necessary, add mail2.borg.org to your hosts file if there is no DNS for it. Or even connect via IP address like this:
ssh root@172.16.20.11
I'll leave the discussion on whether to use root in this fashion even makes sense to others. Suffice to say, once you get something working perhaps consider removing the private key and use a non-root user.
Doug
-----Original Message----- From: Kent Borg via dovecot <dovecot@dovecot.org> Sent: Thursday, January 23, 2025 2:12 PM To: dovecot@dovecot.org Subject: Re: newbie dsync problems
I had a typo (I said I'm a newbie).
On 1/23/25 10:50 AM, Kent Borg via dovecot wrote:
But when I try to make the command more complete and send a username to the remote end, and now I am no longer talking to the remote end:
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix@mail.borg.org remoteprefix@mail.borg.org's password:
This better version also doesn't work:
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix:root@mail.borg.org remoteprefix:root@mail.borg.org's password
Sorry for the error,
-kb//
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
On 1/23/25 12:26 PM, cdm567@yahoo.com wrote:
Kent,
You are being prompted for a password, so it isn't using private key authentication. I recommend you get ssh working first, prove you are indeed connecting to your secondary server, and only then introduce doveadm.
Yes, I checked that:
I think I have root's ssh keys set up correctly, I can run this:
root@la:/etc/dovecot# ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org PTY allocation request failed on channel 0 C-c C-croot@la:/etc/dovecot# …and on the remote end I see some debugging output I put in the remote script, outputting an empty username. Makes sense.
Is mail.borg.org the name of your "matching server" or is that the name of your primary server?
mail.borg.org is the name of the (priority 10) backup, I am running this on my (priority 1) primary server, mail2.borg.org, I am pretty certain I am not ssh-ing to myself.
kb
In your working example you are connecting as root but in your dsync example your user is remoteprefix:root. Try removing the "remoteprefix:" which is being treated as part of the user name.
-----Original Message----- From: Kent Borg via dovecot <dovecot@dovecot.org> Sent: Thursday, January 23, 2025 3:34 PM To: cdm567@yahoo.com; 'Kent Borg' <kentborg-dovecot@borg.org>; dovecot@dovecot.org Subject: Re: newbie dsync problems
On 1/23/25 12:26 PM, cdm567@yahoo.com wrote:
Kent,
You are being prompted for a password, so it isn't using private key authentication. I recommend you get ssh working first, prove you are indeed connecting to your secondary server, and only then introduce doveadm.
Yes, I checked that:
I think I have root's ssh keys set up correctly, I can run this:
root@la:/etc/dovecot# ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org PTY allocation request failed on channel 0 C-c C-croot@la:/etc/dovecot# …and on the remote end I see some debugging output I put in the remote script, outputting an empty username. Makes sense.
Is mail.borg.org the name of your "matching server" or is that the name of your primary server?
mail.borg.org is the name of the (priority 10) backup, I am running this on my (priority 1) primary server, mail2.borg.org, I am pretty certain I am not ssh-ing to myself.
kb
dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-leave@dovecot.org
On 1/23/25 1:41 PM, cdm567@yahoo.com wrote:
In your working example you are connecting as root but in your dsync example your user is remoteprefix:root. Try removing the "remoteprefix:" which is being treated as part of the user name.
If I take off the "remoteprefix" it logs in, but it doesn't send the user to the other end, the wrapper script on mail.borg.org gets "VERSION dsync 3 5" as the parameter.
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync root@mail.borg.org Error: Extraneous arguments found: 3 5 doveadm(kentborg)<1052944><mEMeJ8i9kmcQERAAA8+vgQ>: Error: read(remote) failed: EOF (version not received) doveadm(kentborg)<1052944><mEMeJ8i9kmcQERAAA8+vgQ>: Error: Remote command returned error 64: ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync root@mail.borg.org dsync-server root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix:root@mail.borg.org remoteprefix:root@mail.borg.org's password:
According to the man page, that should be the destination:
ARGUMENTS destination This argument specifies the synchronized destination. It can be one of:
location Same as mail_location setting, e.g. maildir:~/Maildir
remote:login@host Uses dsync_remote_cmd setting to connect to the remote host (usually via ssh)
remoteprefix:login@host This is the same as remote, except "user@domain\n" is sent before dsync protocol starts. This allows imple‐ menting a trusted wrapper script that runs doveadm dsync-server by reading the username from the first line.
tcp:host[:port] Connects to remote doveadm server via TCP. The default port is specified by doveadm_port setting.
tcps:host[:port] This is the same as tcp, but with SSL.
command [arg1 [, arg2, ...]] Runs a local command that connects its standard input & output to a dsync server.
One of the examples on the man page is:
doveadm sync -u username@example.com ssh -i id_dsa.dovecot
mailuser@example.com doveadm dsync-server -u username@example.com
Which I don't understand. What is "mailuser@example.com"? What are the two parameters and the option after that?
Their simpler example:
doveadm sync -u username@example.com remote:server-replica.example.com Makes much more sense, but I can't find anything based on that example works.
Does "doveadm sync" maybe not work in version 2.3.19.1?
root@la:/etc/dovecot# dovecot --version 2.3.19.1 (9b53102964)
Thanks,
-kb, the Kent who would love to see some working "doveadm sync" examples.
I don't have a working example because I do my dsync backups on the local machine with output to shared NFS storage that is accessible to both my primary and backup systems. No ssh required or remote connection required. That is provided by NFS.
This excerpt of my backup script runs dsync in a loop where 'USERS' is populated with email account names to be backed up. The backup target location is on separate storage. If you can't figure out the doveadm sync to another server you could NFS share a file system from your secondary system to your primary and do something similar.
for user in ${USERS}; do dsync -u ${user} backup maildir:/home/${user}/backup/mailboxes done
-----Original Message----- From: Kent Borg <kentborg-dovecot@borg.org> Sent: Thursday, January 23, 2025 5:22 PM To: cdm567@yahoo.com; dovecot@dovecot.org Subject: Re: newbie dsync problems
On 1/23/25 1:41 PM, cdm567@yahoo.com wrote:
In your working example you are connecting as root but in your dsync example your user is remoteprefix:root. Try removing the "remoteprefix:" which is being treated as part of the user name.
If I take off the "remoteprefix" it logs in, but it doesn't send the user to the other end, the wrapper script on mail.borg.org gets "VERSION dsync 3 5" as the parameter.
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync root@mail.borg.org Error: Extraneous arguments found: 3 5 doveadm(kentborg)<1052944><mEMeJ8i9kmcQERAAA8+vgQ>: Error: read(remote) failed: EOF (version not received) doveadm(kentborg)<1052944><mEMeJ8i9kmcQERAAA8+vgQ>: Error: Remote command returned error 64: ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync root@mail.borg.org dsync-server root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix:root@mail.borg.org remoteprefix:root@mail.borg.org's password:
According to the man page, that should be the destination:
ARGUMENTS destination This argument specifies the synchronized destination. It can be one of:
location Same as mail_location setting, e.g. maildir:~/Maildir remote:login@host Uses dsync_remote_cmd setting to connect to theremote host (usually via ssh)
remoteprefix:login@host This is the same as remote, except"user@domain\n" is sent before dsync protocol starts. This allows imple‐ menting a trusted wrapper script that runs doveadm dsync-server by reading the username from the first line.
tcp:host[:port] Connects to remote doveadm server via TCP. Thedefault port is specified by doveadm_port setting.
tcps:host[:port] This is the same as tcp, but with SSL. command [arg1 [, arg2, ...]] Runs a local command that connects its standardinput & output to a dsync server.
One of the examples on the man page is:
doveadm sync -u username@example.com ssh -i id_dsa.dovecot
mailuser@example.com doveadm dsync-server -u username@example.comWhich I don't understand. What is "mailuser@example.com"? What are the two parameters and the option after that?
Their simpler example:
doveadm sync -u username@example.com remote:server- replica.example.com Makes much more sense, but I can't find anything based on that example works.
Does "doveadm sync" maybe not work in version 2.3.19.1?
root@la:/etc/dovecot# dovecot --version 2.3.19.1 (9b53102964)
Thanks,
-kb, the Kent who would love to see some working "doveadm sync" examples.
On 1/23/25 2:44 PM, Doug via dovecot wrote:
I don't have a working example because I do my dsync backups on the local machine with output to shared NFS storage that is accessible to both my primary and backup systems. No ssh required or remote connection required. That is provided by NFS.
Ah, NFS. But I don't think that works so well in my case, my two servers' homes are a very long way apart.
-kb
So trying to debug replication via getting manual "doveadmin sync" hasn't gotten me there,
Any suggests on how to debug the replication approach? Would it be helpful to post the "/etc/dovecot/conf.d/backup.conf.ext" I created?
Thanks,
-kb
On 1/23/25 10:50 AM, Kent Borg wrote:
I have a very basic postfix/dovecot server, with just a few users, and it works. But in the past I have had hardware failures…so I would like use dsync to a nearly matching server for some extra safety, and I have not yet been successful.
I am trying to do this as root over ssh, with a private key for authentication.
I do not managed to get the whole replicator thing working, so as part of simplifying things I am trying to run dsync manually, just on one user (myself). I hope this isn't a silly approach.
What would the manual command look like? The closest example on the man page is:
$ doveadm sync -u username@example.com ssh -i id_dsa.dovecot
mailuser@example.com doveadm dsync-server -u username@example.comI try to change that to match my case and it has never heard of me:
root@la:/etc/dovecot# doveadm sync -1 -u kentborg@mail.borg.org ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync kentborg@mail.borg.org doveadm dsync-server -u kentborg@mail.borg.org doveadm(kentborg@mail.borg.org): Error: User doesn't exist root@la:/etc/dovecot#
I drop the machine part of the name and it knows who I am on this end but can't get to the other end:
root@la:/etc/dovecot# doveadm sync -1 -u kentborg ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync kentborg@mail.borg.org doveadm dsync-server -u kent
borg@mail.borg.org kentborg@mail.borg.org's password:I think I have root's ssh keys set up correctly, I can run this:
root@la:/etc/dovecot# ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org PTY allocation request failed on channel 0 C-c C-croot@la:/etc/dovecot# …and on the remote end I see some debugging output I put in the remote script, outputting an empty username. Makes sense.
If I try to put that in this (incomplete) dsync command:
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org Error: Extraneous arguments found: 3 5 doveadm(kentborg)<1029674><QvOnDPmKkmcqtg8AA8+vgQ>: Error: read(remote) failed: EOF (version not received) doveadm(kentborg)<1029674><QvOnDPmKkmcqtg8AA8+vgQ>: Error: Remote command returned error 64: ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg
.org dsync-server root@la:/etc/dovecot# And the remote end outputs the username as "VERSION dsync 3 5", okay, maybe progress.But when I try to make the command more complete and send a username to the remote end, and now I am no longer talking to the remote end:
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix@mail.borg.org remoteprefix@mail.borg.org's password:
Suggestions?
Thanks,
-kb
participants (2)
-
cdm567＠yahoo.com
-
Kent Borg