[Dovecot] Failing: doveadm sync <--remote host--> dsync mirror
Hi --
I'm used to dsync my two mailservers running 2.0.16.
Today, I did upgrade one server to 2.1, and doveadm sync is throwing to following error:
mail> doveadm sync -f -u test ssh vmail@remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted.
Same error with old running command:
vmail> dsync -v -f -u test mirror ssh vmail@remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted.
Here's an excerpt of my doveconf -n:
mail> doveconf -n mail_home = /var/mail/homedirs/%n mail_location = mdbox:/var/mail/maildirs/%n passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql }
Reverted back for the time being (production servers) until I do understand the issue. Searching Google with "Fatal: All your namespaces have a location setting." only shows closed bugs back in 2010.
Do I have to upgrade both servers to get syncing running? Any hint, help is highly appreciated.
Thanks, Michael
Hi --
On 18.02.2012, at 19:26, Michael Grimm wrote:
Today, I did upgrade one server to 2.1, and doveadm sync is throwing to following error:
mail> doveadm sync -f -u test ssh vmail@remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted.
[...]
Do I have to upgrade both servers to get syncing running?
JFTR:
- Upgrading both servers ends up in identical error messages.
- After some investigation I came across a hidden upgrade info in [1], but 'doveadm dsync-server' doesn't run either :-(
I'm stuck, thus I did upgrade to 2.0.18 instead ...
Regards, Michael
[1] http://dovecot.org/list/dovecot/2012-January/062933.html
On 18.2.2012, at 20.26, Michael Grimm wrote:
doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted.
Hi --
On 21.02.2012, at 02:02, Timo Sirainen wrote:
On 18.2.2012, at 20.26, Michael Grimm wrote:
doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted.
Hmm, now I cannot clients any longer:
dovecot: master: Dovecot v20120221 (7d0d76df719f+) starting up dovecot: imap-login: Login: user=<test>, method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=yyy.yyy.yyy.yyy, mpid=65444, TLS dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test dovecot: imap(test): Error: Invalid user settings. Refer to server log for more information. [... repeated for every user clients are trying to connect ...]
I switched back immediately (production server), thus I didn't test if dsync might run now.
Here's the relevant part of my config:
mail_home = /var/mail/.homedirs/%n
mail_location = mdbox:/var/mail/.maildirs/%n
User's individual mail storage formats are fetched from userdb (sqlite3), which normally are mdbox, except for test which is sdbox.
Any idea what to test next?
Thanks and regards, Michael
Hi,
On 21.2.2012, at 19.29, Michael Grimm wrote:
doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted.
Hmm, now I cannot clients any longer:
dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test
Sorry, that part of the code is a bit weird. http://hg.dovecot.org/dovecot-2.1/rev/3007994141b6 is another attempt at this.
Hi --
On 21.02.2012, at 21:52, Timo Sirainen wrote:
On 21.2.2012, at 19.29, Michael Grimm wrote:
doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted.
Hmm, now I cannot connect clients any longer:
dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test
Sorry, that part of the code is a bit weird. http://hg.dovecot.org/dovecot-2.1/rev/3007994141b6 is another attempt at this.
Thanks, that omits the first and second error, and I believe that you solved this issue.
Now, I can run my old command as used for 2.0.x and ...
vmail> dsync -v -f -u test mirror ssh vmail@remote-host.tld dsync -v -f -u test
... a "newer" call like ...
vmail> doveadm sync -f -u test ssh vmail@remote-host.tld dsync -v -f -u test
... without complaining about namespaces as listed above.
But, now I do end with a message like ...
vmail> dsync -v -f -u test mirror ssh vmail@remote-host.tld dsync -v -f -u test
usage: dsync [-C <alt char>] [-m <mailbox>] [-u <user>] [-frRv]
mirror <local mail_location> | [<user>@]<host> | <remote dsync command>
dsync-local(test): Error: read() from worker server failed: EOF
... irrespectively if I use the old "dsync" or the new "doveadm sync".
Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup, but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way.
Weird. I must have missed something. I'll back to 2.0.18 for the time being. Any hints are highly welcome ...
Thanks for your efforts, and thanks for dovecot!
Regards, Michael
On 21.2.2012, at 23.48, Michael Grimm wrote:
But, now I do end with a message like ...
vmail> dsync -v -f -u test mirror ssh vmail@remote-host.tld dsync -v -f -u test usage: dsync [-C <alt char>] [-m <mailbox>] [-u <user>] [-frRv] mirror <local mail_location> | [<user>@]<host> | <remote dsync command> dsync-local(test): Error: read() from worker server failed: EOF
... irrespectively if I use the old "dsync" or the new "doveadm sync".
Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup,
Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code).
but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way.
Added it there.
Hi --
On 21.02.2012, at 22:57, Timo Sirainen wrote:
On 21.2.2012, at 23.48, Michael Grimm wrote:
But, now I do end with a message like ...
vmail> dsync -v -f -u test mirror ssh vmail@remote-host.tld dsync -v -f -u test usage: dsync [-C <alt char>] [-m <mailbox>] [-u <user>] [-frRv] mirror <local mail_location> | [<user>@]<host> | <remote dsync command> dsync-local(test): Error: read() from worker server failed: EOF
... irrespectively if I use the old "dsync" or the new "doveadm sync".
Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup,
Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code).
Ok. Then I will try both servers upgraded to 2.1 within the next days. Thanks for that valuable info that I really missed, sorry.
but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way.
Added it there.
Great.
And thanks again, Michael
Hi --
On 21.02.2012, at 23:03, Michael Grimm wrote:
On 21.02.2012, at 22:57, Timo Sirainen wrote:
Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code).
Ok. Then I will try both servers upgraded to 2.1 within the next days.
JFTR: It took me some time to find a working syntax ;-)
My working 2.0.18 syntax threw the following error:
vmail> dsync -v -f -u test ssh vmail@remote-host.tld dsync -v -f -u test
doveadm(vmail): Fatal: Unknown print formatter: -u
dsync-local(test): Error: read() from worker server failed: EOF
Omitting '-f' or '-v -f' results in:
vmail> dsync -v -u test ssh vmail@remote-host.tld dsync -v -u test
dsync: illegal option -- u
dsync-local(test): Error: read() from worker server failed: EOF
Now I switched to the recommended new syntax as stated in http://wiki2.dovecot.org/Upgrading/2.1:
vmail> doveadm sync -v -f -u test ssh vmail@remote-host.tld doveadm sync -v -f -u test
doveadm: illegal option -- v
doveadm sync [-u <user>|-A] [-S <socket_path>] [-fR] [-m <mailbox>] <dest>
ok, following those hints:
vmail> doveadm sync -u test -f ssh vmail@remote-host.tld doveadm sync -u test
ssh: Could not resolve hostname dsync-server: hostname nor servname provided, or not known
dsync-local(test): Error: read() from worker server failed: EOF
dsync-local(test): Error: read() from worker server failed: EOF
Hmm, 'man doveadm' doesn't show 'sync', and 'man sync' shows old syntax. Luckily, I remembered finding an mail of yours in http://www.dovecot.org/list/dovecot/2012-January/062933.html :
|>>> dsync mirror -> doveadm mirror
|>>> dsync backup -> doveadm backup
|>>> dsync server -> doveadm dsync-server (could be hidden from the doveadm commands list)
| I did this now, with mirror -> sync.
After some trial by error I finally found a working syntax:
vmail> doveadm sync -u test -f ssh vmail@remote-host.tld doveadm dsync-server -u test
but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way.
Added it there.
Therefore you might add a warning about different syntax as well?
Please, don't get me wrong, I'm not complaining. This mail is JFTR, only. And, this mail might help others to save some time after an upgrade to 2.1 and those who mirror mailboxes sitting at different servers.
Thanks and regards, Michael
On Thu, 2012-02-23 at 20:55 +0100, Michael Grimm wrote:
Hi --
On 21.02.2012, at 23:03, Michael Grimm wrote:
On 21.02.2012, at 22:57, Timo Sirainen wrote:
Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code).
Ok. Then I will try both servers upgraded to 2.1 within the next days.
JFTR: It took me some time to find a working syntax ;-)
My working 2.0.18 syntax threw the following error:
vmail> dsync -v -f -u test ssh vmail@remote-host.tld dsync -v -f -u test doveadm(vmail): Fatal: Unknown print formatter: -u dsync-local(test): Error: read() from worker server failed: EOF
You left out "mirror" from that command, but after adding it the latest hg version works.
Now I switched to the recommended new syntax as stated in http://wiki2.dovecot.org/Upgrading/2.1:
vmail> doveadm sync -v -f -u test ssh vmail@remote-host.tld doveadm sync -v -f -u test doveadm: illegal option -- v doveadm sync [-u <user>|-A] [-S <socket_path>] [-fR] [-m <mailbox>] <dest>
The -v parameter is in wrong place now, needs to be "doveadm -v sync".
After some trial by error I finally found a working syntax:
vmail> doveadm sync -u test -f ssh vmail@remote-host.tld doveadm dsync-server -u test
Oh, hmm. I hadn't thought about this problem, it shouldn't have been necessary to give the dsync-server parameter. But I guess there's not a whole lot of other possibilities to do this. Hmm.
BTW. I think you can do this simply:
doveadm sync -f -u test test@remote-host.tld
Hi --
On 27.02.2012, at 13:45, Timo Sirainen wrote:
On Thu, 2012-02-23 at 20:55 +0100, Michael Grimm wrote:
My working 2.0.18 syntax threw the following error:
vmail> dsync -v -f -u test ssh vmail@remote-host.tld dsync -v -f -u test doveadm(vmail): Fatal: Unknown print formatter: -u dsync-local(test): Error: read() from worker server failed: EOF
You left out "mirror" from that command, but after adding it the latest hg version works.
I did use 'mirror', I just forgot to paste it.
JFTR: vmail> dovecot --version 20120303 (1002733ca266+)
vmail> dsync -v -f -u test mirror ssh vmail@remote-host.tld dsync -v -f -u test
dsync-local(test): Error: remote: dsync: illegal option -- f
doveadm dsync-server [-u <user>|-A] [-S <socket_path>]
dsync-local(test): Error: read() from worker server failed: EOF
If I do omit the remote '-f' the old syntax is being accepted. Thanks for fixing that.
Now I switched to the recommended new syntax as stated in http://wiki2.dovecot.org/Upgrading/2.1:
vmail> doveadm sync -v -f -u test ssh vmail@remote-host.tld doveadm sync -v -f -u test doveadm: illegal option -- v doveadm sync [-u <user>|-A] [-S <socket_path>] [-fR] [-m <mailbox>] <dest>
The -v parameter is in wrong place now, needs to be "doveadm -v sync".
Ah, yes. That was my mistake, sorry.
After some trial by error I finally found a working syntax:
vmail> doveadm sync -u test -f ssh vmail@remote-host.tld doveadm dsync-server -u test
Oh, hmm. I hadn't thought about this problem, it shouldn't have been necessary to give the dsync-server parameter. But I guess there's not a whole lot of other possibilities to do this. Hmm.
BTW. I think you can do this simply:
doveadm sync -f -u test test@remote-host.tld
vmail> doveadm sync -f -u test ssh vmail@remote-host.tld
dsync-local(test): Error: remote: dsync-server: Command not found.
dsync-local(test): Error: read() from worker server failed: EOF
Users are virtual ones at both mail servers, and vmail is the only system user to run ssh. Thus I can't test user suggestion. But it's ok to add a 'doveadm dsync-server -u test'.
Now, I will stick to 2.1.x because syncing is done without loss of mails (after 10 days of testing). The only inconvenience remaining is reappearing of deleted and "undeletable" mail. But it seems that I'm the only one reporting that. How could I help to debug this issue?
Regards, Michael
On 3.3.2012, at 20.03, Michael Grimm wrote:
vmail> dsync -v -f -u test mirror ssh vmail@remote-host.tld dsync -v -f -u test dsync-local(test): Error: remote: dsync: illegal option -- f doveadm dsync-server [-u <user>|-A] [-S <socket_path>] dsync-local(test): Error: read() from worker server failed: EOF
If I do omit the remote '-f' the old syntax is being accepted. Thanks for fixing that.
Right, the remote -f parameter doesn't do anything. But it's anyway now allowed: http://hg.dovecot.org/dovecot-2.1/rev/9c6eeeb810c0
doveadm sync -f -u test test@remote-host.tld
vmail> doveadm sync -f -u test ssh vmail@remote-host.tld dsync-local(test): Error: remote: dsync-server: Command not found. dsync-local(test): Error: read() from worker server failed: EOF
Remove the "ssh" parameter from the middle and change vmail@ to test@
Users are virtual ones at both mail servers, and vmail is the only system user to run ssh. Thus I can't test user suggestion. But it's ok to add a 'doveadm dsync-server -u test'.
In this syntax the test@ means Dovecot user, not system user. Although I'm not sure if that's a good idea. In the latest hg version the preferred way is:
doveadm sync -f -u test remote:vmail@host
It automatically adds the remote -u test.
Now, I will stick to 2.1.x because syncing is done without loss of mails (after 10 days of testing). The only inconvenience remaining is reappearing of deleted and "undeletable" mail. But it seems that I'm the only one reporting that. How could I help to debug this issue?
Is anything else besides Dovecot modifying the mailboxes? Especially deleting mails? No cronjobs or other scripts that "rm" mails?
Hi --
On 04.03.2012, at 12:05, Timo Sirainen wrote:
On 3.3.2012, at 20.03, Michael Grimm wrote:
vmail> dsync -v -f -u test mirror ssh vmail@remote-host.tld dsync -v -f -u test dsync-local(test): Error: remote: dsync: illegal option -- f doveadm dsync-server [-u <user>|-A] [-S <socket_path>] dsync-local(test): Error: read() from worker server failed: EOF
If I do omit the remote '-f' the old syntax is being accepted. Thanks for fixing that.
Right, the remote -f parameter doesn't do anything. But it's anyway now allowed: http://hg.dovecot.org/dovecot-2.1/rev/9c6eeeb810c0
Ok, that means it has always been ignored in 2.0.x ;-)
doveadm sync -f -u test test@remote-host.tld
vmail> doveadm sync -f -u test ssh vmail@remote-host.tld dsync-local(test): Error: remote: dsync-server: Command not found. dsync-local(test): Error: read() from worker server failed: EOF
Remove the "ssh" parameter from the middle and change vmail@ to test@
That doesn't work in my ssh setup, because I'm using a different ssh port and thus have to run:
ssh -p 1234 vmail@remote-host.tld
Sorry, I should have mentioned that before.
In your other mail about 'dsync replication' you refer to a config option: #dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout} -n%{namespace}
Would that allow for ssh options to be set?
Users are virtual ones at both mail servers, and vmail is the only system user to run ssh. Thus I can't test user suggestion. But it's ok to add a 'doveadm dsync-server -u test'.
In this syntax the test@ means Dovecot user, not system user. Although I'm not sure if that's a good idea. In the latest hg version the preferred way is:
doveadm sync -f -u test remote:vmail@host
See above regarding ssh options.
Now, I will stick to 2.1.x because syncing is done without loss of mails (after 10 days of testing). The only inconvenience remaining is reappearing of deleted and "undeletable" mail. But it seems that I'm the only one reporting that. How could I help to debug this issue?
Is anything else besides Dovecot modifying the mailboxes? Especially deleting mails? No cronjobs or other scripts that "rm" mails?
No. Only dovecot is allowed to deliver mail (lmtp). Sieve's 'copy:' and 'fileinto' are used as well. But no scripts or such are used to remove mail. I'm running mdbox only.
Thanks and regards, Michael
On 4.3.2012, at 13.31, Michael Grimm wrote:
That doesn't work in my ssh setup, because I'm using a different ssh port and thus have to run:
ssh -p 1234 vmail@remote-host.tld
Sorry, I should have mentioned that before.
In your other mail about 'dsync replication' you refer to a config option: #dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout} -n%{namespace}
Would that allow for ssh options to be set?
Yes.
doveadm sync -f -u test remote:vmail@host
See above regarding ssh options.
So this works by changing the dsync_remote_cmd.
Now, I will stick to 2.1.x because syncing is done without loss of mails (after 10 days of testing). The only inconvenience remaining is reappearing of deleted and "undeletable" mail. But it seems that I'm the only one reporting that. How could I help to debug this issue?
Is anything else besides Dovecot modifying the mailboxes? Especially deleting mails? No cronjobs or other scripts that "rm" mails?
No. Only dovecot is allowed to deliver mail (lmtp). Sieve's 'copy:' and 'fileinto' are used as well. But no scripts or such are used to remove mail. I'm running mdbox only.
By "undeletable" do you mean you have mails that always come back after expunging them? I'd like to get dovecot.index and dovecot.index.log files from those mailboxes from both servers, they don't contain any sensitive information.
Hi --
On 04.03.2012, at 12:35, Timo Sirainen wrote:
On 4.3.2012, at 13.31, Michael Grimm wrote:
That doesn't work in my ssh setup, because I'm using a different ssh port and thus have to run:
ssh -p 1234 vmail@remote-host.tld
Sorry, I should have mentioned that before.
In your other mail about 'dsync replication' you refer to a config option: #dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -l%{lock_timeout} -n%{namespace}
Would that allow for ssh options to be set?
Yes.
Good news.
Now, I will stick to 2.1.x because syncing is done without loss of mails (after 10 days of testing). The only inconvenience remaining is reappearing of deleted and "undeletable" mail. But it seems that I'm the only one reporting that. How could I help to debug this issue?
Is anything else besides Dovecot modifying the mailboxes? Especially deleting mails? No cronjobs or other scripts that "rm" mails?
No. Only dovecot is allowed to deliver mail (lmtp). Sieve's 'copy:' and 'fileinto' are used as well. But no scripts or such are used to remove mail. I'm running mdbox only.
By "undeletable" do you mean you have mails that always come back after expunging them?
Yes. Deleting by the client will return them after the next dsync run.
I'd like to get dovecot.index and dovecot.index.log files from those mailboxes from both servers, they don't contain any sensitive information.
From all mailboxes? I can do that if you wish. But that will need some time (tomorrow).
Thanks and regards, Michael
On 4.3.2012, at 13.41, Michael Grimm wrote:
By "undeletable" do you mean you have mails that always come back after expunging them?
Yes. Deleting by the client will return them after the next dsync run.
I'd like to get dovecot.index and dovecot.index.log files from those mailboxes from both servers, they don't contain any sensitive information.
From all mailboxes? I can do that if you wish. But that will need some time (tomorrow).
Just one mailbox where that consistently happens is enough:
- Expunge the mail
- Get a copy of the dbox-Mails/dovecot.index, dbox-Mails/dovecot.index.log and dbox-Mails/dovecot.index.log.2 from both servers
- Run dsync, and verify that the message is undeleted
- Get another copy of the indexes from both servers
On 4.3.2012, at 13.54, Timo Sirainen wrote:
On 4.3.2012, at 13.41, Michael Grimm wrote:
By "undeletable" do you mean you have mails that always come back after expunging them?
Yes. Deleting by the client will return them after the next dsync run.
Luckily this just started happening to me as well. After some debugging I found and fixed the problem:
HI --
On 05.03.2012 10:56, Timo Sirainen wrote:
On 4.3.2012, at 13.54, Timo Sirainen wrote:
On 4.3.2012, at 13.41, Michael Grimm wrote:
By "undeletable" do you mean you have mails that always come back after expunging them?
Yes. Deleting by the client will return them after the next dsync run.
Luckily this just started happening to me as well. After some debugging I found and fixed the problem:
I can confirm, that you fixed that issue successfully.
Thanks and regards, Michael
participants (2)
-
Michael Grimm
-
Timo Sirainen