failed: read(/var/run/dovecot/dns-client)
=== dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server
Initially I thought "oh no, not another AppArmor block".
But then surely the second message would not appear if the DNS lookup was not successful ?
Also "dig foobar.example.com" works fine.
How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ?
So many questions !
On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote:
=== dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
This is dovecot's internal dns-client, and something goes wrong when talking to the service.
dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server
This is btw dsync service, not imap service.
===
Initially I thought "oh no, not another AppArmor block".
But then surely the second message would not appear if the DNS lookup was not successful ?
Also "dig foobar.example.com" works fine.
How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ?
Because the "standard OS call" is blocking and we would prefer it to not block everything else.
So many questions !
Aki
On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote:
dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
This is dovecot's internal dns-client, and something goes wrong when talking to the service.
dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server
This is btw dsync service, not imap service.
=== Initially I thought "oh no, not another AppArmor block". But then surely the second message would not appear if the DNS lookup was not successful ? Also "dig foobar.example.com" works fine. How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ?
Because the "standard OS call" is blocking and we would prefer it to not block everything else.
So many questions !
Aki
Thanks for your reply, but both those message are generated from a simple : doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc:
So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ?
I'm still none the wiser as to where to start looking for troubleshoting ?
On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote:
On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote:
dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
This is dovecot's internal dns-client, and something goes wrong when talking to the service.
dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server
This is btw dsync service, not imap service.
=== Initially I thought "oh no, not another AppArmor block". But then surely the second message would not appear if the DNS lookup was not successful ? Also "dig foobar.example.com" works fine. How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ?
Because the "standard OS call" is blocking and we would prefer it to not block everything else.
So many questions !
Aki
Thanks for your reply, but both those message are generated from a simple : doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc:
So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ?
I'm still none the wiser as to where to start looking for troubleshoting ?
Did you check dovecot logs? Maybe there is something useful?
Aki
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote:
==========================================================================
dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
This is dovecot's internal dns-client, and something goes wrong when talking to the service.
dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server
This is btw dsync service, not imap service.
=== Initially I thought "oh no, not another AppArmor block". But then surely the second message would not appear if the DNS lookup was not successful ? Also "dig foobar.example.com" works fine. How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ?
Because the "standard OS call" is blocking and we would prefer it to not block everything else.
So many questions !
Aki
Thanks for your reply, but both those message are generated from a simple : doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? I'm still none the wiser as to where to start looking for troubleshoting ?
Did you check dovecot logs? Maybe there is something useful?
Aki
Only the same old cryptic message about dns-client ? master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied master: Error: service(dns_client): command startup failed, throttling for 16 secs dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed)
On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote:
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote:
==========================================================================
dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
This is dovecot's internal dns-client, and something goes wrong when talking to the service.
dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server
This is btw dsync service, not imap service.
=== Initially I thought "oh no, not another AppArmor block". But then surely the second message would not appear if the DNS lookup was not successful ? Also "dig foobar.example.com" works fine. How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ?
Because the "standard OS call" is blocking and we would prefer it to not block everything else.
So many questions !
Aki
Thanks for your reply, but both those message are generated from a simple : doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? I'm still none the wiser as to where to start looking for troubleshoting ?
Did you check dovecot logs? Maybe there is something useful?
Aki
Only the same old cryptic message about dns-client ? master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied
Something prevents executing the dns-client binary.
master: Error: service(dns_client): command startup failed, throttling for 16 secs dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed)
Aki
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: Sent with ProtonMail Secure Email. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote:
dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
This is dovecot's internal dns-client, and something goes wrong when talking to the service.
dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server
This is btw dsync service, not imap service.
=== Initially I thought "oh no, not another AppArmor block". But then surely the second message would not appear if the DNS lookup was not successful ? Also "dig foobar.example.com" works fine. How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ?
Because the "standard OS call" is blocking and we would prefer it to not block everything else.
So many questions !
Aki
Thanks for your reply, but both those message are generated from a simple : doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? I'm still none the wiser as to where to start looking for troubleshoting ?
Did you check dovecot logs? Maybe there is something useful? Aki
Only the same old cryptic message about dns-client ? master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied
Something prevents executing the dns-client binary.
master: Error: service(dns_client): command startup failed, throttling for 16 secs dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed)
Aki
Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces !
Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ?)
# /usr/lib/dovecot/dns-client Panic: BUG: No IOs or timeouts set. Not waiting for infinity. Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xd879e) [0x7f582c65f79e] -> /usr/lib64/dovecot/libdovecot.so.0(+0xd87e1) [0x7f582c65f7e1] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f582c5c9024] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf045c) [0x7f582c67745c] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x36) [0x7f582c679e96] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x4c) [0x7f582c6786ec] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f582c678908] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f582c5ee203] -> /usr/lib/dovecot/dns-client(main+0x8d) [0x55866c96050d] -> /lib64/libc.so.6(__libc_start_main+0xea) [0x7f582c1edf4a] -> /usr/lib/dovecot/dns-client(_start+0x2a) [0x55866c96055a]
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote:
On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: Sent with ProtonMail Secure Email. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
> > On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: > > ========================================================================== > > dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
> This is dovecot's internal dns-client, and something goes wrong when talking to the service.
> > dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server
> This is btw dsync service, not imap service.
> > === > > Initially I thought "oh no, not another AppArmor block". > > But then surely the second message would not appear if the DNS lookup was not successful ? > > Also "dig foobar.example.com" works fine. > > How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ?
> Because the "standard OS call" is blocking and we would prefer it to not block everything else.
> > So many questions !
> Aki
Thanks for your reply, but both those message are generated from a simple : doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? I'm still none the wiser as to where to start looking for troubleshoting ?
Did you check dovecot logs? Maybe there is something useful? Aki
Only the same old cryptic message about dns-client ? master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied
Something prevents executing the dns-client binary.
master: Error: service(dns_client): command startup failed, throttling for 16 secs dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed)
Aki
Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces !
Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ?
It is started by dovecot's master process when you connect to dns-client unix socket. You can try
socat stdio unix-connect:/var/run/dovecot/dns-client
I thought apparmor tells when something is blocked into kernel log? have you checked dmesg?
Apologies for your frustration.
Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence.
"socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ?
When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : /{,var/}run/dovecot/ rw, /{,var/}run/dovecot/** rw,
On 11/04/2019 00:18, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote:
On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: Sent with ProtonMail Secure Email. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
> On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: >>> On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: >>> ========================================================================== >>> dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer >> This is dovecot's internal dns-client, and something goes wrong when talking to the service. >>> dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server >> This is btw dsync service, not imap service. >>> === >>> Initially I thought "oh no, not another AppArmor block". >>> But then surely the second message would not appear if the DNS lookup was not successful ? >>> Also "dig foobar.example.com" works fine. >>> How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? >> Because the "standard OS call" is blocking and we would prefer it to not block everything else. >>> So many questions ! >> Aki > Thanks for your reply, but both those message are generated from a simple : > doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: > So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? > I'm still none the wiser as to where to start looking for troubleshoting ? Did you check dovecot logs? Maybe there is something useful? Aki Only the same old cryptic message about dns-client ? master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied Something prevents executing the dns-client binary. master: Error: service(dns_client): command startup failed, throttling for 16 secs dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed) Aki Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces !
Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ? It is started by dovecot's master process when you connect to dns-client unix socket. You can try
socat stdio unix-connect:/var/run/dovecot/dns-client
I thought apparmor tells when something is blocked into kernel log? have you checked dmesg?
Apologies for your frustration.
Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence.
"socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ?
When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : /{,var/}run/dovecot/ rw, /{,var/}run/dovecot/** rw,
Laura
Do the above apparmor settings give permission to dovecot to execute /usr/lib/dovecot/dns-client, assuming that the user under which dovecot is running already has file system permissions to do that?
John
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 11:48 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:18, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote: ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote:
On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: Sent with ProtonMail Secure Email. ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
> > On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > > On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > > > > > > On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > > > > > ========================================================================== > > > > > > > > dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer > > > > This is dovecot's internal dns-client, and something goes wrong when talking to the service. > > > > dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server > > > > This is btw dsync service, not imap service. > > > > > > > > =============================================================================================================================================================================================================================================================================================================================================================================================================================================================================== > > > > > > > > Initially I thought "oh no, not another AppArmor block". > > > > But then surely the second message would not appear if the DNS lookup was not successful ? > > > > Also "dig foobar.example.com" works fine. > > > > How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? > > > > Because the "standard OS call" is blocking and we would prefer it to not block everything else. > > > > So many questions ! > > > > Aki > > > > Thanks for your reply, but both those message are generated from a simple : > > > > doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: > > > > So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? > > > > I'm still none the wiser as to where to start looking for troubleshoting ? > > > > Did you check dovecot logs? Maybe there is something useful? > > > > Aki > > > > Only the same old cryptic message about dns-client ? > > > > master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied > > > > Something prevents executing the dns-client binary. > > > > master: Error: service(dns_client): command startup failed, throttling for 16 secs > > > > dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed) > > > > Aki > > > > Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces !
Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ? It is started by dovecot's master process when you connect to dns-client unix socket. You can try
socat stdio unix-connect:/var/run/dovecot/dns-client I thought apparmor tells when something is blocked into kernel log? have you checked dmesg?
Apologies for your frustration.
Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence. "socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ? When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : /{,var/}run/dovecot/ rw, /{,var/}run/dovecot/** rw,
Laura
Do the above apparmor settings give permission to dovecot to execute /usr/lib/dovecot/dns-client, assuming that the user under which dovecot is running already has file system permissions to do that?
John
John,
Here's the definitive answer to your question (and anyone else thinking of pointing the finger at apparmor):
foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: DNS lookup timed out dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server
So. Can we move on from the "blame apparmor" ? ;-)
On 11/04/2019 00:51, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 11:48 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:18, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote: ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote:
> On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > Sent with ProtonMail Secure Email. > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > >>> On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: >>> On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: >>> >>>>> On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: >>>>> >>>>> ========================================================================== >>>>> >>>>> dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer >>>>> This is dovecot's internal dns-client, and something goes wrong when talking to the service. >>>>> dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server >>>>> This is btw dsync service, not imap service. >>>>> >>>>> =============================================================================================================================================================================================================================================================================================================================================================================================================================================================================== >>>>> >>>>> Initially I thought "oh no, not another AppArmor block". >>>>> But then surely the second message would not appear if the DNS lookup was not successful ? >>>>> Also "dig foobar.example.com" works fine. >>>>> How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? >>>>> Because the "standard OS call" is blocking and we would prefer it to not block everything else. >>>>> So many questions ! >>>>> Aki >>>>> Thanks for your reply, but both those message are generated from a simple : >>>>> doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: >>>>> So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? >>>>> I'm still none the wiser as to where to start looking for troubleshoting ? >>>>> Did you check dovecot logs? Maybe there is something useful? >>>>> Aki >>>>> Only the same old cryptic message about dns-client ? >>>>> master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied >>>>> Something prevents executing the dns-client binary. >>>>> master: Error: service(dns_client): command startup failed, throttling for 16 secs >>>>> dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed) >>>>> Aki >>>>> Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces ! Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ? It is started by dovecot's master process when you connect to dns-client unix socket. You can try socat stdio unix-connect:/var/run/dovecot/dns-client I thought apparmor tells when something is blocked into kernel log? have you checked dmesg?
Apologies for your frustration.
Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence. "socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ? When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : /{,var/}run/dovecot/ rw, /{,var/}run/dovecot/** rw, Laura
Do the above apparmor settings give permission to dovecot to execute /usr/lib/dovecot/dns-client, assuming that the user under which dovecot is running already has file system permissions to do that?
John
John,
Here's the definitive answer to your question (and anyone else thinking of pointing the finger at apparmor):
foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: DNS lookup timed out dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server
So. Can we move on from the "blame apparmor" ? ;-)
Laura
I'd suggest doing the test with a restart of dovecot in between stopping apparmor and running the doveadm command. Check your logs to see if there is no longer any message generated about not being able to execv /usr/lib/dovecot/dns-client.
foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # sudo systemctl restart dovecot foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc:
John
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 12:55 AM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:51, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 11:48 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:18, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote: ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote:
> > On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > > Sent with ProtonMail Secure Email. > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > > > > > > On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > > > > > > > > > > On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > > > ========================================================================== > > > > > > dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer > > > > > > This is dovecot's internal dns-client, and something goes wrong when talking to the service. > > > > > > dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server > > > > > > This is btw dsync service, not imap service. > > > > > > =============================================================================================================================================================================================================================================================================================================================================================================================================================================================================== > > > > > > Initially I thought "oh no, not another AppArmor block". > > > > > > But then surely the second message would not appear if the DNS lookup was not successful ? > > > > > > Also "dig foobar.example.com" works fine. > > > > > > How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? > > > > > > Because the "standard OS call" is blocking and we would prefer it to not block everything else. > > > > > > So many questions ! > > > > > > Aki > > > > > > Thanks for your reply, but both those message are generated from a simple : > > > > > > doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: > > > > > > So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? > > > > > > I'm still none the wiser as to where to start looking for troubleshoting ? > > > > > > Did you check dovecot logs? Maybe there is something useful? > > > > > > Aki > > > > > > Only the same old cryptic message about dns-client ? > > > > > > master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied > > > > > > Something prevents executing the dns-client binary. > > > > > > master: Error: service(dns_client): command startup failed, throttling for 16 secs > > > > > > dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed) > > > > > > Aki > > > > > > Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces ! > > > > > > Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ? > > > > > > It is started by dovecot's master process when you connect to dns-client unix socket. You can try > > > > > > socat stdio unix-connect:/var/run/dovecot/dns-client > > > > > > I thought apparmor tells when something is blocked into kernel log? have you checked dmesg?
Apologies for your frustration.
Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence. "socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ? When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : /{,var/}run/dovecot/ rw, /{,var/}run/dovecot/** rw, Laura
Do the above apparmor settings give permission to dovecot to execute /usr/lib/dovecot/dns-client, assuming that the user under which dovecot is running already has file system permissions to do that? John
John, Here's the definitive answer to your question (and anyone else thinking of pointing the finger at apparmor): foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: DNS lookup timed out dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server So. Can we move on from the "blame apparmor" ? ;-)
Laura
I'd suggest doing the test with a restart of dovecot in between stopping apparmor and running the doveadm command. Check your logs to see if there is no longer any message generated about not being able to execv /usr/lib/dovecot/dns-client.
foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # sudo systemctl restart dovecot foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc:
John
Same again....
failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
On 11 April 2019 11:02 Laura Smith via dovecot dovecot@dovecot.org wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 12:55 AM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:51, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 11:48 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:18, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
> On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote: > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote: > > > > On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > Sent with ProtonMail Secure Email. > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > > > > > > > > On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > > On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > > > > > > > > > > > > On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > > > > ========================================================================== > > > > > > > dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer > > > > > > > This is dovecot's internal dns-client, and something goes wrong when talking to the service. > > > > > > > dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server > > > > > > > This is btw dsync service, not imap service. > > > > > > > =============================================================================================================================================================================================================================================================================================================================================================================================================================================================================== > > > > > > > Initially I thought "oh no, not another AppArmor block". > > > > > > > But then surely the second message would not appear if the DNS lookup was not successful ? > > > > > > > Also "dig foobar.example.com" works fine. > > > > > > > How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? > > > > > > > Because the "standard OS call" is blocking and we would prefer it to not block everything else. > > > > > > > So many questions ! > > > > > > > Aki > > > > > > > Thanks for your reply, but both those message are generated from a simple : > > > > > > > doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: > > > > > > > So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? > > > > > > > I'm still none the wiser as to where to start looking for troubleshoting ? > > > > > > > Did you check dovecot logs? Maybe there is something useful? > > > > > > > Aki > > > > > > > Only the same old cryptic message about dns-client ? > > > > > > > master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied > > > > > > > Something prevents executing the dns-client binary. > > > > > > > master: Error: service(dns_client): command startup failed, throttling for 16 secs > > > > > > > dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed) > > > > > > > Aki > > > > > > > Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces ! > > > > > > > Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ? > > > > > > > It is started by dovecot's master process when you connect to dns-client unix socket. You can try > > > > > > > socat stdio unix-connect:/var/run/dovecot/dns-client > > > > > > > I thought apparmor tells when something is blocked into kernel log? have you checked dmesg?
Apologies for your frustration.
Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence. "socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ? When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : /{,var/}run/dovecot/ rw, /{,var/}run/dovecot/** rw, Laura
Do the above apparmor settings give permission to dovecot to execute /usr/lib/dovecot/dns-client, assuming that the user under which dovecot is running already has file system permissions to do that? John
John, Here's the definitive answer to your question (and anyone else thinking of pointing the finger at apparmor): foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: DNS lookup timed out dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server So. Can we move on from the "blame apparmor" ? ;-)
Laura
I'd suggest doing the test with a restart of dovecot in between stopping apparmor and running the doveadm command. Check your logs to see if there is no longer any message generated about not being able to execv /usr/lib/dovecot/dns-client.
foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # sudo systemctl restart dovecot foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc:
John
Same again....
failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
And your logs probably indicate that same Fatal error that the service cannot be started?
Wonder if this is caused by systemd? Can you try also
systemctl stop dovecot dovecot -F
then try socat and see if it works?
Aki
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 9:05 AM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 11 April 2019 11:02 Laura Smith via dovecot dovecot@dovecot.org wrote: ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 12:55 AM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:51, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 11:48 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:18, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
> > On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote: > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote: > > > > > > On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > Sent with ProtonMail Secure Email. > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > > > > > > > > > > On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > > > On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > > > > > > > > > > > > > > On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > > > > > > > > > > > > > ========================================================================== > > > > > > > > > > > > > > > > dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer > > > > > > > > This is dovecot's internal dns-client, and something goes wrong when talking to the service. > > > > > > > > dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server > > > > > > > > This is btw dsync service, not imap service. > > > > > > > > > > > > > > > > =============================================================================================================================================================================================================================================================================================================================================================================================================================================================================== > > > > > > > > > > > > > > > > Initially I thought "oh no, not another AppArmor block". > > > > > > > > But then surely the second message would not appear if the DNS lookup was not successful ? > > > > > > > > Also "dig foobar.example.com" works fine. > > > > > > > > How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? > > > > > > > > Because the "standard OS call" is blocking and we would prefer it to not block everything else. > > > > > > > > So many questions ! > > > > > > > > Aki > > > > > > > > Thanks for your reply, but both those message are generated from a simple : > > > > > > > > doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: > > > > > > > > So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? > > > > > > > > I'm still none the wiser as to where to start looking for troubleshoting ? > > > > > > > > Did you check dovecot logs? Maybe there is something useful? > > > > > > > > Aki > > > > > > > > Only the same old cryptic message about dns-client ? > > > > > > > > master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied > > > > > > > > Something prevents executing the dns-client binary. > > > > > > > > master: Error: service(dns_client): command startup failed, throttling for 16 secs > > > > > > > > dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed) > > > > > > > > Aki > > > > > > > > Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces ! > > > > > > > > Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ? > > > > > > > > It is started by dovecot's master process when you connect to dns-client unix socket. You can try > > > > > > > > socat stdio unix-connect:/var/run/dovecot/dns-client > > > > > > > > I thought apparmor tells when something is blocked into kernel log? have you checked dmesg? > > Apologies for your frustration.
Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence. "socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ? When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : /{,var/}run/dovecot/ rw, /{,var/}run/dovecot/** rw, Laura
Do the above apparmor settings give permission to dovecot to execute /usr/lib/dovecot/dns-client, assuming that the user under which dovecot is running already has file system permissions to do that? John
John, Here's the definitive answer to your question (and anyone else thinking of pointing the finger at apparmor): foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: DNS lookup timed out dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server So. Can we move on from the "blame apparmor" ? ;-)
Laura I'd suggest doing the test with a restart of dovecot in between stopping apparmor and running the doveadm command. Check your logs to see if there is no longer any message generated about not being able to execv /usr/lib/dovecot/dns-client. foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # sudo systemctl restart dovecot foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: John
Same again.... failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
And your logs probably indicate that same Fatal error that the service cannot be started?
Wonder if this is caused by systemd? Can you try also
systemctl stop dovecot dovecot -F
then try socat and see if it works?
Aki
I get no output om the "dovecot -F" side ? Dovecot launches but no consle output, either at launch or in response to my test commands.
On 11.4.2019 11.11, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 9:05 AM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
On 11 April 2019 11:02 Laura Smith via dovecot dovecot@dovecot.org wrote: ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 12:55 AM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:51, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 11:48 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:18, Laura Smith via dovecot wrote:
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > >>> On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote: >>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>> On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote: >>> >>>>> On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: >>>>> Sent with ProtonMail Secure Email. >>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>> On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: >>>>> >>>>>>> On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: >>>>>>> On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: >>>>>>> >>>>>>>>> On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: >>>>>>>>> >>>>>>>>> ========================================================================== >>>>>>>>> >>>>>>>>> dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer >>>>>>>>> This is dovecot's internal dns-client, and something goes wrong when talking to the service. >>>>>>>>> dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server >>>>>>>>> This is btw dsync service, not imap service. >>>>>>>>> >>>>>>>>> =============================================================================================================================================================================================================================================================================================================================================================================================================================================================================== >>>>>>>>> >>>>>>>>> Initially I thought "oh no, not another AppArmor block". >>>>>>>>> But then surely the second message would not appear if the DNS lookup was not successful ? >>>>>>>>> Also "dig foobar.example.com" works fine. >>>>>>>>> How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? >>>>>>>>> Because the "standard OS call" is blocking and we would prefer it to not block everything else. >>>>>>>>> So many questions ! >>>>>>>>> Aki >>>>>>>>> Thanks for your reply, but both those message are generated from a simple : >>>>>>>>> doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: >>>>>>>>> So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? >>>>>>>>> I'm still none the wiser as to where to start looking for troubleshoting ? >>>>>>>>> Did you check dovecot logs? Maybe there is something useful? >>>>>>>>> Aki >>>>>>>>> Only the same old cryptic message about dns-client ? >>>>>>>>> master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied >>>>>>>>> Something prevents executing the dns-client binary. >>>>>>>>> master: Error: service(dns_client): command startup failed, throttling for 16 secs >>>>>>>>> dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed) >>>>>>>>> Aki >>>>>>>>> Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces ! >>>>>>>>> Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ? >>>>>>>>> It is started by dovecot's master process when you connect to dns-client unix socket. You can try >>>>>>>>> socat stdio unix-connect:/var/run/dovecot/dns-client >>>>>>>>> I thought apparmor tells when something is blocked into kernel log? have you checked dmesg? >> Apologies for your frustration. > Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence. > "socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ? > When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : > /{,var/}run/dovecot/ rw, > /{,var/}run/dovecot/** rw, > Laura Do the above apparmor settings give permission to dovecot to execute /usr/lib/dovecot/dns-client, assuming that the user under which dovecot is running already has file system permissions to do that? John John, Here's the definitive answer to your question (and anyone else thinking of pointing the finger at apparmor): foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: DNS lookup timed out dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server So. Can we move on from the "blame apparmor" ? ;-) Laura I'd suggest doing the test with a restart of dovecot in between stopping apparmor and running the doveadm command. Check your logs to see if there is no longer any message generated about not being able to execv /usr/lib/dovecot/dns-client. foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # sudo systemctl restart dovecot foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: John Same again.... failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer And your logs probably indicate that same Fatal error that the service cannot be started?
Wonder if this is caused by systemd? Can you try also
systemctl stop dovecot dovecot -F
then try socat and see if it works?
Aki I get no output om the "dovecot -F" side ? Dovecot launches but no consle output, either at launch or in response to my test commands.
Oh sorry. It will log into syslog anyways unless you set log_path=/dev/stdout in config.
Aki
On 11/04/2019 10:02, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 12:55 AM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:51, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 11:48 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:18, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
> On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote: > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote: > >>> On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: >>> Sent with ProtonMail Secure Email. >>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>> On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: >>> >>>>> On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: >>>>> On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: >>>>> >>>>>>> On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: >>>>>>> ========================================================================== >>>>>>> dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer >>>>>>> This is dovecot's internal dns-client, and something goes wrong when talking to the service. >>>>>>> dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server >>>>>>> This is btw dsync service, not imap service. >>>>>>> =============================================================================================================================================================================================================================================================================================================================================================================================================================================================================== >>>>>>> Initially I thought "oh no, not another AppArmor block". >>>>>>> But then surely the second message would not appear if the DNS lookup was not successful ? >>>>>>> Also "dig foobar.example.com" works fine. >>>>>>> How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? >>>>>>> Because the "standard OS call" is blocking and we would prefer it to not block everything else. >>>>>>> So many questions ! >>>>>>> Aki >>>>>>> Thanks for your reply, but both those message are generated from a simple : >>>>>>> doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: >>>>>>> So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? >>>>>>> I'm still none the wiser as to where to start looking for troubleshoting ? >>>>>>> Did you check dovecot logs? Maybe there is something useful? >>>>>>> Aki >>>>>>> Only the same old cryptic message about dns-client ? >>>>>>> master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied >>>>>>> Something prevents executing the dns-client binary. >>>>>>> master: Error: service(dns_client): command startup failed, throttling for 16 secs >>>>>>> dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed) >>>>>>> Aki >>>>>>> Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces ! >>>>>>> Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ? >>>>>>> It is started by dovecot's master process when you connect to dns-client unix socket. You can try >>>>>>> socat stdio unix-connect:/var/run/dovecot/dns-client >>>>>>> I thought apparmor tells when something is blocked into kernel log? have you checked dmesg? Apologies for your frustration. Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence. "socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ? When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : /{,var/}run/dovecot/ rw, /{,var/}run/dovecot/** rw, Laura Do the above apparmor settings give permission to dovecot to execute /usr/lib/dovecot/dns-client, assuming that the user under which dovecot is running already has file system permissions to do that? John John, Here's the definitive answer to your question (and anyone else thinking of pointing the finger at apparmor): foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: DNS lookup timed out dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server So. Can we move on from the "blame apparmor" ? ;-) Laura
I'd suggest doing the test with a restart of dovecot in between stopping apparmor and running the doveadm command. Check your logs to see if there is no longer any message generated about not being able to execv /usr/lib/dovecot/dns-client.
foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # sudo systemctl restart dovecot foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc:
John
Same again....
failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
do you get any messages in /var/log/audit/audit.log when executing this test?
John
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 9:01 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 10:02, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 12:55 AM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:51, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 11:48 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:18, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote:
> > On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote: > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote: > > > > > > On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > Sent with ProtonMail Secure Email. > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > > > > On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > > > > > > > > > > On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > > > On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > > > > > > > > > > > > > > On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: > > > > > > > > > > > > > > > > ========================================================================== > > > > > > > > > > > > > > > > dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer > > > > > > > > This is dovecot's internal dns-client, and something goes wrong when talking to the service. > > > > > > > > dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server > > > > > > > > This is btw dsync service, not imap service. > > > > > > > > > > > > > > > > =============================================================================================================================================================================================================================================================================================================================================================================================================================================================================== > > > > > > > > > > > > > > > > Initially I thought "oh no, not another AppArmor block". > > > > > > > > But then surely the second message would not appear if the DNS lookup was not successful ? > > > > > > > > Also "dig foobar.example.com" works fine. > > > > > > > > How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? > > > > > > > > Because the "standard OS call" is blocking and we would prefer it to not block everything else. > > > > > > > > So many questions ! > > > > > > > > Aki > > > > > > > > Thanks for your reply, but both those message are generated from a simple : > > > > > > > > doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: > > > > > > > > So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? > > > > > > > > I'm still none the wiser as to where to start looking for troubleshoting ? > > > > > > > > Did you check dovecot logs? Maybe there is something useful? > > > > > > > > Aki > > > > > > > > Only the same old cryptic message about dns-client ? > > > > > > > > master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied > > > > > > > > Something prevents executing the dns-client binary. > > > > > > > > master: Error: service(dns_client): command startup failed, throttling for 16 secs > > > > > > > > dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed) > > > > > > > > Aki > > > > > > > > Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces ! > > > > > > > > Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ? > > > > > > > > It is started by dovecot's master process when you connect to dns-client unix socket. You can try > > > > > > > > socat stdio unix-connect:/var/run/dovecot/dns-client > > > > > > > > I thought apparmor tells when something is blocked into kernel log? have you checked dmesg? > > > > > > > > Apologies for your frustration. > > > > > > > > Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence. > > > > > > > > "socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ? > > > > > > > > When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : > > > > > > > > /{,var/}run/dovecot/ rw, > > > > > > > > /{,var/}run/dovecot/** rw, > > > > > > > > Laura > > > > > > > > Do the above apparmor settings give permission to dovecot to execute > > > > > > > > /usr/lib/dovecot/dns-client, assuming that the user under which dovecot > > > > > > > > is running already has file system permissions to do that? > > > > > > > > John > > > > > > > > John, > > > > > > > > Here's the definitive answer to your question (and anyone else thinking of pointing the finger at apparmor): > > > > > > > > foo:/home/foo # sudo systemctl stop apparmor > > > > > > > > foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: > > > > > > > > dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: DNS lookup timed out > > > > > > > > dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server > > > > > > > > So. Can we move on from the "blame apparmor" ? ;-) > > > > > > > > Laura
I'd suggest doing the test with a restart of dovecot in between stopping apparmor and running the doveadm command. Check your logs to see if there is no longer any message generated about not being able to execv /usr/lib/dovecot/dns-client. foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # sudo systemctl restart dovecot foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: John
Same again.... failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer
do you get any messages in /var/log/audit/audit.log when executing this test?
John
I did (which also lead me to the discovery that stopping the apparmor service doesn't actually do what you think it might, you still need to run 'aa-teardown').
But the answer for posterity is the following :
Put : /usr/lib/dovecot/dns-client mrix, /var/run/dovecot/dns-client mrix, Into: /etc/apparmor.d/local/usr.sbin.dovecot And: systemctl restart apparmor && systemctl restart dovecot
Thank you to all those here for your help. However I would stil like to see a way to be able to manually test 'dns-client' included in future dovecot releases.
On 11/04/2019 22:09, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 9:01 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 10:02, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, April 11, 2019 12:55 AM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:51, Laura Smith via dovecot wrote:
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, April 10, 2019 11:48 PM, John Fawcett via dovecot dovecot@dovecot.org wrote:
On 11/04/2019 00:18, Laura Smith via dovecot wrote:
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Wednesday, April 10, 2019 10:24 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: > >>> On 10 April 2019 23:56 Laura Smith via dovecot < dovecot@dovecot.org> wrote: >>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>> On Wednesday, April 10, 2019 9:14 PM, Aki Tuomi < aki.tuomi@open-xchange.com> wrote: >>> >>>>> On 10 April 2019 23:13 Laura Smith via dovecot dovecot@dovecot.org wrote: >>>>> Sent with ProtonMail Secure Email. >>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ >>>>> On Wednesday, April 10, 2019 8:20 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: >>>>> >>>>>>> On 10 April 2019 22:13 Laura Smith via dovecot dovecot@dovecot.org wrote: >>>>>>> On Wednesday, April 10, 2019 7:57 PM, Aki Tuomi aki.tuomi@open-xchange.com wrote: >>>>>>> >>>>>>>>> On 10 April 2019 21:26 Laura Smith via dovecot dovecot@dovecot.org wrote: >>>>>>>>> >>>>>>>>> ========================================================================== >>>>>>>>> >>>>>>>>> dsync( foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer >>>>>>>>> This is dovecot's internal dns-client, and something goes wrong when talking to the service. >>>>>>>>> dsync( foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server >>>>>>>>> This is btw dsync service, not imap service. >>>>>>>>> >>>>>>>>> =============================================================================================================================================================================================================================================================================================================================================================================================================================================================================== >>>>>>>>> >>>>>>>>> Initially I thought "oh no, not another AppArmor block". >>>>>>>>> But then surely the second message would not appear if the DNS lookup was not successful ? >>>>>>>>> Also "dig foobar.example.com" works fine. >>>>>>>>> How should I be troubleshooting this ? And if it is still likely to be AppArmor, what is calling it ? "doveadm" itself or something else ? What does "/var/run/dovecot/dns-client" do and why doesn't dovecot use standard OS calls like everyone else ? >>>>>>>>> Because the "standard OS call" is blocking and we would prefer it to not block everything else. >>>>>>>>> So many questions ! >>>>>>>>> Aki >>>>>>>>> Thanks for your reply, but both those message are generated from a simple : >>>>>>>>> doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: >>>>>>>>> So I don't know what you mean about dsync service failing ? Surely the DNS lookup succeeded if the 'dsync service' failed due to remote disconnect ? >>>>>>>>> I'm still none the wiser as to where to start looking for troubleshoting ? >>>>>>>>> Did you check dovecot logs? Maybe there is something useful? >>>>>>>>> Aki >>>>>>>>> Only the same old cryptic message about dns-client ? >>>>>>>>> master: Fatal: execv(/usr/lib/dovecot/dns-client) failed: Permission denied >>>>>>>>> Something prevents executing the dns-client binary. >>>>>>>>> master: Error: service(dns_client): command startup failed, throttling for 16 secs >>>>>>>>> dns_client: Fatal: master: service(dns_client): child 14293 returned error 84 (exec() failed) >>>>>>>>> Aki >>>>>>>>> Yes but is it being called by doveadm directly or by some other dovecot program ? If I'm going to have to go down the AppArmor route, then I would prefer if you told me what was calling it instead of me having to un-necessarily spend time doing straces ! >>>>>>>>> Also, should I be able to call dns-client directly myself ? (or is there a way to do so to enable testing ? >>>>>>>>> It is started by dovecot's master process when you connect to dns-client unix socket. You can try >>>>>>>>> socat stdio unix-connect:/var/run/dovecot/dns-client >>>>>>>>> I thought apparmor tells when something is blocked into kernel log? have you checked dmesg? >>>>>>>>> Apologies for your frustration. >>>>>>>>> Yeah nothing in dmesg. I'm still hunting around to find some log somewhere but so far silence. >>>>>>>>> "socat stdio unix-connect:/var/run/dovecot/dns-client" runs but returns nothing. Is that expected ? >>>>>>>>> When you say "dovecot's master process", so doveadm sync talks to the master process ? So in terms of apparmor I would therefore be looking at /usr/sbin/dovecot ? If that's the case, the relevant apparmor permisssions are already provided : >>>>>>>>> /{,var/}run/dovecot/ rw, >>>>>>>>> /{,var/}run/dovecot/** rw, >>>>>>>>> Laura >>>>>>>>> Do the above apparmor settings give permission to dovecot to execute >>>>>>>>> /usr/lib/dovecot/dns-client, assuming that the user under which dovecot >>>>>>>>> is running already has file system permissions to do that? >>>>>>>>> John >>>>>>>>> John, >>>>>>>>> Here's the definitive answer to your question (and anyone else thinking of pointing the finger at apparmor): >>>>>>>>> foo:/home/foo # sudo systemctl stop apparmor >>>>>>>>> foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: >>>>>>>>> dsync(foobar@example.com): Error: imapc(foobar.example.com:993): dns_lookup(foobar.example.com) failed: DNS lookup timed out >>>>>>>>> dsync(foobar@example.com): Error: Failed to initialize user: imapc: Login to foobar.example.com failed: Disconnected from server >>>>>>>>> So. Can we move on from the "blame apparmor" ? ;-) >>>>>>>>> Laura I'd suggest doing the test with a restart of dovecot in between stopping apparmor and running the doveadm command. Check your logs to see if there is no longer any message generated about not being able to execv /usr/lib/dovecot/dns-client. foo:/home/foo # sudo systemctl stop apparmor foo:/home/foo # sudo systemctl restart dovecot foo:/home/foo # doveadm -v -o mail_fsync=never backup -R -u foobar@example.com imapc: John Same again.... failed: read(/var/run/dovecot/dns-client) failed: read(size=512) failed: Connection reset by peer do you get any messages in /var/log/audit/audit.log when executing this test?
John
I did (which also lead me to the discovery that stopping the apparmor service doesn't actually do what you think it might, you still need to run 'aa-teardown').
But the answer for posterity is the following :
Put : /usr/lib/dovecot/dns-client mrix, /var/run/dovecot/dns-client mrix, Into: /etc/apparmor.d/local/usr.sbin.dovecot And: systemctl restart apparmor && systemctl restart dovecot
Thank you to all those here for your help. However I would stil like to see a way to be able to manually test 'dns-client' included in future dovecot releases.
Glad you found the problem. It would be good to report those apparmor configuration changes (and any others you find along the way) back to the package maintainer. It doesn't make sense for everyone to have to make these local appamor changes to support standard software configurations that should work out of the box.
John
participants (3)
-
Aki Tuomi
-
John Fawcett
-
Laura Smith