- dovecot - dovecot.org

OAUTH2 tokeninfo is doing a GET instead of a POST request
by Alexander Leidinger 26 Oct '23

26 Oct '23

Hi, I try to setup oauth2 authentication with dovecot 2.3.21. The debug log of dovecot shows that it tries to do a HTTP GET request to the tokeninfo url with the token appended to the end of the URL. This gives a 404 error. The openidconnect server I use (keycloak) tells that this API endpoint conforms to https://openid.net/specs/openid-connect-core-1_0.html#TokenEndpoint which specifies that the request has to be a HTTP POST request. So dovecot is trying do to something (GET request) which the OIDC specification does not agree with (shall be POST request). Here is the dovecot debug log of it: ---snip--- Oct 17 12:11:19 imap dovecot[81589]: auth: Debug: http-client[1]: request [Req1: GET https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/tokeneyJhb…<rest_omitted>...: Submitted (requests left=1) [...] Oct 17 12:11:19 imap dovecot[81589]: auth: Debug: oauth2.domain.tld: SSL: where=0x1001, ret=1: SSLv3/TLS read server session ticket Oct 17 12:11:19 imap dovecot[81589]: auth: Debug: oauth2.domain.tld: where=0x1002, ret=1: SSL negotiation finished successfully Oct 17 12:11:19 imap dovecot[81589]: auth: Debug: oauth2.domain.tld: SSL: where=0x1001, ret=1: SSL negotiation finished successfully Oct 17 12:11:19 imap syslogd: last message repeated 1 times Oct 17 12:11:19 imap dovecot[81589]: auth: Debug: oauth2.domain.tld: SSL: where=0x1001, ret=1: SSLv3/TLS read server session ticket Oct 17 12:11:19 imap dovecot[81589]: auth: Debug: oauth2.domain.tld: SSL: where=0x1002, ret=1: SSL negotiation finished successfully Oct 17 12:11:19 imap dovecot[81589]: auth: Debug: http-client: conn <IP>:443 [1]: Got 404 response for request [Req1: GET https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/tokeneyJhb…<rest_omitted> ---snip--- My passdb config (only showing the oauth part): ---snip--- passdb { driver = oauth2 mechanisms = oauthbearer xoauth2 args = /usr/local/etc/dovecot/auth-oauth2.token.conf.ext } passdb { driver = oauth2 mechanisms = plain args = /usr/local/etc/dovecot/auth-oauth2.plain.conf.ext } ---snip--- auth-oauth2.token.conf.ext: ---snip--- openid_configuration_url = https://oauth2.domain.tld/realms/MyRealm/.well-known/openid-configuration tokeninfo_url = https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/token introspection_url = https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/token/intr… introspection_mode = post active_attribute = active active_value = true client_id = myid client_secret = mysecret use_grant_password = no debug = yes username_attribute = email pass_attrs = pass=%{oauth2:access_token} ---snip--- auth-oauth2.plain.conf.ext: ---snip--- openid_configuration_url = https://oauth2.domain.tld/realms/MyRealm/.well-known/openid-configuration #tokeninfo_url = https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/token introspection_url = https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/token/intr… introspection_mode = post active_attribute = active active_value = true client_id = myid client_secret = mysecret use_grant_password = yes debug = yes username_attribute = email pass_attrs = host=<IP of webmail> proxy=y proxy_mech=xoauth2 pass=%{oauth2:access_token} ---snip--- On https://doc.dovecot.org/configuration_manual/authentication/oauth2/ I can not find any way to tell that the tokeninfo url shall do a POST request instead of a GET request. I found something on reddit how to make it work with keycloak, but this seems to be a workaround, and not a proper fix... The first comment at https://www.reddit.com/r/selfhosted/comments/omwb2j/any_one_get_dovecot_key… makes this work for me. The working but not really up to the OIDC spec dovecot config is: auth-oauth2.token.conf.ext: ---snip--- openid_configuration_url = https://oauth2.domain.tld/realms/MyRealm/.well-known/openid-configuration #tokeninfo_url = https://oauth2.domain.tld/realms/MyRealm/Leidinger/protocol/openid-connect/… tokeninfo_url = https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/userinfo?t… introspection_url = https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/token/intr… introspection_mode = auth #active_attribute = active #active_value = true client_id = myid client_secret = mysecret use_grant_password = no #debug = yes username_attribute = email pass_attrs = pass=%{oauth2:access_token} ---snip--- auth-oauth2.plain.conf.ext: ---snip--- openid_configuration_url = https://oauth2.domain.tld/realms/MyRealm/.well-known/openid-configuration #tokeninfo_url = https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/token tokeninfo_url = https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/userinfo?t… introspection_url = https://oauth2.domain.tld/realms/MyRealm/protocol/openid-connect/token/intr… introspection_mode = auth #active_attribute = active #active_value = true client_id = myid client_secret = mysecret use_grant_password = yes #debug = yes username_attribute = email pass_attrs = host=<IP of webmail> proxy=y proxy_mech=xoauth2 pass=%{oauth2:access_token} ---snip--- Bye, Alexander. -- http://www.Leidinger.net Alexander(a)Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild(a)FreeBSD.org : PGP 0x8F31830F9F2772BF

2 8

Is it possible to show different mechanism according to client IP ?
by Philippe MARASSE 26 Oct '23

26 Oct '23

Hello, Say I'm wondering if it's possible to show 3 mechanisms for clients in private network (XOAUTH2 / PLAIN / EXTERNAL) and only 2 to others (only OAUTH / EXTERNAL) ? I've seen allow_nets extra field for args, but it seems that's not working with ldap passdb. Regards. -- Philippe MARASSE Responsable pôle Infrastructures - DSIO Centre Hospitalier Henri Laborit CS 10587 - 370 avenue Jacques Cœur 86021 Poitiers Cedex Tel : 05.49.44.57.19 Hello, Say I'm wondering if it's possible to show 3 mechanisms for clients in private network (XOAUTH2 / PLAIN / EXTERNAL) and only 2 to others (only OAUTH / EXTERNAL) ? I've seen allow_nets extra field for args, but it seems that's not working with ldap passdb. Regards. -- Philippe MARASSE Responsable pôle Infrastructures - DSIO Centre Hospitalier Henri Laborit CS 10587 - 370 avenue Jacques Cœur 86021 Poitiers Cedex Tel : 05.49.44.57.19

2 1

Unclear output from force-resync.
by Justin Wheeler 25 Oct '23

25 Oct '23

Hi, After a hard drive issue from my VPS provider, one of my customers' mailboxes has been corrupted, and has been complaining about index sync issues. So I am attempting a force-resync. The command I used was: doveadm -v force-resync -u user(a)mailbox.com '*' The output it's been giving doesn't look anything like a 'progress counter', so I have no idea if it's just losing its mind, or if it's actually achieving anything. The mailbox is 20G, so theoretically it's just working its way through it, but I can't tell. The output looks like: doveadm(user(a)mailbox.com): Warning: fscking index file /home/mail/mailbox.com/user/mdbox/storage/dovecot.map.index doveadm(user(a)mailbox.com): Warning: mdbox /home/mail/mailbox.com/user/mdbox/storage: rebuilding indexes doveadm(user(a)mailbox.com): Warning: Transaction log file /home/mail/mailbox.com/user/mdbox/storage/dovecot.map.index.log was locked for 90 seconds (mdbox storage rebuild) doveadm(user(a)mailbox.com): Warning: fscking index file /home/mail/mailbox.com/user/mdbox/storage/dovecot.map.index doveadm(user(a)mailbox.com): Warning: fscking index file /home/mail/mailbox.com/user/mdbox/storage/dovecot.map.index doveadm(user(a)mailbox.com): Warning: mdbox /home/mail/mailbox.com/user/mdbox/storage: rebuilding indexes doveadm(user(a)mailbox.com): Warning: Transaction log file /home/mail/mailbox.com/user/mdbox/storage/dovecot.map.index.log was locked for 90 seconds (mdbox storage rebuild) It's just this over and over and over and over. Does this mean it's working, or that something else is going on? fuser /home/mail/mailbox.com/user/mdbox/storage/dovecot.map.index.log shows only the PID of the force-resync, and ls -l /proc/pid/fd shows it has a bunch of index and index.log and index.cache files open, as well as scrolling through mdbox/storage/m.{digits} a bunch, and strace -fp on the process shows a boatload of pread64 on those m.{digits} files, so I *think* it's working, but I can't tell if it's just restarting after the above messages and doing the same thing over and over again. I've locked the mailbox so there's no mail delivery being attempted on it, and changed its password and kicked out any logged in users on the mailbox, so the only process anywhere near that entire directory structure is the force-resync. Thanks, Justin

1 0

dovecot lmtp and smtputf8
by Kamil Jońca 25 Oct '23

25 Oct '23

Does dovecot handle smtputf8? Last articles regarding this are several years old. Anything changed? KJ

2 2

dovecot 2.0 supports EC private key?
by Marc 24 Oct '23

24 Oct '23

Does dovecot 2.0 supports EC private key?

2 1

Question: ETA OpenSSL 3.0 Support
by Niklas Meyer 23 Oct '23

23 Oct '23

Hello everyone, i stumbled across some serious problems with decrypting of e-mails if you use a Dovecot installation with OpenSSL 3.X so i started digging. It turns out that Dovecot is not OpenSSL 3.X ready yet, at least if you compile it yourself. The Debian Project adapted a OpenSSL 3.X patch to their repo version of Dovecot to make it run with Debian 12. Alpine Linux has not added a Workaround Patch for Dovecot yet. This is actually really bad if you want to Upgrade your Dovecot Server to a Distro which has OpenSSL 3.X as the new standard and you are not using their patched Repo Versions but instead compiling dovecot by yourself or install it from your package repos (which are not Debian 12 compatible also, which is probably because of OpenSSL 3.X i presume?) However though i think that this compatibility should be added soon as more and more Distros are using OpenSSL 3.X. So: Is there a approx. ETA when the OpenSSL Compatibility the devs worked/are working on is coming? Thank you for your answers. Kind regards from Germany Niklas Meyer aka. DerLinkman

2 3

Confusing documentation for 'address/listen' setting
by Bruno Victal 21 Oct '23

21 Oct '23

Hi, The 'address' setting in the “Service configuration” of the Configuration Manual [1] says: Space separated list of IP addresses / host names to listen on. […] Defaults to listen setting. I'm finding this part confusing as it seems that by “listen setting” it is referring to the 'listen' setting in the “Dovecot Core Manual” [2] which is described as: A comma-separated list of IP addresses or […] which leads me to suspect that either “listen setting” refers to something else (what?) or there's some inconsistency with the documentation or perhaps a bug/foot-gun here. [1]: <https://doc.dovecot.org/configuration_manual/service_configuration/#address> [2]: <https://doc.dovecot.org/settings/core/#core_setting-listen> Regards, -- Bruno.

1 0

Re: Dovecot behind Reverse Proxy and IP families
by Arnaud Cordier 20 Oct '23

20 Oct '23

On 10/13/23 06:55, Arnaud Cordier wrote: > Hi Justas, > > Thank you for your answer. Unfortunately, I already have a dual stack > cluster with an IPv6 assigned to the pod: > > 3: eth0@if22: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1430 qdisc > noqueue state UP qlen 1000 > link/ether c6:90:d2:cf:b8:77 brd ff:ff:ff:ff:ff:ff > inet 10.42.4.52/32 scope global eth0 > valid_lft forever preferred_lft forever > inet6 fd23:23:42:0:251d:192a:bb7f:a1f4/128 scope global > valid_lft forever preferred_lft forever > inet6 fe80::c490:d2ff:fecf:b877/64 scope link > valid_lft forever preferred_lft forever > > Any additional idea? > > --- > Arnaud Cordier > --------------- > CryptID > Freelance cybersecurity engineer - Pentester - Red teamer > Tel: +33 (0)6 61 44 82 04 > On 10/12/23 22:54, Justas Umbrasas wrote: >> >> Hey, >> >> >> we are running a similar setup without an issue (not on k8s). I >> believe the reason is that the remote dovecot pod is only binding on >> IPv4 and has no IPv6 configured. This is kind of a hack, however you >> could try binding the pod to both IPv4 and IPv6 and see if it works. >> >> >> Best, >> >> Justas Umbrasas >> >> >> On 2023-10-12 20:25, Arnaud Cordier via dovecot wrote: >> >>> Dear all, >>> >>> I am running Dovecot 2.3.20 in a Kubernetes cluster. The Dovecot >>> container is exposed behind a Traefik load-balancer, which acts just >>> like HAProxy would, using the Proxy Protocol in order to allow >>> Dovecot to be aware of the real source IP of the client, as follows: >>> >>> Internet <-> Traefik load-balancer <---- Proxy protocol ----> Dovecot >>> >>> My setup works well over IPv4. >>> >>> Oct 12 19:11:33 imap-login: Info: Login: user=<arnaud@xxx>, >>> method=PLAIN, rip=<my_ipv4_public_address> lip=10.42.4.21, >>> mpid=1369531, session=<xxxx> >>> >>> Howevever, if the client originates from an IPv6 address, Dovecot >>> does not seem to take the real IPv6 origin into account and fails. >>> >>> Oct 12 19:09:04 imap-login: Error: haproxy(v1): Client disconnected: >>> Proxied local address is invalid (address=`10.42.4.21', rip=10.42.4.21) >>> >>> A network capture shows that the proxy protocol indeed sends the >>> real IPv6 IP, that should be taken into account by dovecot: >>> >>> ......&.PROXY TCP6 2001:<rest_of_my_public_ip> 10.42.4.21 20200 993 >>> >>> The error seems to be raised in >>> "core/src/lib-master/master-service-haproxy.c" line 553: >>> >>> 551 if (net_addr2ip(*fields, &local_ip) < 0 || >>> 552 local_ip.family != family) { >>> 553 e_error(hpconn->event, "Client disconnected: " >>> 554 "Proxied local address >>> is invalid " >>> 555 "(address=`%s', >>> rip=%s)", str_sanitize(*fields, 64), >>> 556 net_ip2addr(real_remote_ip)); >>> 557 return -1; >>> 558 } >>> 559 fields++; >>> >>> I suppose that the issue is related to the fact that the client >>> originates from an IPv6 then the reverse proxy forwards the >>> connection to Dovecot over IPv4, which results in the >>> "local_ip.family != family" condition to be true. >>> >>> Why is such a restriction implemented ? >>> >>> It is to be noted that I implemented the exact same load-balancer >>> architecture for postfix, which successfully handles an IPv6 real IP >>> over an IPv4 connection. >>> >>> I temporarily fixed the issue by setting an IPv4 only DNS record for >>> my imaps domain name, however, do you think that a fix could be >>> implemented in Dovecot? >>> >>> Kind regards, >>> >>> _______________________________________________ >>> dovecot mailing list -- dovecot(a)dovecot.org >>> To unsubscribe send an email to dovecot-leave(a)dovecot.org >> >> On 10/13/23 06:55, Arnaud Cordier wrote: Hi Justas, Thank you for your answer. Unfortunately, I already have a dual stack cluster with an IPv6 assigned to the pod: 3: eth0@if22: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1430 qdisc noqueue state UP qlen 1000 link/ether c6:90:d2:cf:b8:77 brd ff:ff:ff:ff:ff:ff inet 10.42.4.52/32 scope global eth0 valid_lft forever preferred_lft forever inet6 fd23:23:42:0:251d:192a:bb7f:a1f4/128 scope global valid_lft forever preferred_lft forever inet6 fe80::c490:d2ff:fecf:b877/64 scope link valid_lft forever preferred_lft forever Any additional idea? --- Arnaud Cordier --------------- CryptID Freelance cybersecurity engineer - Pentester - Red teamer Tel: +33 (0)6 61 44 82 04 On 10/12/23 22:54, Justas Umbrasas wrote: Hey, we are running a similar setup without an issue (not on k8s). I believe the reason is that the remote dovecot pod is only binding on IPv4 and has no IPv6 configured. This is kind of a hack, however you could try binding the pod to both IPv4 and IPv6 and see if it works. Best, Justas Umbrasas On 2023-10-12 20:25, Arnaud Cordier via dovecot wrote: Dear all, I am running Dovecot 2.3.20 in a Kubernetes cluster. The Dovecot container is exposed behind a Traefik load-balancer, which acts just like HAProxy would, using the Proxy Protocol in order to allow Dovecot to be aware of the real source IP of the client, as follows: Internet <-> Traefik load-balancer <---- Proxy protocol ----> Dovecot My setup works well over IPv4. Oct 12 19:11:33 imap-login: Info: Login: user=<arnaud@xxx>, method=PLAIN, rip=<my_ipv4_public_address> lip=10.42.4.21, mpid=1369531, session=<xxxx> Howevever, if the client originates from an IPv6 address, Dovecot does not seem to take the real IPv6 origin into account and fails. Oct 12 19:09:04 imap-login: Error: haproxy(v1): Client disconnected: Proxied local address is invalid (address=`10.42.4.21', rip=10.42.4.21) A network capture shows that the proxy protocol indeed sends the real IPv6 IP, that should be taken into account by dovecot: ......&.PROXY TCP6 2001:<rest_of_my_public_ip> 10.42.4.21 20200 993 The error seems to be raised in "core/src/lib-master/master-service- haproxy.c" line 553: 551 if (net_addr2ip(*fields, &local_ip) < 0 || 552 local_ip.family != family) { 553 e_error(hpconn->event, "Client disconnected: " 554 "Proxied local address is invalid " 555 "(address=`%s', rip=%s)", str_sanitize(*fields, 64), 556 net_ip2addr(real_remote_ip)); 557 return -1; 558 } 559 fields++; I suppose that the issue is related to the fact that the client originates from an IPv6 then the reverse proxy forwards the connection to Dovecot over IPv4, which results in the "local_ip.family != family" condition to be true. Why is such a restriction implemented ? It is to be noted that I implemented the exact same load-balancer architecture for postfix, which successfully handles an IPv6 real IP over an IPv4 connection. I temporarily fixed the issue by setting an IPv4 only DNS record for my imaps domain name, however, do you think that a fix could be implemented in Dovecot? Kind regards, _______________________________________________ dovecot mailing list -- dovecot(a)dovecot.org To unsubscribe send an email to dovecot-leave(a)dovecot.org

2 1

submission server relay to localhost
by Michael Grant 20 Oct '23

20 Oct '23

I'm trying to set up dovecot-submission server which will listen on external ports 465 (SSL) and 587 (StartTLS) and relay mail to sendmail waiting on localhost port 587. I have dovecot submission listening on the external ports and sendmail listening on the localhost port. I want dovecot-submission doing the authentication on the external ports because sendmail doesn't use the /etc/dovecot/users file. I can authenticate to dovecot: auth: Debug: client in: CONT<hidden> auth: Debug: passwd-file(mgrant(a)top.networkguild.org,217.35.29.56,<blablabla>): Performing passdb lookup auth: Debug: passwd-file(mgrant(a)top.networkguild.org,217.35.29.56,<blablabla>): lookup: user=mgrant(a)top.networkguild.org file=/etc/dovecot/users auth: Debug: passwd-file(mgrant(a)top.networkguild.org,217.35.29.56,<blablabla>): Finished passdb lookup auth: Debug: auth(mgrant(a)top.networkguild.org,217.35.29.56,<blablabla>): Auth request finished auth: Debug: client passdb out: OK 1 user=mgrant(a)top.networkguild.org But in the sendmail logs, dovecot *is* trying to authenticate and it's trying to use a username that sendmail can't look up in the password file: top sm-mta[1012721]: 39KCg8h31012721: --- 220 top.networkguild.org ESMTP Sendmail 8.17.2/8.17.2/Debian-1~bpo12+1; Fri, 20 Oct 2023 12:42:08 GMT; (No UCE/UBE) logging access from: localhost(OK)-localhost [IPv6:0:0:0:0:0:0:0:1] top sm-mta[1012721]: 39KCg8h31012721: <-- EHLO top.networkguild.org top sm-mta[1012721]: 39KCg8h31012721: --- 250-top.networkguild.org Hello localhost [IPv6:0:0:0:0:0:0:0:1], pleased to meet you top sm-mta[1012721]: 39KCg8h31012721: --- 250-ENHANCEDSTATUSCODES top sm-mta[1012721]: 39KCg8h31012721: --- 250-PIPELINING top sm-mta[1012721]: 39KCg8h31012721: --- 250-EXPN top sm-mta[1012721]: 39KCg8h31012721: --- 250-VERB top sm-mta[1012721]: 39KCg8h31012721: --- 250-8BITMIME top sm-mta[1012721]: 39KCg8h31012721: --- 250-SIZE top sm-mta[1012721]: 39KCg8h31012721: --- 250-AUTH DIGEST-MD5 CRAM-MD5 top sm-mta[1012721]: 39KCg8h31012721: --- 250-STARTTLS top sm-mta[1012721]: 39KCg8h31012721: --- 250-DELIVERBY top sm-mta[1012721]: 39KCg8h31012721: --- 250 HELP top sm-mta[1012721]: 39KCg8h31012721: <-- MAIL FROM:<mgrant(a)top.networkguild.org> AUTH=mgrant(a)top.networkguild.org top sm-mta[1012721]: 39KCg8h31012721: --- 530 5.7.0 Authentication required top dovecot: submission(mgrant(a)top.networkguild.org)<1012719><blablabla>: Error: Relay server requires authentication: 530 5.7.0 Authentication required top dovecot: submission(mgrant(a)top.networkguild.org)<1012719><blablabla>: Disconnected: Internal error occurred. Refer to server log for more information. (unfinished MAIL command) (state=MAIL FROM) in=41 out=121 top sm-mta[1012721]: 39KCg8h31012721: <-- QUIT How do I stop dovecot from proposing AUTH to the relay server? Once I am authenticated via dovecot, the relay which is only available on localhost, doesn't need to authenticate. It should be as if bin-mail is submitting to localhost. I tried setting up a user with a password but no shell and configure this into submission_relay_master_user and submission_relay_password but this leads to other problems. Dovecot wants to do PLAIN auth, so I then enable starttls, but then the ssl certificate doesn't match because I'm connecting to localhost, not top.networkguild.org. So it seems clear, the relay should a) not auth, and b) not do ssl. Note that this is not an open relay, it's only open on the loopback interface. Michael Grant

1 1

Extract body preview in lua notify script
by Filip Hanes 20 Oct '23

20 Oct '23

Hi, I am trying to extract short text preview from mail body in function dovecot_lua_notify_end_txn. I want to send notification about new email with perex included. How can it be done? Is it possible? thanks -- Filip Hanes

2 1