I got it to connect to my MySQL server using SSL. After looking into the code and understanding the option file (Google helped), I added the following option:
/etc/dovecot/mysql-client.cnf: [client] ssl-verify-server-cert=off
/etc/dovecot/dovecot.conf: mysql host.docker.internal { ssl = yes user = ... password = ... dbname = ... option_file = /etc/dovecot/mysql-client.cnf }
My MySQL server automatically generates certificates on startup, e.g. /var/lib/mysql/ca.pem
Those generated files contain a NUL char for whatever reason, which dovecot complains about. So I copied the file and removed the NUL char at the end. Then I set ssl_client_cert_file (within mysql) and pointed it to the new ca.pem. Then the error simply changed into "TLS/SSL error: unsupported". Funny enough, I could successfully test the connection within the docker container using openssl and the same ca.pem file dovecot was using. Open SSL reported that everything was fine.
So even though I couldn't make it verify the certificate, I could make it to connect which is enough for me at the moment.
Thank you all for your help!