2.3.13 broken submission relay smtp parser

[Stephan Bosch]

On 09/06/2021 07:57, Tony Hain wrote:
> I have a new install of dovecot 2.3.13, along with exim 4.94, in an Azure
> hosted FreeBSD 12.2 VM. I have been running exim on local hardware with
> FreeBSD for 15+ years, but dovecot and Azure are a new "learning
> experience". I am getting an error response in dovecot.log when trying to
> use the submission relay function, which is apparently new in 2.3...  It
> would appear the parser is either broken or has a character set limitation
> that no other smtp implementation has. I finally gave up trying to figure
> out what I might have done wrong in setting up exim and pointed dovecot at
> mailjet and got the same error.
>
> Jun 08 19:39:42
> submission(testing at dispatch.tndh.net)<89538><lOfAL0zEFNmsOCrh>: Warning:
> smtp-client: conn in-v3.mailjet.com:587 (104.199.96.85:587) [1]: Received
> invalid EHLO response line: Unexpected character in EHLO keyword
> Jun 08 19:39:42
> submission(testing at dispatch.tndh.net)<89538><lOfAL0zEFNmsOCrh>: Warning:
> smtp-client: conn in-v3.mailjet.com:587 (104.199.96.85:587) [1]: Received
> invalid EHLO response line: Unexpected character in EHLO keyword
>
> I didn't try the mailjet path with telnet, but I had done that earlier with
> the local exim server and I can't see any invalid characters, even in the
> tcpdump pcap file.
>
> Jun 08 10:49:42
> submission(testing at dispatch.tndh.net)<29791><j8NnyETEqV2sOCq3>: Warning:
> smtp-client: conn 127.0.0.1:58 [1]: Received invalid EHLO response line:
> Unexpected character in EHLO keyword
> # telnet localhost 58
> Trying 127.0.0.1...
> Connected to localhost.
> Escape character is '^]'.
> 220 secure smtp server
> ehlo dovecot.tndh.net
> 250-exim.tndh.net Hello dovecot.tndh.net [127.0.0.1]
> 250-SIZE 536870912
> 250-8BITMIME
> 250-VRFY
> 250-PIPELINING
> 250-X_PIPE_CONNECT
> 250-AUTH CRAM-MD5
> 250-CHUNKING
> 250-SMTPUTF8
> 250 HELP

There is  your problem. We should probably allow this in Dovecot (seen 
this problem before), but the underscore in the X_PIPE_CONNECT 
capability is not allowed in SMTP.

Regards,

Stephan.

>
> This might be some confusion about starttls on the mailjet path, but if that
> is true the error message is wrong; and it wouldn't be true for the local
> exim open smtp port. If it really is smtp, it would be most helpful if the
> error message actually reported what string it is taking issue with.
>
> I have the dovecot-sysreport, but I am not encouraged about sending it when
> stdout presented:
> # dovecot-sysreport
> Gathering configurations ...
> grep: The -P option is not supportedgrep:
> The -P option is not supported
> grep: The -P option is not supported
> Gathering system informations ...
> Creating archive ...
> All done! Please report file dovecot-sysreport-TNDH-mail-1623209001.tar.gz
> Removing temp files at /tmp/tmp.kphlba44 ...
> #
>
> While dovecot -n stdout presented the line:
> ssl_key = # hidden, use -P to show it
>
> expecting people to put sensitive configuration on a public mail list
> without knowing what the tool is including is a challenge, but when the tool
> is errantly using the command line option that is also used for exposing the
> private data by a related tool, it is even less likely that I want to do
> that. While the dovecot -n option did hide passwords, it did not hide the
> username associated with that. I will put dovecot -n (redacted) here, but
> until I have time to see exactly what the sysreport included, I am not
> releasing that.
>
> # 2.3.13 (89f716dc2): /usr/local/etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.13 (cdd19fe3)
> # OS: FreeBSD 12.2-RELEASE-p4 amd64  ufs
> # Hostname: TNDH-mail.g4msrgoph2uevil3ys5jvbbpza.jx.internal.cloudapp.net
> auth_debug = yes
> auth_debug_passwords = yes
> auth_verbose = yes
> debug_log_path = /var/log/dovecot-debug.log
> disable_plaintext_auth = no
> first_valid_uid = 220
> hostname = dispatch.tndh.net
> imap_idle_notify_interval = 20 mins
> info_log_path = /var/log/dovecot-info.log
> last_valid_uid = 220
> log_debug = (event=* AND cat=*)
> log_path = /var/log/dovecot.log
> login_greeting = tndh.net Mailer Server Ready ...
> login_trusted_networks = 127.0.0.1 10.0.0.4
> mail_debug = yes
> mail_location = maildir:/usr/local/var/dovecot/vhosts/%d/%n
> mail_plugins = mail_log notify notify_status
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
> copy include variables body enotify environment mailbox date index ihave
> duplicate mime foreverypart extracttext
> namespace inbox {
>    inbox = yes
>    location =
>    mailbox Drafts {
>      auto = subscribe
>      special_use = \Drafts
>    }
>    mailbox Junk {
>      auto = subscribe
>      special_use = \Junk
>    }
>    mailbox Sent {
>      auto = subscribe
>      special_use = \Sent
>    }
>    mailbox Trash {
>      auto = subscribe
>      special_use = \Trash
>    }
>    mailbox virtual/Flagged {
>      auto = subscribe
>      special_use = \Flagged
>    }
>    prefix =
>    separator = /
>    type = private
> }
> passdb {
>    args = username_format=%n /usr/local/var/dovecot/db/%d/passwd
>    driver = passwd-file
> }
> plugin {
>    expire = Trash
>    mail_home = /usr/local/var/dovecot/vhosts/%d/%n
>    mail_log_events = delete undelete expunge copy mailbox_delete
> mailbox_rename
>    mail_log_fields = uid box msgid size
>    recipient_delimiter = +
>    sieve = /usr/local/var/dovecot/vhosts/%d/%n/sieve/.dovecot.sieve
>    sieve_after = /usr/local/var/dovecot/vhosts/%d/%n/sieve/sieve-after.d
>    sieve_before = /usr/local/var/dovecot/vhosts/%d/%n/sieve/sieve-before.d
>    sieve_dir = /usr/local/var/dovecot/vhosts/%d/%n/sieve
>    sieve_global_path = /usr/local/var/dovecot/vhosts/sieve/default.sieve
> }
> pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
> protocols = imap pop3 lmtp submission
> service auth-worker {
>    user = vmail
> }
> service imap-login {
>    inet_listener imap {
>      port = 143
>    }
>    inet_listener imaps {
>      port = 993
>      ssl = yes
>    }
> }
> service stats {
>    unix_listener stats-writer {
>      mode = 0666
>    }
> }
> service submission-login {
>    inet_listener submission {
>      port = 465
>      ssl = yes
>    }
> }
> ssl_cert = </usr/local/etc/dovecot/ssl/certs/dovecot.pem
> ssl_key = # hidden, use -P to show it
> submission_relay_host = in-v3.mailjet.com
> submission_relay_password = # hidden, use -P to show it
> submission_relay_port = 587
> submission_relay_rawlog_dir = /var/log
> submission_relay_ssl = starttls
> submission_relay_user = **-as-if-I-want-this-on-a-public-list-**
> userdb {
>    args = username_format=%n /usr/local/var/dovecot/db/%d/passwd
>    driver = passwd-file
> }
> verbose_ssl = yes
> protocol lmtp {
>    mail_fsync = optimized
>    mail_plugins = mail_log notify notify_status sieve
> }
> protocol imap {
>    mail_max_userip_connections = 10
>    mail_plugins = mail_log notify notify_status imap_sieve
> }
> protocol pop3 {
>    mail_max_userip_connections = 10
>    mail_plugins = mail_log notify notify_status
> }
> protocol lda {
>    mail_fsync = optimized
>    mail_plugins = mail_log notify notify_status sieve
> }
>