Re: SMTPUTF8 support

On 2019-04-05 08:57, David Bürgin via dovecot wrote:

André, are you quite sure you have it working?

In this thread someone from Open-Xchange stated that no, Dovecot doesn’t have SMTPUTF8 support implemented, and the same response was given by another Dovecot developer last September (it ‘is being considered’ was the answer then, see https://dovecot.org/pipermail/dovecot/2018-September/112887.html).

I am using LMTP to deliver mail to Dovecot from Postfix, and delivery fails with Postfix reporting: ‘SMTPUTF8 is required, but was not offered by host mail.my.org[private/dovecot-lmtp]’. I doubt that this can work without a change in Dovecot LMTP.

Anyway, I am also interested in SMTPUTF8 support, so +1 from me!

Hello David,

Well, it seems to work from one Homebox to another, for instance:

Here the server logs:

Apr 05 19:20:46 portal postfix/qmgr[21470]: 6AC3240807: from=andre@rodier.me, size=3066, nrcpt=1 (queue active) Apr 05 19:20:46 portal clamsmtpd[14542]: 10004D: from=andre@rodier.me, to=andré@homebox.space, status=CLEAN Apr 05 19:20:46 portal postfix/smtp[30761]: 1FCA540467: to=, relay=127.0.0.1[127.0.0.1]:34001, delay=1.4, delays=0.73/0.02/0.07/0.59, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 6AC3240807) Apr 05 19:20:46 portal postfix/smtpd[30763]: disconnect from localhost[127.0.0.1] ehlo=1 xforward=2 mail=1 rcpt=1 data=1 quit=1 commands=7 Apr 05 19:20:46 portal postfix/qmgr[21470]: 1FCA540467: removed Apr 05 19:20:46 portal dovecot[12813]: lmtp(30766): Connect from local Apr 05 19:20:46 portal dovecot[12813]: lmtp(andre): WGTdAo6qp1wueAAAeNB+QA: sieve: msgid=7d53-5ca7aa80-1-51005a80@31476855: stored mail into mailbox 'INBOX' Apr 05 19:20:46 portal postfix/lmtp[30765]: 6AC3240807: to=andre@homebox.space, orig_to=, relay=portal.homebox.space[private/dovecot-lmtp], delay=0.62, delays=0.59/0.01/0.01/0.01, dsn=2.0.0, status=sent (250 2.0.0 andre@homebox.space WGTdAo6qp1wueAAAeNB+QA Saved) Apr 05 19:20:46 portal dovecot[12813]: lmtp(30766): Disconnect from local: Successful quit Apr 05 19:20:46 portal postfix/qmgr[21470]: 6AC3240807: removed

And the message source:

Return-Path: andre@rodier.me Delivered-To: andre@homebox.space Received: from portal.homebox.space by portal (Dovecot) with LMTP id WGTdAo6qp1wueAAAeNB+QA for andre@homebox.space; Fri, 05 Apr 2019 19:20:46 +0000 Received: from portal.homebox.space (localhost [127.0.0.1]) by portal.homebox.space (Postfix) with ESMTP id 6AC3240807 for ; Fri, 5 Apr 2019 19:20:45 +0000 (UTC) Authentication-Results: portal.homebox.space; dkim=pass (4096-bit key; unprotected) header.d=rodier.me header.i=@rodier.me header.b="bNaIhJcQ"; dkim-atps=neutral Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=92.19.253.42; helo=osaka.rodier.me; envelope-from=andre@rodier.me; receiver=<UNKNOWN> Received: from osaka.rodier.me (host-92-19-253-42.static.as13285.net [92.19.253.42]) by portal.homebox.space (Postfix) with ESMTPS id 1FCA540467 for ; Fri, 5 Apr 2019 19:20:44 +0000 (UTC) Received: from osaka.rodier.me (localhost [127.0.0.1]) by osaka.rodier.me (Postfix) with ESMTP id 68658201DE for ; Fri, 5 Apr 2019 20:20:43 +0100 (BST) Received: from localhost (localhost [127.0.0.1]) by osaka.rodier.me (Postfix) with ESMTPA id 32C1A200CC for ; Fri, 5 Apr 2019 20:20:43 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rodier.me; s=main; t=1554492043; bh=ojnzW/eF3u3/ihaSAhQfvUWEX7Blx4fCTTVCC9mfgZE=; h=From:Date:To:Subject:From; b=bNaIhJcQtD7/IbjI/VGUIWpOk1QyYioKsgsCva8sL040L7VDMmLgfbQ5TWFwO/wub +8THBgOTWj/39FyoKQJYk0BahQ7Z4LkUrH20Jcq5yDEOC7q/xfwIwUV/sUaj6C3wLZ VGUp8Fg2oVDD+ciGEnfa2OuEC/LajvmXoDqUHHtSwTFZvfkh1nRDqxcNd/FSyvYQe8 frgBySFgzI+xTWVG7Oa07CQPG+HenqwrPyaZD0SNpHnrvH5ICdAriv2UMKgQnsWO80 aSz4EDySdwMah17kfh4u06GlXd3duQCN8hyxBBDjcI879N/+CQM3Uz1NTbIDe82Lf7 bSUJzvMczCdZFm5J84hvLd+j85pY7T9052JTcUANUQPnTc1Q3yrQFC3QuKD8Amb8VO qHR0q0OP6SYqoQ0g/moMF2FhWaQg3pCRpIEOS73Ncnnf+fJGQJnPi8G8YcLVLZCDCx RlSauA/YuF5tiIBHFLdjtlgVgwk38tT8ck+D8kS3BAB4nXBVlreCiK0vMK8HyoCpsC LMJ0e5KhBzCDXOPvSLiohn2zcmYpa9DywmWFVTZNx3INzTH7nsa40AYGFl+k+G+pe+ ej+MRDwmdOePfl3DGGEgpn0JMNiAmECe6gHA+/N1TTk5SbbiKLNh8KXzrZECWVrvUx 55kqY/LPuQEwxCnA/VZsQbDk= Content-Type: multipart/alternative; boundary="----=_=-_OpenGroupware_org_NGMime-32083-1554492042.981625-0------" From: =?utf-8?q?Andr=C3=A9_Rodier?= andre@rodier.me X-Forward: 127.0.0.1 Date: Fri, 05 Apr 2019 20:20:42 +0100 To: =?utf-8?q?andr=E9=40homebox=2Espace?=@rodier.me MIME-Version: 1.0 Message-ID: 7d53-5ca7aa80-1-51005a80@31476855 Disposition-Notification-To: andre@rodier.me Subject: test 2020 User-Agent: SOGoMail X-AV-Checked: ClamSMTP (Client IP: 127.0.0.1) Authentication-Results: portal.homebox.space; spf=temperror (portal.homebox.space: error in processing during lookup of andre@rodier.me: DNS error) smtp.mailfrom=andre@rodier.me X-Spam-Status: Yes, score=6.61 X-AV-Checked: ClamSMTP (Client IP: 127.0.0.1) X-Spam: Yes

------=_=-_OpenGroupware_org_NGMime-32083-1554492042.981625-0------ Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Length: 15

This is a test

------=_=-_OpenGroupware_org_NGMime-32083-1554492042.981625-0------ Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Length: 27

<html>This is a test</html> ------=_=-_OpenGroupware_org_NGMime-32083-1554492042.981625-0--------

However, it seems that mails from GMails are not received, and I have an error message from the postfix mailter:

Apr 5 10:16:39 portal postfix/smtpd[2566]: Anonymous TLS connection established from mail-lf1-f51.google.com[209.85.167.51]: TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits) Apr 5 10:16:39 portal policyd-spf[2574]: prepend Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.167.51; helo=mail-lf1-f51.google.com; envelope-from=andre.rodier@gmail.com; receiver=<UNKNOWN> Apr 5 10:16:39 portal postfix/smtpd[2566]: EB38240467: client=mail-lf1-f51.google.com[209.85.167.51] Apr 5 10:16:40 portal postfix/cleanup[2576]: EB38240467: message-id=CAKr9_Nv1pgDAC31cFF4YPE-mnYxSusmKsWHOCxWPSeJ2Adj8cQ@mail.gmail.com Apr 5 10:16:40 portal opendkim[15477]: EB38240467: mail-lf1-f51.google.com [209.85.167.51] not internal Apr 5 10:16:40 portal opendkim[15477]: EB38240467: not authenticated Apr 5 10:16:40 portal opendkim[15477]: EB38240467: DKIM verification successful Apr 5 10:16:40 portal opendkim[15477]: EB38240467: s=20161025 d=gmail.com SSL Apr 5 10:16:40 portal opendmarc[16548]: implicit authentication service: portal.homebox.space Apr 5 10:16:40 portal opendmarc[16548]: EB38240467: gmail.com pass Apr 5 10:16:40 portal postfix/cleanup[2576]: EB38240467: milter-reject: END-OF-MESSAGE from mail-lf1-f51.google.com[209.85.167.51]: 4.7.1 Try again later; from=andre.rodier@gmail.com to= proto=ESMTP helo= Apr 5 10:16:40 portal postfix/smtpd[2566]: disconnect from mail-lf1-f51.google.com[209.85.167.51] ehlo=2 starttls=1 mail=1 rcpt=1 data=0/1 quit=1 commands=6/7

Thanks to you, I have another bug to chase with gmail and postfix cleanup. I could ask Google to use HomeBox, but I am not sure they would agree ;-)

Regarding your issue, there might be many causes.

I remember one, with the dovecot, for the Quota plugin that failed to lookup the user, I had to add "quota_status_nouser = DUNNO", so it does not fail when the user is not found (if I understood well).

Another note, I am using LDAP lookups, _and_ a specific schema for unicode email addresses. Otherwise, this would not work:

root@portal:/etc/postfix# cat ldap-aliases.cf # Basic LDAP settings for postfix server_host = ldap://ldap.homebox.space/ start_tls = yes

search_base = dc=homebox,dc=space search_scope = sub

# Allow advanced LDAP search with the recipient delimiter query_filter = (|(mail=%s)(intlMailAddr=%s)) result_attribute = mail

# Force support for UTF8 version = 3

Thanks to this, ldap lookup is unicode compliant:

root@portal:/etc/postfix# postmap -q andré@homebox.space ldap:/etc/postfix/ldap-aliases.cf andre@homebox.space

The LDAP schema for internationalised email addresses:

# The attribute type 'intlMailAddr' is defined for storing SMTPUTF8 # compliant addresses [RFC6530] # https://tools.ietf.org/html/draft-stroeder-mailboxrelatedobject-07

dn: cn=intlMailAddr,cn=schema,cn=config objectClass: olcSchemaConfig cn: intlMailAddr olcAttributeTypes: ( 1.3.6.1.4.1.5427.1.389.4.18 NAME 'intlMailAddr' DESC 'Internationalized Email Address' EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 ) olcObjectClasses: ( 1.3.6.1.4.1.5427.1.389.6.9 NAME 'mailboxRelatedObject' DESC 'Associated RFC 5321 mailbox for any entry' AUXILIARY MAY ( displayName $ mail $ intlMailAddr ) )

As I wrote earlier, All the software in the chain should be UTF-8 ready

-- André Rodier HomeBox: https://github.com/progmaticltd/homebox