Mail addresses with quotes + Postfix

Philipp Berger

1 Mar 2018 1 Mar '18

1:07 p.m.

Dear all,

I have a working setup with Postfix + Dovecot, storing users in a MySQL table.

I ran into problems setting up and using a mail address like a"@"b@mydomain.tld, which by RFC should be valid, but leads to problems in Dovecot. From my debugging, I can see that on the lookup, Dovecot replaces %n with "a" (quotes added by me) and %d with "b@mydomain.tld" (quotes added by me). So the original quotes are gone, and the domain/user split is incorrect.

I am not sure of Postfix passes the address without quotes to Dovecot or whether Dovecot can not handle quotes correctly, but I was hoping that someone with more insight into Dovecot could help me out here.

System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot 2.2.27-3+deb9u1 Postfix master.cf for Dovecot: dovecot unix - n n - - pipe flags=DRhu user=virtual:virtual argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop}

I am grateful for any hints :)

Thank you in advance, kind regards,

Philipp

Show replies by date

LuKreme

1 Mar 1 Mar

3:48 p.m.

On Mar 1, 2018, at 05:07, Philipp Berger newsletters@philippberger.de wrote:

...

mail address like a"@"b@mydomain.tld, which by RFC should be valid

Shouldn't it be "a@b"@mydoamin.tld ?

-- My main job is trying to come up with new and innovative and effective ways to reject even more mail. I'm up to about 97% now.

Philipp Berger

4:13 p.m.

Of course you are right! It should either be "a@b"@mydomain.tld or a."@".b@mydomain.tld, both would be valid.

Sadly, it does not change the outcome. Even with a pure "@"@mydomain.tld, Dovecot splits it into the empty string user (%n) and @mydomain.tld as the domain part (%d).

Am 01-Mar-18 um 16:48 schrieb LuKreme:

...

On Mar 1, 2018, at 05:07, Philipp Berger newsletters@philippberger.de wrote:

...
mail address like a"@"b@mydomain.tld, which by RFC should be valid Shouldn't it be "a@b"@mydoamin.tld ?

LuKreme

4 Mar 4 Mar

1:27 p.m.

On Mar 1, 2018, at 08:13, Philipp Berger newsletters@philippberger.de wrote:

...

Sadly, it does not change the outcome. Even with a pure "@"@mydomain.tld, Dovecot splits it into the empty string user (%n) and @mydomain.tld as the domain part (%d).

That's a shame. Doe sit handle other quoted user names? I know it breaks on UTF-8 specific characters, but I think that is documented as unsupported.

RFC-5322 3.2.4 and 3.4.1 are pretty clear, a quoted scoring is to be treated as an atom.

“The locally interpreted string is either a quoted-string or a dot-atom.”

That said, I'm not sure who supports a username portion in this form, so it's probably an edge-case on an edge-case.

-- My main job is trying to come up with new and innovative and effective ways to reject even more mail. I'm up to about 97% now.

Stephan Bosch

8:12 p.m.

Op 3/1/2018 om 2:07 PM schreef Philipp Berger:

...

Dear all,

I have a working setup with Postfix + Dovecot, storing users in a MySQL table.

I ran into problems setting up and using a mail address like a"@"b@mydomain.tld, which by RFC should be valid, but leads to problems in Dovecot. From my debugging, I can see that on the lookup, Dovecot replaces %n with "a" (quotes added by me) and %d with "b@mydomain.tld" (quotes added by me). So the original quotes are gone, and the domain/user split is incorrect.

I am not sure of Postfix passes the address without quotes to Dovecot or whether Dovecot can not handle quotes correctly, but I was hoping that someone with more insight into Dovecot could help me out here.

System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot 2.2.27-3+deb9u1 Postfix master.cf for Dovecot: dovecot unix - n n - - pipe flags=DRhu user=virtual:virtual argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop}

I am grateful for any hints :)

Thank you in advance, kind regards,

Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address parsing. Dovecot v2.3 should do a better job. I am still not sure if this can be fixed well for Dovecot v2.2.

Regards,

Stephan.

Philipp Berger

6 Mar 6 Mar

1:34 p.m.

I upgraded to Dovecot 2.3.0.1 as advised, but it still seems broken. In the Postfix log I now see: Mar 6 13:49:03 myhost amavis[7165]: (07165-10) K00VtLRHdrYw FWD from -> <"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37 Mar 6 13:49:03 myhost amavis[7165]: (07165-10) Passed CLEAN {RelayedInbound}, [someIp]:11439 [someIp] -> <"@"@mydomain.tld>, Queue-ID: D27792DA167C, Message-ID: <22b95756-e95e-86cf-219c-3b603f7587c8@mydomain.tld>, mail_id: K00VtLRHdrYw, Hits: -3, size: 7472, queued_as: B8CA22DA1B37, 1863 ms Mar 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C: to=<@@mydomain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=2, delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37) Mar 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37: to=<@@mydomain.tld>, relay=dovecot, delay=0.2, delays=0.11/0/0/0.09, dsn=5.1.1, status=bounced (user unknown) Mar 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80: message-id=<20180306124903.EA37E2DA1F80@mydomain.tld> Mar 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37: sender non-delivery notification: EA37E2DA1F80 Mar 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80: from=<>, size=9830, nrcpt=1 (queue active) Mar 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37: removed Mar 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80: to=, relay=dovecot, delay=0.07, delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command line usage error. Command output: lda: Fatal: Invalid -f parameter: Missing domain ) I can see in my MySQL Log that Dovecot queried at the same time with %n = "" (empty) and %d = "@mydomain.tld". So now the "@" is dropped entirely. What does the "Fatal: Invalid -f parameter: Missing domain" tell us? Do I need to change the way Postfix calls Dovecot? Add more quotes? :D Kind regards, Philipp Am 04-Mar-18 um 21:12 schrieb Stephan Bosch:

...

Op 3/1/2018 om 2:07 PM schreef Philipp Berger:

...
Dear all,

I have a working setup with Postfix + Dovecot, storing users in a MySQL table.

I ran into problems setting up and using a mail address like a"@"b@mydomain.tld, which by RFC should be valid, but leads to problems in Dovecot. From my debugging, I can see that on the lookup, Dovecot replaces %n with "a" (quotes added by me) and %d with "b@mydomain.tld" (quotes added by me). So the original quotes are gone, and the domain/user split is incorrect.

I am not sure of Postfix passes the address without quotes to Dovecot or whether Dovecot can not handle quotes correctly, but I was hoping that someone with more insight into Dovecot could help me out here.

System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot 2.2.27-3+deb9u1 Postfix master.cf for Dovecot: dovecot unix - n n - - pipe flags=DRhu user=virtual:virtual argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop}

I am grateful for any hints :)

Thank you in advance, kind regards, Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address parsing. Dovecot v2.3 should do a better job. I am still not sure if this can be fixed well for Dovecot v2.2.

Regards,

Stephan.

Stephan Bosch

2:08 p.m.

Op 6-3-2018 om 14:34 schreef Philipp Berger:

...

I upgraded to Dovecot 2.3.0.1 as advised, but it still seems broken. In the Postfix log I now see:

Mar 6 13:49:03 myhost amavis[7165]: (07165-10) K00VtLRHdrYw FWD from -> <"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37 Mar 6 13:49:03 myhost amavis[7165]: (07165-10) Passed CLEAN {RelayedInbound}, [someIp]:11439 [someIp] -> <"@"@mydomain.tld>, Queue-ID: D27792DA167C, Message-ID: <22b95756-e95e-86cf-219c-3b603f7587c8@mydomain.tld>, mail_id: K00VtLRHdrYw, Hits: -3, size: 7472, queued_as: B8CA22DA1B37, 1863 ms Mar 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C: to=<@@mydomain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=2, delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37) Mar 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37: to=<@@mydomain.tld>, relay=dovecot, delay=0.2, delays=0.11/0/0/0.09, dsn=5.1.1, status=bounced (user unknown) Mar 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80: message-id=<20180306124903.EA37E2DA1F80@mydomain.tld> Mar 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37: sender non-delivery notification: EA37E2DA1F80 Mar 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80: from=<>, size=9830, nrcpt=1 (queue active) Mar 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37: removed Mar 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80: to=, relay=dovecot, delay=0.07, delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command line usage error. Command output: lda: Fatal: Invalid -f parameter: Missing domain )

I can see in my MySQL Log that Dovecot queried at the same time with %n = "" (empty) and %d = "@mydomain.tld". So now the "@" is dropped entirely.

What does the "Fatal: Invalid -f parameter: Missing domain" tell us? Do I need to change the way Postfix calls Dovecot? Add more quotes? :D

Can you check what Postfix is feeding to Dovecot exactly (i.e., whether it is properly escaped)? Maybe wrap dovecot-lda in some shell script and see what is passed. Regards, Stephan.

...

Kind regards, Philipp

Am 04-Mar-18 um 21:12 schrieb Stephan Bosch:

...
Op 3/1/2018 om 2:07 PM schreef Philipp Berger:

...
Dear all,

I have a working setup with Postfix + Dovecot, storing users in a MySQL table.

I ran into problems setting up and using a mail address like a"@"b@mydomain.tld, which by RFC should be valid, but leads to problems in Dovecot. From my debugging, I can see that on the lookup, Dovecot replaces %n with "a" (quotes added by me) and %d with "b@mydomain.tld" (quotes added by me). So the original quotes are gone, and the domain/user split is incorrect.

I am not sure of Postfix passes the address without quotes to Dovecot or whether Dovecot can not handle quotes correctly, but I was hoping that someone with more insight into Dovecot could help me out here.

System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot 2.2.27-3+deb9u1 Postfix master.cf for Dovecot: dovecot unix - n n - - pipe flags=DRhu user=virtual:virtual argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop}

I am grateful for any hints :)

Thank you in advance, kind regards, Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address parsing. Dovecot v2.3 should do a better job. I am still not sure if this can be fixed well for Dovecot v2.2.

Regards,

Stephan.

Philipp Berger

7 Mar 7 Mar

12:46 p.m.

I wrapped the LDA command in a script. I can see that Postfix passes "@@mydomain.tld" as the -d argument, without quotes. I then adapted the script to specifically replace this address with "@"@mydomain.tld, but this results in the following error message by Dovecot: auth: Info: userdb(?): Username character disallowed by auth_username_chars: 0x22 (username: "@"@mydomain.tld) So what would be the appropriate quoting/setting for this address? Kind regards, Philipp Am 06-Mar-18 um 15:08 schrieb Stephan Bosch:

...

Op 6-3-2018 om 14:34 schreef Philipp Berger:

...
I upgraded to Dovecot 2.3.0.1 as advised, but it still seems broken. In the Postfix log I now see:

Mar 6 13:49:03 myhost amavis[7165]: (07165-10) K00VtLRHdrYw FWD from -> <"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37 Mar 6 13:49:03 myhost amavis[7165]: (07165-10) Passed CLEAN {RelayedInbound}, [someIp]:11439 [someIp] -> <"@"@mydomain.tld>, Queue-ID: D27792DA167C, Message-ID: <22b95756-e95e-86cf-219c-3b603f7587c8@mydomain.tld>, mail_id: K00VtLRHdrYw, Hits: -3, size: 7472, queued_as: B8CA22DA1B37, 1863 ms Mar 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C: to=<@@mydomain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=2, delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37) Mar 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37: to=<@@mydomain.tld>, relay=dovecot, delay=0.2, delays=0.11/0/0/0.09, dsn=5.1.1, status=bounced (user unknown) Mar 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80: message-id=<20180306124903.EA37E2DA1F80@mydomain.tld> Mar 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37: sender non-delivery notification: EA37E2DA1F80 Mar 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80: from=<>, size=9830, nrcpt=1 (queue active) Mar 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37: removed Mar 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80: to=, relay=dovecot, delay=0.07, delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command line usage error. Command output: lda: Fatal: Invalid -f parameter: Missing domain )

I can see in my MySQL Log that Dovecot queried at the same time with %n = "" (empty) and %d = "@mydomain.tld". So now the "@" is dropped entirely.

What does the "Fatal: Invalid -f parameter: Missing domain" tell us? Do I need to change the way Postfix calls Dovecot? Add more quotes? :D

Can you check what Postfix is feeding to Dovecot exactly (i.e., whether it is properly escaped)? Maybe wrap dovecot-lda in some shell script and see what is passed.

Regards,

Stephan.

...
Kind regards, Philipp

Am 04-Mar-18 um 21:12 schrieb Stephan Bosch:

...
Op 3/1/2018 om 2:07 PM schreef Philipp Berger:

...
Dear all,

I have a working setup with Postfix + Dovecot, storing users in a MySQL table.

I ran into problems setting up and using a mail address like a"@"b@mydomain.tld, which by RFC should be valid, but leads to problems in Dovecot. From my debugging, I can see that on the lookup, Dovecot replaces %n with "a" (quotes added by me) and %d with "b@mydomain.tld" (quotes added by me). So the original quotes are gone, and the domain/user split is incorrect.

I am not sure of Postfix passes the address without quotes to Dovecot or whether Dovecot can not handle quotes correctly, but I was hoping that someone with more insight into Dovecot could help me out here.

System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot 2.2.27-3+deb9u1 Postfix master.cf for Dovecot: dovecot unix - n n - - pipe flags=DRhu user=virtual:virtual argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop}

I am grateful for any hints :)

Thank you in advance, kind regards, Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address parsing. Dovecot v2.3 should do a better job. I am still not sure if this can be fixed well for Dovecot v2.2.

Regards,

Stephan.

Reio Remma

12:50 p.m.

Hello! Maybe experiment with auth_username_chars: # List of allowed characters in username. If the user-given username contains # a character not listed in here, the login automatically fails. This is just # an extra check to make sure user can't exploit any potential quote escaping # vulnerabilities with SQL/LDAP databases. If you want to allow all characters, # set this value to empty. #auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ Good luck, Reio On 07.03.18 14:46, Philipp Berger wrote:

...

I wrapped the LDA command in a script. I can see that Postfix passes "@@mydomain.tld" as the -d argument, without quotes. I then adapted the script to specifically replace this address with "@"@mydomain.tld, but this results in the following error message by Dovecot:

auth: Info: userdb(?): Username character disallowed by auth_username_chars: 0x22 (username: "@"@mydomain.tld)

So what would be the appropriate quoting/setting for this address?

Kind regards, Philipp

Am 06-Mar-18 um 15:08 schrieb Stephan Bosch:

...
...
I upgraded to Dovecot 2.3.0.1 as advised, but it still seems broken. In the Postfix log I now see:

Mar 6 13:49:03 myhost amavis[7165]: (07165-10) K00VtLRHdrYw FWD from -> <"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37 Mar 6 13:49:03 myhost amavis[7165]: (07165-10) Passed CLEAN {RelayedInbound}, [someIp]:11439 [someIp] -> <"@"@mydomain.tld>, Queue-ID: D27792DA167C, Message-ID: <22b95756-e95e-86cf-219c-3b603f7587c8@mydomain.tld>, mail_id: K00VtLRHdrYw, Hits: -3, size: 7472, queued_as: B8CA22DA1B37, 1863 ms Mar 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C: to=<@@mydomain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=2, delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37) Mar 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37: to=<@@mydomain.tld>, relay=dovecot, delay=0.2, delays=0.11/0/0/0.09, dsn=5.1.1, status=bounced (user unknown) Mar 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80: message-id=<20180306124903.EA37E2DA1F80@mydomain.tld> Mar 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37: sender non-delivery notification: EA37E2DA1F80 Mar 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80: from=<>, size=9830, nrcpt=1 (queue active) Mar 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37: removed Mar 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80: to=, relay=dovecot, delay=0.07, delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command line usage error. Command output: lda: Fatal: Invalid -f parameter: Missing domain )

I can see in my MySQL Log that Dovecot queried at the same time with %n = "" (empty) and %d = "@mydomain.tld". So now the "@" is dropped entirely.

What does the "Fatal: Invalid -f parameter: Missing domain" tell us? Do I need to change the way Postfix calls Dovecot? Add more quotes? :D Can you check what Postfix is feeding to Dovecot exactly (i.e., whether it is properly escaped)? Maybe wrap dovecot-lda in some shell

Op 6-3-2018 om 14:34 schreef Philipp Berger: script and see what is passed.

Regards,

Stephan.

...
Kind regards, Philipp

Am 04-Mar-18 um 21:12 schrieb Stephan Bosch:

...
Op 3/1/2018 om 2:07 PM schreef Philipp Berger:

...
Dear all,

I have a working setup with Postfix + Dovecot, storing users in a MySQL table.

I ran into problems setting up and using a mail address like a"@"b@mydomain.tld, which by RFC should be valid, but leads to problems in Dovecot. From my debugging, I can see that on the lookup, Dovecot replaces %n with "a" (quotes added by me) and %d with "b@mydomain.tld" (quotes added by me). So the original quotes are gone, and the domain/user split is incorrect.

I am not sure of Postfix passes the address without quotes to Dovecot or whether Dovecot can not handle quotes correctly, but I was hoping that someone with more insight into Dovecot could help me out here.

System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot 2.2.27-3+deb9u1 Postfix master.cf for Dovecot: dovecot unix - n n - - pipe flags=DRhu user=virtual:virtual argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop}

I am grateful for any hints :)

Thank you in advance, kind regards, Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address parsing. Dovecot v2.3 should do a better job. I am still not sure if this can be fixed well for Dovecot v2.2.

Regards,

Stephan.

Stephan Bosch

2:40 p.m.

Op 7-3-2018 om 13:46 schreef Philipp Berger:

...

I wrapped the LDA command in a script. I can see that Postfix passes "@@mydomain.tld" as the -d argument, without quotes. I then adapted the script to specifically replace this address with "@"@mydomain.tld, but this results in the following error message by Dovecot:

auth: Info: userdb(?): Username character disallowed by auth_username_chars: 0x22 (username: "@"@mydomain.tld)

So what would be the appropriate quoting/setting for this address?

Adding " to auth_username_chars would help. But why are you trying to accept such incredibly horrible email-addresses/usernames? Regards, Stephan.

...

Kind regards, Philipp

Am 06-Mar-18 um 15:08 schrieb Stephan Bosch:

...
...
I upgraded to Dovecot 2.3.0.1 as advised, but it still seems broken. In the Postfix log I now see:

Mar 6 13:49:03 myhost amavis[7165]: (07165-10) K00VtLRHdrYw FWD from -> <"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37 Mar 6 13:49:03 myhost amavis[7165]: (07165-10) Passed CLEAN {RelayedInbound}, [someIp]:11439 [someIp] -> <"@"@mydomain.tld>, Queue-ID: D27792DA167C, Message-ID: <22b95756-e95e-86cf-219c-3b603f7587c8@mydomain.tld>, mail_id: K00VtLRHdrYw, Hits: -3, size: 7472, queued_as: B8CA22DA1B37, 1863 ms Mar 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C: to=<@@mydomain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=2, delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37) Mar 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37: to=<@@mydomain.tld>, relay=dovecot, delay=0.2, delays=0.11/0/0/0.09, dsn=5.1.1, status=bounced (user unknown) Mar 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80: message-id=<20180306124903.EA37E2DA1F80@mydomain.tld> Mar 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37: sender non-delivery notification: EA37E2DA1F80 Mar 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80: from=<>, size=9830, nrcpt=1 (queue active) Mar 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37: removed Mar 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80: to=, relay=dovecot, delay=0.07, delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command line usage error. Command output: lda: Fatal: Invalid -f parameter: Missing domain )

I can see in my MySQL Log that Dovecot queried at the same time with %n = "" (empty) and %d = "@mydomain.tld". So now the "@" is dropped entirely.

What does the "Fatal: Invalid -f parameter: Missing domain" tell us? Do I need to change the way Postfix calls Dovecot? Add more quotes? :D Can you check what Postfix is feeding to Dovecot exactly (i.e., whether it is properly escaped)? Maybe wrap dovecot-lda in some shell

Op 6-3-2018 om 14:34 schreef Philipp Berger: script and see what is passed.

Regards,

Stephan.

...
Kind regards, Philipp

Am 04-Mar-18 um 21:12 schrieb Stephan Bosch:

...
Op 3/1/2018 om 2:07 PM schreef Philipp Berger:

...
Dear all,

I have a working setup with Postfix + Dovecot, storing users in a MySQL table.

I ran into problems setting up and using a mail address like a"@"b@mydomain.tld, which by RFC should be valid, but leads to problems in Dovecot. From my debugging, I can see that on the lookup, Dovecot replaces %n with "a" (quotes added by me) and %d with "b@mydomain.tld" (quotes added by me). So the original quotes are gone, and the domain/user split is incorrect.

I am not sure of Postfix passes the address without quotes to Dovecot or whether Dovecot can not handle quotes correctly, but I was hoping that someone with more insight into Dovecot could help me out here.

System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot 2.2.27-3+deb9u1 Postfix master.cf for Dovecot: dovecot unix - n n - - pipe flags=DRhu user=virtual:virtual argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop}

I am grateful for any hints :)

Thank you in advance, kind regards, Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address parsing. Dovecot v2.3 should do a better job. I am still not sure if this can be fixed well for Dovecot v2.2.

Regards,

Stephan.

Stephan Bosch

2:50 p.m.

Op 7-3-2018 om 15:40 schreef Stephan Bosch:

...

Op 7-3-2018 om 13:46 schreef Philipp Berger:

...
I wrapped the LDA command in a script. I can see that Postfix passes "@@mydomain.tld" as the -d argument, without quotes. I then adapted the script to specifically replace this address with "@"@mydomain.tld, but this results in the following error message by Dovecot:

auth: Info: userdb(?): Username character disallowed by auth_username_chars: 0x22 (username: "@"@mydomain.tld)

So what would be the appropriate quoting/setting for this address?

Adding " to auth_username_chars would help.

But why are you trying to accept such incredibly horrible email-addresses/usernames?

Also, -d is used to provide the username. For -d, you could leave out the double quotes and provide the full e-mail address--including the required quotes--to the -r argument. It all depends on what your userdb expects. Regards, Stephan.

...

...
Am 06-Mar-18 um 15:08 schrieb Stephan Bosch:

...
...
I upgraded to Dovecot 2.3.0.1 as advised, but it still seems broken. In the Postfix log I now see:

Mar 6 13:49:03 myhost amavis[7165]: (07165-10) K00VtLRHdrYw FWD from -> <"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37 Mar 6 13:49:03 myhost amavis[7165]: (07165-10) Passed CLEAN {RelayedInbound}, [someIp]:11439 [someIp] -> <"@"@mydomain.tld>, Queue-ID: D27792DA167C, Message-ID: <22b95756-e95e-86cf-219c-3b603f7587c8@mydomain.tld>, mail_id: K00VtLRHdrYw, Hits: -3, size: 7472, queued_as: B8CA22DA1B37, 1863 ms Mar 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C: to=<@@mydomain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=2, delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37) Mar 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37: to=<@@mydomain.tld>, relay=dovecot, delay=0.2, delays=0.11/0/0/0.09, dsn=5.1.1, status=bounced (user unknown) Mar 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80: message-id=<20180306124903.EA37E2DA1F80@mydomain.tld> Mar 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37: sender non-delivery notification: EA37E2DA1F80 Mar 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80: from=<>, size=9830, nrcpt=1 (queue active) Mar 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37: removed Mar 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80: to=, relay=dovecot, delay=0.07, delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command line usage error. Command output: lda: Fatal: Invalid -f parameter: Missing domain )

I can see in my MySQL Log that Dovecot queried at the same time with %n = "" (empty) and %d = "@mydomain.tld". So now the "@" is dropped entirely.

What does the "Fatal: Invalid -f parameter: Missing domain" tell us? Do I need to change the way Postfix calls Dovecot? Add more quotes? :D Can you check what Postfix is feeding to Dovecot exactly (i.e., whether it is properly escaped)? Maybe wrap dovecot-lda in some shell

Op 6-3-2018 om 14:34 schreef Philipp Berger: script and see what is passed.

Regards,

Stephan.

...
Kind regards, Philipp

Am 04-Mar-18 um 21:12 schrieb Stephan Bosch:

...
...
Dear all,

I have a working setup with Postfix + Dovecot, storing users in a MySQL table.

I ran into problems setting up and using a mail address like a"@"b@mydomain.tld, which by RFC should be valid, but leads to problems in Dovecot. From my debugging, I can see that on the lookup, Dovecot replaces %n with "a" (quotes added by me) and %d with "b@mydomain.tld" (quotes added by me). So the original quotes are gone, and the domain/user split is incorrect.

I am not sure of Postfix passes the address without quotes to Dovecot or whether Dovecot can not handle quotes correctly, but I was hoping that someone with more insight into Dovecot could help me out here.

System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot 2.2.27-3+deb9u1 Postfix master.cf for Dovecot: dovecot unix - n n - - pipe flags=DRhu user=virtual:virtual argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop}

I am grateful for any hints :)

Thank you in advance, kind regards, Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address

Op 3/1/2018 om 2:07 PM schreef Philipp Berger: parsing. Dovecot v2.3 should do a better job. I am still not sure if this can be fixed well for Dovecot v2.2.

Regards,

Stephan.

Philipp Berger

5:28 p.m.

Interesting, the documentation and the build-in help of dovecot-lda do not mention -r exists :) case 'r': /* final recipient address */ if (smtp_address_parse_path(ctx.pool, optarg, SMTP_ADDRESS_PARSE_FLAG_ALLOW_LOCALPART | SMTP_ADDRESS_PARSE_FLAG_BRACKETS_OPTIONAL, &final_rcpt_to, &errstr) < 0) { i_fatal_status(EX_USAGE, "Invalid -a parameter: %s", errstr); Btw: Should the error message not read "Invalid -r parameter: %s" instead? I think the problem is the different handling in Postfix and Dovecot: Postfix seems to automatically strip the quotes out of the username, at least for SQL lookup (but does the splitting correctly, which Dovecot 2.3 does NOT). Dovecot does not strip quotes. What is the recommended (i.e. endorsed by its developers) way for Dovecot to deal with Mail addresses containing Quotation marks? Strip them out? Keep them? Since they are disabled by default (auth_username_chars), it seems like stripping, but then the splitting of local part and domain does not work correctly. Am I really the first one to play around with this? Does it make sense to prepare patches for example for fixing the splitting? On a side note: Has support for UTF8 in local parts already landed in Dovecot? I could not find up-to-date information on this topic. Kind regards, Philipp Am 07-Mar-18 um 15:50 schrieb Stephan Bosch:

...

Op 7-3-2018 om 15:40 schreef Stephan Bosch:

...
Op 7-3-2018 om 13:46 schreef Philipp Berger:

...
I wrapped the LDA command in a script. I can see that Postfix passes "@@mydomain.tld" as the -d argument, without quotes. I then adapted the script to specifically replace this address with "@"@mydomain.tld, but this results in the following error message by Dovecot:

auth: Info: userdb(?): Username character disallowed by auth_username_chars: 0x22 (username: "@"@mydomain.tld)

So what would be the appropriate quoting/setting for this address?

Adding " to auth_username_chars would help.

But why are you trying to accept such incredibly horrible email-addresses/usernames?

Also, -d is used to provide the username. For -d, you could leave out the double quotes and provide the full e-mail address--including the required quotes--to the -r argument.

It all depends on what your userdb expects.

Regards,

Stephan.

...
...
Am 06-Mar-18 um 15:08 schrieb Stephan Bosch:

...
...
I upgraded to Dovecot 2.3.0.1 as advised, but it still seems broken. In the Postfix log I now see:

Mar 6 13:49:03 myhost amavis[7165]: (07165-10) K00VtLRHdrYw FWD from -> <"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37 Mar 6 13:49:03 myhost amavis[7165]: (07165-10) Passed CLEAN {RelayedInbound}, [someIp]:11439 [someIp] -> <"@"@mydomain.tld>, Queue-ID: D27792DA167C, Message-ID: <22b95756-e95e-86cf-219c-3b603f7587c8@mydomain.tld>, mail_id: K00VtLRHdrYw, Hits: -3, size: 7472, queued_as: B8CA22DA1B37, 1863 ms Mar 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C: to=<@@mydomain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=2, delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37) Mar 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37: to=<@@mydomain.tld>, relay=dovecot, delay=0.2, delays=0.11/0/0/0.09, dsn=5.1.1, status=bounced (user unknown) Mar 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80: message-id=<20180306124903.EA37E2DA1F80@mydomain.tld> Mar 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37: sender non-delivery notification: EA37E2DA1F80 Mar 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80: from=<>, size=9830, nrcpt=1 (queue active) Mar 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37: removed Mar 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80: to=, relay=dovecot, delay=0.07, delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command line usage error. Command output: lda: Fatal: Invalid -f parameter: Missing domain )

I can see in my MySQL Log that Dovecot queried at the same time with %n = "" (empty) and %d = "@mydomain.tld". So now the "@" is dropped entirely.

What does the "Fatal: Invalid -f parameter: Missing domain" tell us? Do I need to change the way Postfix calls Dovecot? Add more quotes? :D Can you check what Postfix is feeding to Dovecot exactly (i.e., whether it is properly escaped)? Maybe wrap dovecot-lda in some shell

Op 6-3-2018 om 14:34 schreef Philipp Berger: script and see what is passed.

Regards,

Stephan.

...
Kind regards, Philipp

Am 04-Mar-18 um 21:12 schrieb Stephan Bosch:

...
Op 3/1/2018 om 2:07 PM schreef Philipp Berger: > Dear all, > > I have a working setup with Postfix + Dovecot, storing users in a > MySQL > table. > > I ran into problems setting up and using a mail address like > a"@"b@mydomain.tld, which by RFC should be valid, but leads to > problems > in Dovecot. From my debugging, I can see that on the lookup, > Dovecot > replaces %n with "a" (quotes added by me) and %d with > "b@mydomain.tld" > (quotes added by me). So the original quotes are gone, and the > domain/user split is incorrect. > > I am not sure of Postfix passes the address without quotes to > Dovecot or > whether Dovecot can not handle quotes correctly, but I was > hoping that > someone with more insight into Dovecot could help me out here. > > System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot > 2.2.27-3+deb9u1 > Postfix master.cf for Dovecot: > dovecot unix - n n - - pipe > flags=DRhu user=virtual:virtual > argv=/usr/lib/dovecot/dovecot-lda -f > ${sender} -d ${user}@${nexthop} > > I am grateful for any hints :) > > Thank you in advance, kind regards, Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address parsing. Dovecot v2.3 should do a better job. I am still not sure if this can be fixed well for Dovecot v2.2.

Regards,

Stephan.

Stephan Bosch

6:13 p.m.

Op 3/7/2018 om 6:28 PM schreef Philipp Berger:

...

Interesting, the documentation and the build-in help of dovecot-lda do not mention -r exists :)

case 'r':

/* final recipient address */

if (smtp_address_parse_path(ctx.pool, optarg,

SMTP_ADDRESS_PARSE_FLAG_ALLOW_LOCALPART |

SMTP_ADDRESS_PARSE_FLAG_BRACKETS_OPTIONAL,

&final_rcpt_to, &errstr) < 0) {

i_fatal_status(EX_USAGE,

"Invalid -a parameter: %s", errstr);

Btw: Should the error message not read "Invalid -r parameter: %s" instead?

Yes, both problems fixed. Thanks. Full documentation is here: https://wiki2.dovecot.org/LDA

...

I think the problem is the different handling in Postfix and Dovecot: Postfix seems to automatically strip the quotes out of the username, at least for SQL lookup (but does the splitting correctly, which Dovecot 2.3 does NOT).

Can you specify which split is wrong in your opinion? If the MTA strips necessary quotes before it is passed to dovecot- lda, Dovecot will reject the address. That is not wrong.

...

Dovecot does not strip quotes.

That depends on what Dovecot is working operating on. When it is provided as a username (-d), it will not strip quotes. If it is an email address (-a, -r, -f), it will strip quotes if they're unnecessary. Internally, it uses an unquoted, split representation (localpart and domain separately). Once it needs to compose the split addresses back into a string, quotes are added only when needed.

...

What is the recommended (i.e. endorsed by its developers) way for Dovecot to deal with Mail addresses containing Quotation marks? Strip them out? Keep them? Since they are disabled by default (auth_username_chars), it seems like stripping, but then the splitting of local part and domain does not work correctly. Am I really the first one to play around with this? Does it make sense to prepare patches for example for fixing the splitting?

Only if it is broken or breaks compatibility for existing setups in a bad way (e.g. Dovecot allows some address syntax violations commonly seen for some Asian ISPs).

...

On a side note: Has support for UTF8 in local parts already landed in Dovecot? I could not find up-to-date information on this topic.

Not yet. We will be tending to that pretty soon though. Regards, Stephan.

...

Am 07-Mar-18 um 15:50 schrieb Stephan Bosch:

...
Op 7-3-2018 om 15:40 schreef Stephan Bosch:

...
Op 7-3-2018 om 13:46 schreef Philipp Berger:

...
I wrapped the LDA command in a script. I can see that Postfix passes "@@mydomain.tld" as the -d argument, without quotes. I then adapted the script to specifically replace this address with "@"@mydomain.tld, but this results in the following error message by Dovecot:

auth: Info: userdb(?): Username character disallowed by auth_username_chars: 0x22 (username: "@"@mydomain.tld)

So what would be the appropriate quoting/setting for this address?

Adding " to auth_username_chars would help.

But why are you trying to accept such incredibly horrible email-addresses/usernames?

Also, -d is used to provide the username. For -d, you could leave out the double quotes and provide the full e-mail address--including the required quotes--to the -r argument.

It all depends on what your userdb expects.

Regards,

Stephan.

...
...
Am 06-Mar-18 um 15:08 schrieb Stephan Bosch:

...
...
I upgraded to Dovecot 2.3.0.1 as advised, but it still seems broken. In the Postfix log I now see:

Mar 6 13:49:03 myhost amavis[7165]: (07165-10) K00VtLRHdrYw FWD from -> <"@"@mydomain.tld>, BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37 Mar 6 13:49:03 myhost amavis[7165]: (07165-10) Passed CLEAN {RelayedInbound}, [someIp]:11439 [someIp] -> <"@"@mydomain.tld>, Queue-ID: D27792DA167C, Message-ID: <22b95756-e95e-86cf-219c-3b603f7587c8@mydomain.tld>, mail_id: K00VtLRHdrYw, Hits: -3, size: 7472, queued_as: B8CA22DA1B37, 1863 ms Mar 6 13:49:03 myhost postfix/smtp[2329]: D27792DA167C: to=<@@mydomain.tld>, relay=127.0.0.1[127.0.0.1]:10024, delay=2, delays=0.15/0.01/0/1.9, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as B8CA22DA1B37) Mar 6 13:49:03 myhost postfix/pipe[2282]: B8CA22DA1B37: to=<@@mydomain.tld>, relay=dovecot, delay=0.2, delays=0.11/0/0/0.09, dsn=5.1.1, status=bounced (user unknown) Mar 6 13:49:03 myhost postfix/cleanup[2280]: EA37E2DA1F80: message-id=<20180306124903.EA37E2DA1F80@mydomain.tld> Mar 6 13:49:04 myhost postfix/bounce[2334]: B8CA22DA1B37: sender non-delivery notification: EA37E2DA1F80 Mar 6 13:49:04 myhost postfix/qmgr[21911]: EA37E2DA1F80: from=<>, size=9830, nrcpt=1 (queue active) Mar 6 13:49:04 myhost postfix/qmgr[21911]: B8CA22DA1B37: removed Mar 6 13:49:04 myhost postfix/pipe[2282]: EA37E2DA1F80: to=, relay=dovecot, delay=0.07, delays=0.05/0/0/0.02, dsn=5.3.0, status=bounced (command line usage error. Command output: lda: Fatal: Invalid -f parameter: Missing domain )

I can see in my MySQL Log that Dovecot queried at the same time with %n = "" (empty) and %d = "@mydomain.tld". So now the "@" is dropped entirely.

What does the "Fatal: Invalid -f parameter: Missing domain" tell us? Do I need to change the way Postfix calls Dovecot? Add more quotes? :D Can you check what Postfix is feeding to Dovecot exactly (i.e., whether it is properly escaped)? Maybe wrap dovecot-lda in some shell

Op 6-3-2018 om 14:34 schreef Philipp Berger: script and see what is passed.

Regards,

Stephan.

...
Kind regards, Philipp

Am 04-Mar-18 um 21:12 schrieb Stephan Bosch: > Op 3/1/2018 om 2:07 PM schreef Philipp Berger: >> Dear all, >> >> I have a working setup with Postfix + Dovecot, storing users in a >> MySQL >> table. >> >> I ran into problems setting up and using a mail address like >> a"@"b@mydomain.tld, which by RFC should be valid, but leads to >> problems >> in Dovecot. From my debugging, I can see that on the lookup, >> Dovecot >> replaces %n with "a" (quotes added by me) and %d with >> "b@mydomain.tld" >> (quotes added by me). So the original quotes are gone, and the >> domain/user split is incorrect. >> >> I am not sure of Postfix passes the address without quotes to >> Dovecot or >> whether Dovecot can not handle quotes correctly, but I was >> hoping that >> someone with more insight into Dovecot could help me out here. >> >> System Info: Debian 9, Postifx 3.1.6-0+deb9u1, Dovecot >> 2.2.27-3+deb9u1 >> Postfix master.cf for Dovecot: >> dovecot unix - n n - - pipe >> flags=DRhu user=virtual:virtual >> argv=/usr/lib/dovecot/dovecot-lda -f >> ${sender} -d ${user}@${nexthop} >> >> I am grateful for any hints :) >> >> Thank you in advance, kind regards, > Yeah, Dovecot v2.2 is doing some funky stuff with SMTP address > parsing. > Dovecot v2.3 should do a better job. I am still not sure if this > can be > fixed well for Dovecot v2.2. > > Regards, > > Stephan.

2402

Age (days ago)

2408

Last active (days ago)

List overview

12 comments

4 participants

participants (4)

LuKreme
Philipp Berger
Reio Remma
Stephan Bosch

Mail addresses with quotes + Postfix

tags

participants (4)