Bug: lmtp proxy does not quote local parts with spaces

[Stephan Bosch]

Hi,

Sorry, we're in a bit of a v2.3 merge frenzy. Much of the LMTP code will 
be replaced in v2.3, but I'll give the  older code a look as well.

This can take a while though.

Regards,

Stephan.



Op 1-11-2017 om 18:34 schreef David Zambonini:
> Hi again,
>
> I've not heard anything further regarding this bug, so I've had a look at the code.
>
> To restate the bug in a more precise way: LMTP in dovecot treats external RFC822
> email addresses in the envelope recipient and internal usernames as almost
> identical/interchangeable. This is incorrect and leads to issues when attempting
> to use director as an LMTP proxy to proxy to recipients with quoted-local parts,
> as it is issuing invalid email addresses at the LMTP protocol level (it strips
> quotes from the local part and then does not add them again when proxying). It's
> also causing issues with director username hashing.
>
> I've created a "bugfix" patch to indicate what I mean, it appears to solve the
> issue, although I do not think it is anywhere near a production ready change.
>
> 1. The first problem is that dovecot is not performing a full decode on the
> envelope recipient address when creating the username, leading to escaped
> characters left in escaped form, and is not treating it consistently, choosing
> to either strip the surrounding quotes or not depending on whether or not it
> contains an @. I've fixed this by changing the code in lmtp_unescape_address()
> [src/lmtp/commands.c] to use rfc822_parse_quoted_string().
>
> 2. This leads to the second problem where the username becomes ambiguous if the
> local-part contains an @ (regardless of whether or not the first fix is applied
> or not). I've worked around this by using strrchr() instead of strchr() on the
> username string to split the domain out in mail_user_hash()
> [src/lib-mail/mail-user-hash.c] and message_detail_address_parse()
> [src/lib-mail/message-address.c], although likely I've missed some place this
> change should be made.
>
> 3. The third problem is then re-encoding the username in the envelope recipient
> when proxying, which was not done at all. I've added a function
> lmtp_client_rfc822_escape_address(), which is similar to
> str_append_maybe_escape() to escape the address at protocol time in
> lmtp_client_send_rcpts() [src/lib-smtp/lmtp-client.c], although I suspect it
> should be done earlier, this is just a working proof.
>
> The other reason I don't believe this patch is production quality is that I have
> not examined any interaction between these changes and sieve's use of the
> envelope recipient. I'm hoping that a developer can chip in here? (hint!)
>
> (Apologies for top posting)
>
> On 30/10/2017 13:18, David Zambonini wrote:
>> On 26/10/2017 19:33, David Zambonini wrote:
>>> On 26/10/2017 18:38, Alexander Dalloz wrote:
>>>> Am 26.10.2017 um 12:20 schrieb David Zambonini:
>>>>> There seems to be a bug with RFC822 processing in ltmp proxying that
>>>>> doesn't
>>>>> quote local parts that, for example, contain spaces.
>>>> Newer related RFCs are RFC 5321 and 5322.
>>> Typo, meant to say RFC2822, which they still supercede, not that the
>>> local-part spec has changed. :)
>>>
>>>> [ ... ]
>>>>
>>>>> MAIL FROM:<test at testdomain.com>\r\n
>>>>> RCPT TO:<deemzed.uk+Junk E-mail>\r\n
>>>>>
>>>>> 501 5.5.4 Invalid.parameters\r\n
>>>> That recipient address is totally invalid. It is neither just a local
>>>> part without a domain, nor a plussed address destination.
>>>>
>>>> Check your setup with i.e.
>>>>
>>>> RCPT TO:<"Junk E-mail"@deemzed.uk>
>>>>
>>>> or
>>>>
>>>> RCPT TO:<"test+Junk E-mail"@deemzed.uk>
>>> Apologies, I was attempting to cut the config down at the time the dump
>>> was taken. Correcting (I can provide config privately, but not share to
>>> list), I still get:
>>>
>>> MAIL FROM:<test at testdomain.com>\r\n
>>> RCPT TO:<"deemzed.uk+Junk E-mail"@mailbox.localhost>\r\n
>>> DATA\r\n
>>> (etc)
>>> .\r\n
>>>
>>> 501 5.5.4 Invalid parameters\r\n
>>>
>>> QUIT\r\n
>>>
>>> from director -> dovecot LMTP network dump:
>>>
>>> I could have a look at
>>> starting to get a fix together tomorrow with an aim to providing a pull
>>> request, if it turns out there are no side-effects to treating
>>> lmtp_rcpt.address like this and you'd like an example of what I mean.
>> My apologies for not adding your address on my initial response, Alexander - not
>> sure if you noticed what I replied with or not.
>>
>> Nope, this isn't going to happen. I'm not familiar with the dovecot internals
>> but lmtp uses just the address string in the form of "full address with quotes
>> stripped from local part but otherwise not decoded" and nothing else throughout,
>> which touches on quite a bit of code. It makes it indeterminate and not always
>> possible to reassemble the original, it's a bit of a trainwreck.
>>
>> The sanest option to me seems to me to be to store a decoded local part and
>> domain in addition to the detail in mail_recipient, and keeping a now properly
>> rfc822 encoded address in sync with it. However, this would cause a deviation
>> from existing behaviour for the full original user (the quotes would be seen).
>>
>> I'm between a rock and a hard place here - at the very least I'd like this bug
>> to be officially recognised.