Question about login_log_format_elements in a proxy environment

[John Fawcett]

On 08/12/2020 01:01, Adi Pircalabu wrote:
> On 08-12-2020 10:33, Adi Pircalabu wrote:
>> On 08-12-2020 9:41, John Fawcett wrote:
>>> On 07/12/2020 23:22, John Fawcett wrote:
>>>> On 07/12/2020 23:09, Adi Pircalabu wrote:
>>>>> On 08-12-2020 3:13, John Fawcett wrote:
>>>>>> On 07/12/2020 06:02, Adi Pircalabu wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have a Dovecot proxy setup with several proxy machines (currently
>>>>>>> running 2.3.11.3) in front of the real Dovecot servers (2.3.10.1)
>>>>>>> storing the mailboxes. "doveconf -a | egrep lip" returns:
>>>>>>> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l
>>>>>>> mpid=%e
>>>>>>> %c session=<%{session}>
>>>>>>>
>>>>>>> In the real server maillog I'm expecting to have "lip" replaced
>>>>>>> with
>>>>>>> the IP address of the proxy. It works as expected for imap-login
>>>>>>> processes, however for pop3-login processes I still see the real
>>>>>>> server IP instead of the proxy IP. Ideas?
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>> Hi Adi
>>>>>>
>>>>>> in general people want to get the original ip not the proxied ip.
>>>>>> The
>>>>>> proxying of the original ip is done by a different method for imap
>>>>>> and pop3
>>>>>>
>>>>>> https://wiki.dovecot.org/Design/ParameterForwarding
>>>>>>
>>>>>> However, unless I'm reading this wrongly, both methods are
>>>>>> affected by
>>>>>> trusted_networks settings. I guess for people to help further, you'd
>>>>>> need to give more info your configuration settings.
>>>>> Thanks John. login_trusted_networks, if this is the setting you're
>>>>> referring to, lists the proxy IPs. I'd have thought, by having this
>>>>> setting on the real servers, the proxy IP will be logged by both IMAP
>>>>> and POP3 login processes, but it appears it isn't the case. It works
>>>>> for IMAP, not for POP3.
>>>>> The reason I need the proxy IP in the "lip" instead of the local
>>>>> IP in
>>>>> the real server mail log is that I need to filter certain
>>>>> connections,
>>>>> both IMAP and POP3, that are coming directly into the real server IP.
>>>>> By capturing the IMAP & POP3 traffic on the real servers and matching
>>>>> the results to the mail log entries I *should* be able to tell what
>>>>> mail accounts from which remote IP addresses are coming in via the
>>>>> proxies and which ones are coming into the real servers directly.
>>>>> Hope
>>>>> that makes sense.
>>>>> Cheers,
>>>>>
>>>> The way I read it is that by specifing login_trusted_networks the
>>>> proxy
>>>> ip can be overwritten by the real ip. I think that's the opposite of
>>>> what you need.
>>>>
>>>> I can't throw any light on why that is not working for imap but is
>>>> working for pop3. But as you don't want the overwriting, maybe you
>>>> should try without login_trusted_networks.
>>>>
>>>> John
>>>>
>>> You're probably not getting the real ip logged for imap despite having
>>> login_trusted_networks due to the default for imap_id_retain on the
>>> proxies.
>>>
>>> John
>>
>> (Aki cc-ed)
>> Thanks. I actually need login_trusted_networks on the real servers so
>> that the real server has access to the client IP address, aka "rip" in
>> the log entry. What I need is consistent values for "lip" field for
>> both IMAP and POP3 login processes. Looking at
>> https://doc.dovecot.org/configuration_manual/proxy_settings/ there's
>> no mention the setting is working for IMAP only, not for POP3. What I
>> need for my use case is to get consistent logging for both protocols.
>> More precisely, considering:
>> - REALSERVER.IP as the real server IP address
>> - CLIENT.IP as the client IP address
>> - "login_trusted_networks = PROXY.IP" set in the real server config
>> I'm expecting to see the following information in the mail log of real
>> server for both IMAP and POP3 login processes:
>> user=<USER>, method=<METHOD>, rip=CLIENT.IP, lip=PROXY.IP, mpid=MPID,
>> TLS, session=<SESSION>
>> What I'm seeing instead is:
>> 1. imap-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP,
>> lip=PROXY.IP, mpid=MPID, TLS, session=<SESSION>
>> 2. pop3-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP,
>> lip=REALSERVER.IP, mpid=MPID, TLS, session=<SESSION>
>> If I didn't have "login_trusted_networks = PROXY.IP" I'd get
>> "rip=PROXY.IP" instead of "rip=CLIENT.IP" and this isn't what I want.
>> login_trusted_networks does its job just fine for the purpose, but I
>> was expecting it to effect "lip=%l" field for both IMAP and POP3
>> services in the same way.
>
> Making some inroads here. Following
> https://doc.dovecot.org/settings/core/#setting-login-log-format-elements
> I'm now using:
> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e
> %c session=<%{session}> real_rip=%{real_rip} real_lip=%{real_lip}
>
> And these are the results in the real server mail log:
>
> I. For connections coming via the proxy:
> 1. imap-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP,
> lip=PROXY.IP, mpid=MPID, TLS, session=<SESSION>, real_rip=PROXY.IP,
> real_lip=REALSERVER.IP
> 2. pop3-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP,
> lip=REALSERVER.IP, mpid=MPID, TLS, session=<SESSION>,
> real_rip=PROXY.IP, real_lip=REALSERVER.IP
>
> II. For connections coming into the real server directly:
> 1. imap-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP,
> lip=REALSERVER.IP, mpid=MPID, TLS, session=<SESSION>,
> real_rip=CLIENT.IP, real_lip=REALSERVER.IP
> 2. pop3-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP,
> lip=REALSERVER.IP, mpid=MPID, TLS, session=<SESSION>,
> real_rip=CLIENT.IP, real_lip=REALSERVER.IP
>
> Looking at II. (no proxy) the logged values are identical for both
> IMAP and POP3 login processes. Compare it to I. where the "lip" value
> is different between the 2, while all other fields have the same value.
> Is it supposed to work like this?
>
Ok, I see what you're getting at. Ignore my previous comments which were
totally off track. The following documentation seems to throw some light
on it

https://doc.dovecot.org/configuration_manual/forwarding_parameters/#forwarding-parameters

With Imap proxying among the parameters sent in the ID command are:

x-connected-ip - Server IP

x-connected-port - Server port

which get copied into the local ip and port as you are seeing.

For POP3 parameters forwarded by XCLIENT there seems to be no equivalent.

Only thing I can suggest is a work around along these lines: setup a
different login_log_format_elements to use only with the pop3
configuration section which has the lip element defined as:

lip=%{if;%{real_rip};eq;%r;%l;%{real_rip}}

Hopefully it works, though it's somewhat ugly.

John