Question about login_log_format_elements in a proxy environment

[Adi Pircalabu]

On 08-12-2020 10:33, Adi Pircalabu wrote:
> On 08-12-2020 9:41, John Fawcett wrote:
>> On 07/12/2020 23:22, John Fawcett wrote:
>>> On 07/12/2020 23:09, Adi Pircalabu wrote:
>>>> On 08-12-2020 3:13, John Fawcett wrote:
>>>>> On 07/12/2020 06:02, Adi Pircalabu wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> I have a Dovecot proxy setup with several proxy machines 
>>>>>> (currently
>>>>>> running 2.3.11.3) in front of the real Dovecot servers (2.3.10.1)
>>>>>> storing the mailboxes. "doveconf -a | egrep lip" returns:
>>>>>> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l 
>>>>>> mpid=%e
>>>>>> %c session=<%{session}>
>>>>>> 
>>>>>> In the real server maillog I'm expecting to have "lip" replaced 
>>>>>> with
>>>>>> the IP address of the proxy. It works as expected for imap-login
>>>>>> processes, however for pop3-login processes I still see the real
>>>>>> server IP instead of the proxy IP. Ideas?
>>>>>> 
>>>>>> Regards,
>>>>>> 
>>>>> Hi Adi
>>>>> 
>>>>> in general people want to get the original ip not the proxied ip. 
>>>>> The
>>>>> proxying of the original ip is done by a different method for imap
>>>>> and pop3
>>>>> 
>>>>> https://wiki.dovecot.org/Design/ParameterForwarding
>>>>> 
>>>>> However, unless I'm reading this wrongly, both methods are affected 
>>>>> by
>>>>> trusted_networks settings. I guess for people to help further, 
>>>>> you'd
>>>>> need to give more info your configuration settings.
>>>> Thanks John. login_trusted_networks, if this is the setting you're
>>>> referring to, lists the proxy IPs. I'd have thought, by having this
>>>> setting on the real servers, the proxy IP will be logged by both 
>>>> IMAP
>>>> and POP3 login processes, but it appears it isn't the case. It works
>>>> for IMAP, not for POP3.
>>>> The reason I need the proxy IP in the "lip" instead of the local IP 
>>>> in
>>>> the real server mail log is that I need to filter certain 
>>>> connections,
>>>> both IMAP and POP3, that are coming directly into the real server 
>>>> IP.
>>>> By capturing the IMAP & POP3 traffic on the real servers and 
>>>> matching
>>>> the results to the mail log entries I *should* be able to tell what
>>>> mail accounts from which remote IP addresses are coming in via the
>>>> proxies and which ones are coming into the real servers directly. 
>>>> Hope
>>>> that makes sense.
>>>> Cheers,
>>>> 
>>> The way I read it is that by specifing login_trusted_networks the 
>>> proxy
>>> ip can be overwritten by the real ip. I think that's the opposite of
>>> what you need.
>>> 
>>> I can't throw any light on why that is not working for imap but is
>>> working for pop3. But as you don't want the overwriting, maybe you
>>> should try without login_trusted_networks.
>>> 
>>> John
>>> 
>> You're probably not getting the real ip logged for imap despite having
>> login_trusted_networks due to the default for imap_id_retain on the 
>> proxies.
>> 
>> John
> 
> (Aki cc-ed)
> Thanks. I actually need login_trusted_networks on the real servers so
> that the real server has access to the client IP address, aka "rip" in
> the log entry. What I need is consistent values for "lip" field for
> both IMAP and POP3 login processes. Looking at
> https://doc.dovecot.org/configuration_manual/proxy_settings/ there's
> no mention the setting is working for IMAP only, not for POP3. What I
> need for my use case is to get consistent logging for both protocols.
> More precisely, considering:
> - REALSERVER.IP as the real server IP address
> - CLIENT.IP as the client IP address
> - "login_trusted_networks = PROXY.IP" set in the real server config
> I'm expecting to see the following information in the mail log of real
> server for both IMAP and POP3 login processes:
> user=<USER>, method=<METHOD>, rip=CLIENT.IP, lip=PROXY.IP, mpid=MPID,
> TLS, session=<SESSION>
> What I'm seeing instead is:
> 1. imap-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP,
> lip=PROXY.IP, mpid=MPID, TLS, session=<SESSION>
> 2. pop3-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP,
> lip=REALSERVER.IP, mpid=MPID, TLS, session=<SESSION>
> If I didn't have "login_trusted_networks = PROXY.IP" I'd get
> "rip=PROXY.IP" instead of "rip=CLIENT.IP" and this isn't what I want.
> login_trusted_networks does its job just fine for the purpose, but I
> was expecting it to effect "lip=%l" field for both IMAP and POP3
> services in the same way.

Making some inroads here. Following 
https://doc.dovecot.org/settings/core/#setting-login-log-format-elements 
I'm now using:
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c 
session=<%{session}> real_rip=%{real_rip} real_lip=%{real_lip}

And these are the results in the real server mail log:

I. For connections coming via the proxy:
1. imap-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP, 
lip=PROXY.IP, mpid=MPID, TLS, session=<SESSION>, real_rip=PROXY.IP, 
real_lip=REALSERVER.IP
2. pop3-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP, 
lip=REALSERVER.IP, mpid=MPID, TLS, session=<SESSION>, real_rip=PROXY.IP, 
real_lip=REALSERVER.IP

II. For connections coming into the real server directly:
1. imap-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP, 
lip=REALSERVER.IP, mpid=MPID, TLS, session=<SESSION>, 
real_rip=CLIENT.IP, real_lip=REALSERVER.IP
2. pop3-login: user=<USER>, method=<METHOD>, rip=CLIENT.IP, 
lip=REALSERVER.IP, mpid=MPID, TLS, session=<SESSION>, 
real_rip=CLIENT.IP, real_lip=REALSERVER.IP

Looking at II. (no proxy) the logged values are identical for both IMAP 
and POP3 login processes. Compare it to I. where the "lip" value is 
different between the 2, while all other fields have the same value.
Is it supposed to work like this?

-- 
Adi Pircalabu