Feature request: Add capability for IIS log parsing to recognize custom header X-Forwarded-For

[ku4eto]

As the tile says.
From IIS 8.5 onwards, its possible to add a custom Logging option, to include X-Forwarded-For at the end of the line.

Field selection looks like this:
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken X-Forwarded-For

Microsoft documentation: https://learn.microsoft.com/en-us/iis/get-started/whats-new-in-iis-85/enhanced-logging-for-iis85

Actual log with the above mentioned field seleciton:
2024-10-01 00:00:06 172.18.21.9 GET /api/somee-end-point 443 - 172.18.23.68 Mozilla/5.0+(Macintosh;+Intel+Mac+OS+X+10_15_7)+AppleWebKit/605.1.15+(KHTML,+like+Gecko)+Version/16.6.1+Safari/605.1.15 https://somesite.com/some-url/ 200 0 0 591 3336 85 123.123.123:33453,+172.16.0.1:49770

Highlighting the important part: 123.123.123:33453,+172.16.0.1:49770 This is from an IIS instance behind an Azure application gateway. The Azure application gateway uses X-Forwarded-For header and passes to the IIS the real IP of the user making the request. Then there is a , (comma), + (plus) and the internal IP address of the Application gateway.

We will need to have ONLY the real user IP and not the ports used or the internal IP.

[0bi-w6n-K3nobi]

Hi, @ku4eto
It's good talk with You here.

May to be try this:
goaccess --date-format='%Y-%m-%d' --time-format='%H:%M:%S' --log-format='%d %t %v %m %U %q %^ %^ %^ %u %R %s %^ %^ %^ %b %L %h:^%' --http-protocol=no ...

Or at goaccess.conf without quotes:

date-format %Y-%m-%d
time-format %H:%M:%S
log-format %d %t %v %m %U %q %^ %^ %^ %u %R %s %^ %^ %^ %b %L %h:^%
http-protoco no

But, I need talk about some notes:

• QUERY field is missing at your LOG above; so I did just try ... /api/somee-end-pointQUERY443 ... ;
• Client IP is incorrect; may You means 123.123.123.123 ?
• Did not have Protocol field and I disable it at log-format;
• Did not have Domain or VHost, so I use Server IP for that;
• I do not know time-taken unit, so I use milliseconds, is correct ?
• I use cs-bytes as bytes response, is correct ?

I hope that helped you. Feel free for more asks.