Reason for browser not showing X-Forwarded-For headers

Question

Note: Please do read the full question

I'm trying to understand as to why the browsers doesn't show me any X-Forwarded-For header every time a request a page

BTW here are my Request Headers look like

Request URL:http://localhost:3000/users/sign_in
Request Method:GET
Status Code:304 Not Modified

Request Headers:

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
Cache-Control:max-age=0
Connection:keep-alive
Cookie:undefined=0; poasterapp=s%3A4faaa6b1723e7c6fbd949083532c52598652547b.sNX%2BKOEed2TEQkQN7I7K5lgpoHMRpwerKFvUegMnTVI; _minerva_session=BAh7CUkiD3Nlc3Npb25faWQGOgZFRkkiJWEyM2Q0ZTViMWEyODBiYmFmODEwZTJhZmUwNWU5ODk5BjsAVEkiE3VzZXJfcmV0dXJuX3RvBjsARiIGL0kiCmZsYXNoBjsARm86JUFjdGlvbkRpc3BhdGNoOjpGbGFzaDo6Rmxhc2hIYXNoCToKQHVzZWRvOghTZXQGOgpAaGFzaHsGOgphbGVydFQ6DEBjbG9zZWRGOg1AZmxhc2hlc3sGOwpJIgAGOwBUOglAbm93MEkiEF9jc3JmX3Rva2VuBjsARkkiMUN0Uk56SXU0dUdIdzgwcFZJM3R0L2N4dlovRllTSGRrQ2o1R0VVanhIaVk9BjsARg%3D%3D--6bd89ce9d29e9bdcf56573f9a153dc663a8fe755
Host:localhost:3000
If-None-Match:"785d34e3998360353567fc710af123fb"
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.102 Safari/537.36

Response Headers(Not need but still )

Cache-Control:max-age=0, private, must-revalidate
Connection:close
ETag:"785d34e3998360353567fc710af123fb"
Server:thin 1.5.0 codename Knife
Set-Cookie:_minerva_session=BAh7CEkiD3Nlc3Npb25faWQGOgZFRkkiJWEyM2Q0ZTViMWEyODBiYmFmODEwZTJhZmUwNWU5ODk5BjsAVEkiE3VzZXJfcmV0dXJuX3RvBjsARiIGL0kiEF9jc3JmX3Rva2VuBjsARkkiMUN0Uk56SXU0dUdIdzgwcFZJM3R0L2N4dlovRllTSGRrQ2o1R0VVanhIaVk9BjsARg%3D%3D--dfb3ce9f5c97463cfcd0229a133654e6cc606d98; path=/; HttpOnly
X-Request-Id:41a6f3062dc8bc36b7b3eae71dc5075d
X-Runtime:89.238257
X-UA-Compatible:IE=Edge

Now as said, I dont see any X-Forwarded-For in request headers

Reading through the wiki pages of X-Forwarded-For make me feel that ,it is something done by caching server(which in my case I believe is my ISP provider) so am I safe to believe that the **X-Forwarded-For** headers is something that is added at the caching server side (ISP provider)

If yes their is this one bugging me about it then that is

why? is the same true (i.e X-Forwarded-For not appearing in the request-headers ) for my server running locally on my machine and I accessing them via browser like http://localhost:3000

Sheridansherie

Answer 1

X-Forwarded-For is not a standard request header as specified in RFC 2616 Section 5.3 that addresses the protocol standard request headers, which are (as specified in the RFC)

• Accept
• Accept-Charset
• Accept-Encoding
• Accept-Language
• Authorization
• Expect
• From
• Host
• If-Match
• If-Modified-Since
• If-None-Match
• If-Range
• If-Unmodified-Since
• Max-Forwards
• Proxy-Authorization
• Range
• Referer
• TE
• User-Agent

In order for your incoming request to have the custom [X-Forwarded-For] header, it must be explicitly added to that request by the calling client. The easiest explanation of why you are not seeing that header is that the client sending the request did not manually add it.

The tricky thing is that the header you are expecting to see is not a header that you should necessarily be expecting to receive unless there is a contract in place between your service and the caller that is apart from the HTTProtocol indicating that you should expect an X-Forwarded-For value to be specified in your request header. As others have already stated, the XFF header is typically set by a proxy server or a load balancer to indicate who the real requester is that is acting through their proxy.

As a service provider, if you demand that an [X-Forwarded-For] header be set in all requests, you must enforce it at a service policy level. If you do not want to service proxy accounts that do not identify who they are shielding with their proxy IP, bounce their request with a 403 Forbidden. If you are in a situation where you must service these requests but depend on this header being set, then you're going to have to come up with a custom process where you could communicate their error back.

Here is what the HTTProtocol has to say about anonymity:

Because the source of a link might be private information or might reveal an otherwise private information source, it is strongly recommended that the user be able to select whether or not the Referer field is sent. For example, a browser client could have a toggle switch for browsing openly/anonymously, which would respectively enable/disable the sending of Referer and From information.

Clients SHOULD NOT include a Referer header field in a (non-secure)
HTTP request if the referring page was transferred with a secure
protocol.

Authors of services which use the HTTP protocol SHOULD NOT use GET
based forms for the submission of sensitive data, because this will
cause this data to be encoded in the Request-URI. Many existing
servers, proxies, and user agents will log the request URI in some
place where it might be visible to third parties. Servers can use
POST-based form submission instead ...

Elaborate user-customized accept header fields sent in every request, in particular if these include quality values, can be used by servers as relatively reliable and long-lived user identifiers. Such user identifiers would allow content providers to do click-trail tracking, and would allow collaborating content providers to match cross-server click-trails or form submissions of individual users. Note that for many users not behind a proxy, the network address of the host running the user agent will also serve as a long-lived user identifier. In environments where proxies are used to enhance privacy, user agents ought to be conservative in offering accept header configuration options to end users. As an extreme privacy measure, proxies could filter the accept headers in relayed requests. General purpose user agents which provide a high degree of header configurability SHOULD warn users about the loss of privacy which can be involved.

Personally, I would bounce the request with a 401.2 and route the requester to a challenge screen via the WWW-Authenticate response header that presents them with notification that they will not be allowed anonymous access to your site. It's kind of a bastardized way of using the WWW-Authenticate header, but it seems like you're expecting the X-Forwarded-For header to acknowledge and identify the real requester and allowing public non-anonymous access to your service. To me, that's an authentication concern.

Discriminant

Answer 2

Why are you expecting X-Forwarded-For to appear in the first place? You are connecting to a web server running on localhost, so there is no ISP provider involved at all. Even if you were connecting to your web server via an ISP, it is still not likely to add X-Forwarded-For to the requests. X-Forwarded-For is typically added by an HTTP proxy server or load balancer, neither of which you are going through. X-Forwarded-For is never included by a web browser.

Pediatrics

Yes I know that running in localhost does not involve ISP and that what I want to know for sake of it why ? the X-Forwarded-For getting listed in my request headers in my local application server when there is no ISP involve as definitely no load balancer involve so the only assumption I have for that appear in application resquest headers is browser adding that is why I mention in question I cant find it in the web inspector tool – Sheridansherie

Why should a local browser connecting to a local web server involve the ISP at all? Everything is on the same PC. It does not even generate network traffic, as the OS is smart enough to know both endpoints are local and will route the packets accordingly. As for the header, a browser does not create it, as proved by your inspector tool. So it has to be coming from somewhere else. Either your web server is creating it internally, or an app on your PC is intercepting the connection and injecting it, or the like. – Pediatrics

Answer 3

• ISP providers does not adds X-Forwarded-For.
• X-Forwarded-For is not for end user to identify application behind proxy/balancer.
• X-Forwarded-For is for application behind proxy/balancer to identify user.

For example: You got web application (php, java, etc.) also you have http server (Apache, nginx, etc.) then:

1. User do request to http server.
2. Http server redirect request to web application with X-Forwarded-For as user ip.
3. Yours web application know that it is behind http server so it does read X-Forwarded-For as user ip.

Hudnall

Your second and third bullet points are slightly presumptuous. The X-Forwarded-For header is merely intended to identify the real requester who is acting through a proxy account. The intent behind setting the XFF header has multiple purposes, one of which would be to internally identify an account for internal purposes that will be anonymized through the proxy for external calls. Another would be to prevent their own proxy service from being an HTTP anonymizer for abusers by identifying the real requester in a request header value. – Discriminant