Preventing cURL Referrer spoofing

Question

We received PHP code from a developer with a web-stats script that relies solely on $_SERVER['HTTP_REFERER']. With cURL, you can easily fake it as follows:

curl_setopt($curl, CURLOPT_REFERER, "client website");

and I'm looking for a way to prevent it. This can even be done by the client website as well, to have higher stats. I'm looking for a way to prevent this spoofing. Is this possible at all? If so, how can this be achieved?

Scribbler

You can't. The best you can do is filter obviously invalid referrers and accept the remaining indistinguishable spoofs as margin of error. – Doersten

Is the final objective of preventing the spoofing identifying the client who made the request? Because if that is the idea, you can use browser fingerprint. It's not 100% accurate, but it's as good as it gets and in most cases, it will be very very close to uniquely identify the client – Adduction

This question is similar to https://mcmap.net/q/203853/-determining-referer-in-php The solution would be a custom referrer token, that is cryptographically signed and can be verified server-side. But this would require you to work closely together with the page sending the traffic - which you don't want to trust either, as I understand it. – Baerl

@Adduction The objective is to gather actual stats of client websites and not let anyone artificially increase it. Not sure how browser fingerprinting will help us though. – Scribbler

@Baerl We don't gather stats of non-client websites anyway. Our purpose is to identify fake stats and reject them. – Scribbler

@demechanico: When I read your question, I thought for a moment, that you wanted to identify the client (web browser), not the page. Fingerprint of the browser would work to detect the client, but not the page. So it's my mistake for not understanding your question properly. – Adduction

I added this code to my 404 page if (($website.$_SERVER['REQUEST_URI']) == $_SERVER['HTTP_REFERER']) { #is someone probing for vulnerabiliites - by spoofing sleep(600); #make them wait a long time die(); #and die } – Intromission

Answer 1

No, there's no definitive way of determing the URL Referrer.

As per the HTTP spec, HTTP_REFERER is optional. Some firewall packages strip these out by default, some clients don't send the referer value, and and there are numerous ways (like the one you showed in the question) to modify this value.

In short, the HTTP_REFERER value cannot be trusted. There will always be some way to modify these values. This is mentioned in the PHP manual documentation for $_SERVER (emphasis mine):

The address of the page (if any) which referred the user agent to the current page. This is set by the user agent. Not all user agents will set this, and some provide the ability to modify HTTP_REFERER as a feature. In short, it cannot really be trusted.

To answer your question: no, there is no way to prevent HTTP_REFERER value being altered. I'd suggest you double-check the value before using it (optionally, apply htmlspecialchars() on it to prevent injection) or don't use it at all. Unfortunately, it is a "take it or go home" deal.

Are

OP already knows that it can be faked. But he is looking for a way to prevent the issue by any other means... – Pectoralis

@ShankarDamodaran: But there's no way. I thought that answered the question, but I've updated the answer to reflect this fact. If you know a way to prevent the issue, please feel free to post it as an answer :) – Are

Thanks a lot, but I indeed hope to get some (at least partly working) solution. If I can't find a way to deal with it, I will implement some audit code, that will clean-up seemingly-fake stats. Note: I can easily ignore requests without HTTP_REFERRER and it will result in only small decrease for clients (as most browsers provide it anyway?!). – Scribbler

@demechanico: The value isn't sent if the user types out the URL manually. It's not sent when doing a <meta> refresh either. See here for information on browsers that do not pass HTTP_REFERRER value. – Are

@AmalMurali: Thanks for the resource, it's enlightening. A dead end?! – Scribbler

@demechanico: I think so. Now you can wait for more knowledgeable answers (most people are not active today as it's a Sunday). Alternatively, you can try asking your question on Information Security. – Are

Answer 2

There is nothing you can do about this referrer tempering. All of the web-stats scripts are depending on this referrer. Even the giant web-stats sites including google-analytics gets fooled by this fake referrer.

It could be a nice solution to check back the referrer url. I mean visit the referrer and check whether your url exists there or not. But of course its time consuming, slow, and also requires a huge bandwidth as well. However it is not enough to overcome this issue.

Here are few problems at where you'll not find your link when you are tracking back the referrer url:

• What if the referred url is behind the session? For example a link came from email like yahoo, google, or from a private forum.

• What if the url came from a javascript link/click?

• Link from an iframe is nonetheless of javascript link as well.

Mandelbaum

Thanks for your input. Well, the code is supposed to be originating from client websites, in a way, it's supposed to keep view-stats of the website. I am mostly concerned about websites trying to increase their stats by fake requests. So, iframe is the only issue we may have from what you listed above. – Scribbler