Duplicate Referrer Header - HTTP.sys barfs

Question

I have been testing the PS3 browser (NetFront) with embedded Flash components on a web page and there seems to be a rather serious bug with the way that any requests from Flash are issued. The browser is adding a referrer (referer) and so is the Flash player resulting in duplicate headers. This is actually OK if the referrer headers are the same, however they are not, and as a result the request is stopped by http.sys on W2k8. This is a major issue for me as I have a Flash component that needs to call web services in order to function.

Is there some way that we can filter or allow these requests through, possibly stripping the duplicate headers along the way.

Some solutions that we have thought of but are either suboptimal or not possible with our development and live environments are:

• Adding a proxy in front of the servers to clean the request.
• Fixing the request at our load balancers. This may work on our live environment but not in dev. Also using a load balancer to perform this operation is bad practise, our operations team would not be pleased :)
• Send all data to Flash during initialisation via flashvars. Possibly doing any updates/callbacks via Javascript. This feels messy and is would require significant custom code for one platform.
• Prevent the browser or Flash from sending referrer headers. This would be acceptable, however there doesn't seem to be a way of enabling this in Flash or NetFront.

For reference the following request will reproduce the problem (copy and paste into Fiddler or other tool, I'm assuming you are running IIS7, W2K8 as we are). You should get a 400 Bad Request response.

GET / HTTP/1.1
Host: localhost
Referer: http://localhost/NetFrontBrowser/
Referer: http://localhost/Flash/

Many thanks

Outpoint

Answer 1

stmedit from the Windows DDK "demonstrates replacing a string pattern for a Transmission Control Protocol (TCP) connection using the Windows Filtering Platform (WFP)".

A little bit of additional hackery turns it into an FSM which can dump all but the first referrer string.

Pyrazole

Answer 2

If I number these 1-4, I would say that 2 makes the best sense for your live environment, while 3 might be something you'd need to do if your development environment cannot similarly match your live environment. Number 1 and 2 are quite similar since a load balancer is basically a reverse proxy. Number 4, as you pointed out, seems impossible without waiting for Sony to update the browser and plugin they're using, which seems unlikely since they probably farmed it out and have poor control over it in the first place (kind of like the firmware on my korean made TV is made in Palo Alto, and it took them 10 months to comply with the GPL due to not being able to communicate well enough).

Cressi

Answer 3

Apache has the module mod_headers. This would solve your problem, but if you have to use http.sys, then I would try to find something equivalent to mod_headers. Perhaps someone has ported mod_headers to http.sys?

Also I would evaluate URL rewriting. I know this is about headers and not URLs, but perhaps the URL rewriting module of http.sys (if it exists) knows about headers.

I assume that the mod_headers or mod_rewrite equivalent for http.sys can handle the request before http.sys barfs.

Good luck!

Satiable

Answer 4

Adding this to the HTML will stop the referrer from being used if its not needed.

<meta name="referrer" content="never">

Entomostracan

Thanks for the answer, unfortunately I can't test it, and the issue has long since past, but I like the thinking! Unfortunately that tag looks like it was developed after the NetFront browser that we would have been working with, but this solution could work for anyone else with a more current problem. – Outpoint