How to set headers in python selenium chrome webdriver

Question

I cant find how to set request headers within selenium chrome webdriver (python). I need to set "host" header. I tryed that:

from selenium import webdriver
from pyvirtualdisplay import Display
from selenium.webdriver import DesiredCapabilities


url = 'http://localhost:888/test'

display = Display(visible=0, size=(800, 800))
display.start()

desired_capabilities = DesiredCapabilities.CHROME.copy()
desired_capabilities['chrome.page.customHeaders.host'] = 'test.local'

driver = webdriver.Chrome('/tmp/chromedriver',
                          desired_capabilities=desired_capabilities)

driver.get(url)

And result:

GET /test HTTP/1.1
Host: localhost:888
Connection: keep-alive
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/53.0.2785.143 Chrome/53.0.2785.143 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8

So, I expected header host:test.local but found that Host: localhost:888

What I am doing wrong? I cant find how to set headers with Chrome webdriver :(

Fichte

Answer 1

Host in the request header is the name of the server (destination), and that's set by your driver.get(url).

Host request header is constructed from the serverUrl using nodejs url.parse(serverUrl). So you cannot send the request to server "a", with the Host header set to "b".

This means if your server is being hosted on your local computer, you can (likely) access it as http://localhost:888/test, in which case your header will include:

GET /test HTTP/1.1
Host: localhost:888

or access it as http://127.0.0.1:888/ resulting in

GET /test HTTP/1.1
Host: 127.0.0.1:888

or access it as http://Johns-iMac.local:888/test... assuming you're using an iMac whose hostname is "Johns-iMac.local":

GET /test HTTP/1.1
Host: Johns-iMac.local:888

or access it using your network interface IP address, perhaps http://192.168.1.5:888/test:

GET /test HTTP/1.1
Host: 192.168.1.5:888

And this is the key to your solution: You need to access it using the URL http://test.local:888/test.

So what you want to do is modify DNS so that your computer thinks host test.local has an IP address of 127.0.0.1 (or whatever you prefer).

Doing this depends on your OS, but commonly, it's simply adding a line to the end of your /etc/hosts file:

test.local    127.0.0.1

(you may need to reboot).

From now on, all requests to test.local will be sent to the server at 127.0.0.1 and http://test.local:888/test will give you the request header:

GET /test HTTP/1.1
Host: test.local:888

• NOTE: I say your server will "likely" respond to any/all of these different requests. That's true, assuming it's configured to respond to a default host, or configured with a VirtualHost (Apache) or Server Block (nginx) which matches whatever host you're attempting to access.

Diomedes

I need to send request to localhost with different host headers. How can I do that? – Prerecord

So the server thinks your browser is somewhere else? You can't, not really. You don't send YOUR host name to the server, rather it's a TCP connection, which means the server will get your IP address. The server may do a reverse lookup to attempt to determine hostname from IP, but not automatically. Proxies allow you to "change" you IP by using intermediate routing, but that doesn't sound like what you're looking for either. – Diomedes

Currently researching the same issue. Looking to discover how you would provide a referrer to your opened webpage in selenium tests, the app being opened uses the referrer and that is undefined as there is no referrer to the app from the test suite. – Australopithecus