Headless Chrome (with selenium) CANNOT request with PROXY server, but requests can?

I am trying to use Chrome along with python webdriver + selenium, but it seems not working when I set the proxy settings? Here is my code:

from selenium import webdriver

PROXY = 'http://42.115.88.220:53281'
chromeOptions = webdriver.ChromeOptions()
chromeOptions.add_argument('--proxy-server=%s' % PROXY)
chromeOptions.add_argument("ignore-certificate-errors")

wbe = webdriver.Chrome(options=chromeOptions)
wbe.get("http://icanhazip.com")

When I run the above codes, the browser gives me: "This site can’t be reached" error:

This site can’t be reached
The connection was reset.
Try:

Checking the connection

Checking the proxy and the firewall

Running Windows Network Diagnostics

ERR_CONNECTION_RESET

Some Efforts: I tried requests with my proxy server, and it works. So it shouldn't be the problem of my proxy server.

import requests

proxies = {"http": "http://42.115.88.220:53281"}
r = requests.get("http://icanhazip.com", proxies = proxies)
print (r.status_code)

This gives me a response code of 200 and good response.

Goal: My final goal is to build a web-crawler with headless chrome with PROXY, so now I am testing a non-headless one first. But it seems there have been something wrong with this PROXY issue.

I would be really appreciated if anyone could help me out with this problem!!!

from selenium import webdriver PROXY = 'http://ip:port' chromeOptions = webdriver.ChromeOptions() chromeOptions.add_argument('--proxy-server=%s' % PROXY) chromeOptions.add_argument("ignore-certificate-errors") # Headless mode for chrome browser chromeOptions.add_argument('--headless=chrome') wbe = webdriver.Chrome('your_driver_path_or_service', options=chromeOptions) wbe.get("http://icanhazip.com") print(wbe.title) print(wbe.current_url) print(wbe.page_source) # Output: # http://icanhazip.com/ # <html><head><meta name="color-scheme" content="light dark"></head><body><pre # style="word-wrap: break-word; white-space: pre-wrap;">your ip # </pre></body></html>

Recommended topics

Hot tags