Proxy: Selenium + Python, Firefox

Question

How can I redirect the traffic of Firefox launched by Selenium in Python to a proxy? I have used the solutions suggested on the web but they don't work!

I have tried:

profile = webdriver.FirefoxProfile() 
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http", "54.213.66.208")
profile.set_preference("network.proxy.http_port", 80)
profile.update_preferences() 
driver = webdriver.Firefox(profile)

Stephainestephan

There might be more than one solution on the web. What did you try? (In particular, did you try this?) – Kimberlykimberlyn

I have tried with this one: profile = webdriver.FirefoxProfile() profile.set_preference("network.proxy.type", 1) profile.set_preference("network.proxy.http", "54.213.66.208") profile.set_preference("network.proxy.http_port", 80) profile.update_preferences() driver = webdriver.Firefox(profile) – Stephainestephan

Is your URL using http: or https:? – Kimberlykimberlyn

http. I lunch this command driver.get("whatismyipaddress.com") and the ip is not the ip of the proxy but by ip... – Stephainestephan

did you find out how to make this work? I have the same issue and I am using Firefox46 – Prank

Answer 1

You need to import the following:

from selenium.webdriver.common.proxy import Proxy, ProxyType

Then setup the proxies:

myProxy = "xx.xx.xx.xx:xxxx"

proxy = Proxy({
    'proxyType': ProxyType.MANUAL,
    'httpProxy': myProxy,
    'ftpProxy': myProxy,
    'sslProxy': myProxy,
    'noProxy': '' # set this value as desired
    })

Then call the webdriver.Firefox() function as follows:

driver = webdriver.Firefox(proxy=proxy)
driver.get("http://www.google.com")

Don't remember where exactly I found this solution, but its out there somewhere. Will surely provide a link if I find it again. Just took this part out of my code.

Abingdon

I think you get that code from the JAVA standalone-server, wich uses this way to found desired browser capabilities. Is really this working? – Geneticist

this is the solution for python, not java. This definitely works for me. – Abingdon

My mistake. The way you wrote that function is very similar to the Java example guides, when the python guides do not offer all the possible ways to code something, while supporting. Maybe you are a expert coder, o maybe you simply learned this way, but is not present on the noob guides. I have look a way to do that using profiles with name, but didn't was aware about howto edit the current one, the random created profile, the current instance. I will test that for sure. Thank you. – Geneticist

Important to note: myProxy has to be IPAddress:port or hostname:port where hostname doesn't contain the "http://" prefix – Prank

How about changing the proxy later? – Monthly

What does noProxy mean? What is an example of a value for it? – Hypogene

This is deprecated! – Faroff

Answer 2

Try this for firefox/geckodriver:

proxy = "212.66.117.168:41258"

firefox_capabilities = webdriver.DesiredCapabilities.FIREFOX
firefox_capabilities['marionette'] = True

firefox_capabilities['proxy'] = {
    "proxyType": "MANUAL",
    "httpProxy": proxy,
    "ftpProxy": proxy,
    "sslProxy": proxy
}

driver = webdriver.Firefox(capabilities=firefox_capabilities)

And for Chrome you can use:

proxy = "212.66.117.168:41258"
prox = Proxy()
prox.proxy_type = ProxyType.MANUAL
prox.http_proxy = proxy
prox.socks_proxy = proxy
prox.ssl_proxy = proxy

capabilities = webdriver.DesiredCapabilities.CHROME
prox.add_to_capabilities(capabilities)
driver = webdriver.Chrome(desired_capabilities=capabilities)

Impress

Thanks. If you want you can also use my python library to get free and fresh proxy github.com/OceanFireSoftware/python-simple-proxy – Impress

This worked for me!!!! So many of the solutions for this are missing the firefox_capabilities['marionette'] = True. Thank you for sharing! – Undeceive

This is useful. After so many trial and errors this is the only solution worked in Latest Firefox/Geckodriver setup. As after 46 version of firefox you need to use geckodriver and for that you can set up proxy this way. – Frontier

thx. The only solution that actually works. Should be marked as "Accepted answer" – Elk

worked without ['marionette'] = True for me.. I guess since it is True by default =) – Elk

Answer 3

Your problem is on the driver init. Try webdriver = webdriver.Firefox(firefox_profile=profile) All the other code is OK and you can also remove profile.update_preferences() line.

I found YOUR solution with a 2 minute google search. How much time did you spend waiting for? :D

Your problem is that you maybe read code from other langs but Python. Replace this webdriver.Firefox(profile) with webdriver.Firefox(firefox_profile=profile).

Your code should be:

profile = webdriver.FirefoxProfile() 
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http", "54.213.66.208")
profile.set_preference("network.proxy.http_port", 80)
profile.update_preferences() 
driver = webdriver.Firefox(firefox_profile=profile)

Geneticist

No. __init__ method of WebDriver (imported as Firefox) accepts firefox_profile as first argument. – Olympian

Have to admit that if you use the named variable is not same as first "positional", but we are talking different things at all. This will be similar to run Firefox("firefox", profile) without named arguments. – Geneticist

I'm not sure what are you talking about. Check the source. Firefox(firefox_profile=profile) is equal to Firefox(profile). – Olympian

@Olympian Then explain the OP why didn't work for him, but using named variable worked. Maybe the source changed between versions? I don't know. – Geneticist

I never reply without test the code, so I said. Thank you for being unnecesarily rude. Stay away to make this unuseful comments in stackoverflow. – Geneticist

Answer 4

Here is the code for Selenium 4.x, to enable the proxy, you need to set browser options, these parameters from the bottom (ssl is https), and the rest is clear from the names, in order to find out these parameters, I went into the browser config and manually checked what to pick up.

I also know how to remove the bot check in the driver, for this you need to compile your driver on rast

def main():
    url = "https://2ip.ru/"

    options = Options()
    options.set_preference("network.proxy.type", 1)
    options.binary_location = r"C:\Program Files\Mozilla Firefox\firefox.exe"
    options.set_preference("network.proxy.http", "xx.xxx.xxx.xxxx")
    options.set_preference("network.proxy.http_port", 8000)
    options.set_preference("network.proxy.ssl", "xx.xxx.xxx.xxx")
    options.set_preference("network.proxy.ssl_port", 8000)

    serviceDriver = Service(
    executable_path=r"C:\Users\User\PycharmProjects\driver\geckodriver.exe")

    driver = webdriver.Firefox(options=options, service=serviceDriver)
    driver.get("https://2ip.ru")

Peristyle

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. – Blistery

That didn't work for me – Kemppe