Running Selenium Webdriver with a proxy in Python
Asked Answered
M

18

124

I am trying to run a Selenium Webdriver script in Python to do some basic tasks. I can get the robot to function perfectly when running it through the Selenium IDE inteface (ie: when simply getting the GUI to repeat my actions). However when I export the code as a Python script and try to execute it from the command line, the Firefox browser will open but cannot ever access the starting URL (an error is returned to command line and the program stops). This is happening me regardless of what website etc I am trying to access.

I have included a very basic code here for demonstration purposes. I don't think that I have included the proxy section of the code correctly as the error being returned seems to be generated by the proxy.

Any help would be hugely appreciated.

The below code is simply meant to open www.google.ie and search for the word "selenium". For me it opens a blank firefox browser and stops.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
import unittest, time, re
from selenium.webdriver.common.proxy import *

class Testrobot2(unittest.TestCase):
    def setUp(self):

        myProxy = "http://149.215.113.110:70"

        proxy = Proxy({
        'proxyType': ProxyType.MANUAL,
        'httpProxy': myProxy,
        'ftpProxy': myProxy,
        'sslProxy': myProxy,
        'noProxy':''})

        self.driver = webdriver.Firefox(proxy=proxy)
        self.driver.implicitly_wait(30)
        self.base_url = "https://www.google.ie/"
        self.verificationErrors = []
        self.accept_next_alert = True

    def test_robot2(self):
        driver = self.driver
        driver.get(self.base_url + "/#gs_rn=17&gs_ri=psy-ab&suggest=p&cp=6&gs_id=ix&xhr=t&q=selenium&es_nrs=true&pf=p&output=search&sclient=psy-ab&oq=seleni&gs_l=&pbx=1&bav=on.2,or.r_qf.&bvm=bv.47883778,d.ZGU&fp=7c0d9024de9ac6ab&biw=592&bih=665")
        driver.find_element_by_id("gbqfq").clear()
        driver.find_element_by_id("gbqfq").send_keys("selenium")

    def is_element_present(self, how, what):
        try: self.driver.find_element(by=how, value=what)
        except NoSuchElementException, e: return False
        return True

    def is_alert_present(self):
        try: self.driver.switch_to_alert()
        except NoAlertPresentException, e: return False
        return True

    def close_alert_and_get_its_text(self):
        try:
            alert = self.driver.switch_to_alert()
            alert_text = alert.text
            if self.accept_next_alert:
                alert.accept()
            else:
                alert.dismiss()
            return alert_text
        finally: self.accept_next_alert = True

    def tearDown(self):
        self.driver.quit()
        self.assertEqual([], self.verificationErrors)

if __name__ == "__main__":
    unittest.main()
Microelement answered 13/6, 2013 at 8:21 Comment(0)
S
74

Works for me this way (similar to @Amey and @user4642224 code, but shorter a bit):

from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy, ProxyType

prox = Proxy()
prox.proxy_type = ProxyType.MANUAL
prox.http_proxy = "ip_addr:port"
prox.socks_proxy = "ip_addr:port"
prox.ssl_proxy = "ip_addr:port"

capabilities = webdriver.DesiredCapabilities.CHROME
prox.add_to_capabilities(capabilities)

driver = webdriver.Chrome(desired_capabilities=capabilities)
Silviasilviculture answered 16/11, 2016 at 9:20 Comment(5)
this works, thank you. strange that the docs say that you need to use the remote driver.Koloski
driver = webdriver.Firefox(desired_capabilities=capabilities) TypeError: __init__() got an unexpected keyword argument 'desired_capabilities' why?Brigand
This answer does not work for me, I get an error "Specifying 'socksProxy' requires an integer for 'socksVersion'".Calvo
@Calvo Depending on the proxy you're using, just add prox.socks_version = 5 or prox.socks_version = 4. That should clear up the error.Propulsion
Thanks a lot. This worked absolutely fine for me. I removed prox.socks_proxy = "ip_addr:port" prox.ssl_proxy = "ip_addr:port" and added prox.https_proxy = "ip_addr:port"Hay
M
41

How about something like this

PROXY = "149.215.113.110:70"

webdriver.DesiredCapabilities.FIREFOX['proxy'] = {
    "httpProxy":PROXY,
    "ftpProxy":PROXY,
    "sslProxy":PROXY,
    "noProxy":None,
    "proxyType":"MANUAL",
    "class":"org.openqa.selenium.Proxy",
    "autodetect":False
}

# you have to use remote, otherwise you'll have to code it yourself in python to 
driver = webdriver.Remote("http://localhost:4444/wd/hub", webdriver.DesiredCapabilities.FIREFOX)

You can read more about it here.

Marsh answered 13/6, 2013 at 17:14 Comment(2)
This answer worked well for me. In case anyone else is trying to do this with Edge, webdriver.DesiredCapabilities.EDGE['proxy'] has no effect because Microsoft Edge currently doesn't have a setting to configure a proxy server (to use Edge with a proxy, you must configure the proxy under the Windows network connection settings).Taproot
For full detailed document, see: github.com/SeleniumHQ/selenium/wiki/…Surround
D
17

My solution:

def my_proxy(PROXY_HOST,PROXY_PORT):
        fp = webdriver.FirefoxProfile()
        # Direct = 0, Manual = 1, PAC = 2, AUTODETECT = 4, SYSTEM = 5
        print PROXY_PORT
        print PROXY_HOST
        fp.set_preference("network.proxy.type", 1)
        fp.set_preference("network.proxy.http",PROXY_HOST)
        fp.set_preference("network.proxy.http_port",int(PROXY_PORT))
        fp.set_preference("general.useragent.override","whater_useragent")
        fp.update_preferences()
        return webdriver.Firefox(firefox_profile=fp)

Then call in your code:

my_proxy(PROXY_HOST,PROXY_PORT)

I had issues with this code because I was passing a string as a port #:

 PROXY_PORT="31280"

This is important:

int("31280")

You must pass an integer instead of a string or your firefox profile will not be set to a properly port and connection through proxy will not work.

Darrel answered 23/7, 2014 at 20:57 Comment(3)
Port needs to be converted to int? That would make the Firefox proxy example on the official page wrong: seleniumhq.org/docs/04_webdriver_advanced.jsp In their example, PROXYHOST:PROXYPORT is passed as a string.Histogenesis
@Pyderman, you're confusing Proxy() class with FirefoxProfile() class. Using profile preferences you have to pass ip and port separately, and casting port to int(). In Proxy() class you just pass the string containig "IP:PORT", and surely it does the rest of work for you.Ieshaieso
Also firefox_profile has been deprecated, please pass in an Options object See https://mcmap.net/q/67433/-running-selenium-webdriver-with-a-proxy-in-python @Zyy AnswerDnieper
Z
14

It's quite an old post, however, for others, it might still benefit by providing the answer as of today, and yet originally author was extremely close to a working solution.

First of all, the ftpProxy setting is no longer supported at this time and will throw an error

proxy = Proxy({
        'proxyType': ProxyType.MANUAL,
        'httpProxy': myProxy,
        'ftpProxy': myProxy, # this will throw an error
        'sslProxy': myProxy,
        'noProxy':''})

Next, instead of setting the proxy property, you should be using firefox options like so

proxy = Proxy({
    'proxyType': ProxyType.MANUAL,
    'httpProxy': myProxy,
    'sslProxy': myProxy,
    'noProxy': ''})

options = Options()
options.proxy = proxy
driver = webdriver.Firefox(options=options)

Additionally, don't define the scheme when specifying the proxy, especially if you want to use the same proxy for multiple protocols

myProxy = "149.215.113.110:70"

All together it looks like this

from selenium import webdriver
from selenium.webdriver.common.proxy import *
from selenium.webdriver.firefox.options import Options

myProxy = "149.215.113.110:70"
proxy = Proxy({
    'proxyType': ProxyType.MANUAL,
    'httpProxy': myProxy,
    'sslProxy': myProxy,
    'noProxy': ''})

options = Options()
options.proxy = proxy
driver = webdriver.Firefox(options=options)
driver.get("https://www.google.ie")
Zoochemistry answered 1/2, 2022 at 8:30 Comment(1)
and in the case of authentication?Bemoan
R
9

Proxy with verification. This is a whole new python script in reference from a Mykhail Martsyniuk sample script.

# Load webdriver
from selenium import webdriver

# Load proxy option
from selenium.webdriver.common.proxy import Proxy, ProxyType

# Configure Proxy Option
prox = Proxy()
prox.proxy_type = ProxyType.MANUAL

# Proxy IP & Port
prox.http_proxy = “0.0.0.0:00000”
prox.socks_proxy = “0.0.0.0:00000”
prox.ssl_proxy = “0.0.0.0:00000”

# Configure capabilities 
capabilities = webdriver.DesiredCapabilities.CHROME
prox.add_to_capabilities(capabilities)

# Configure ChromeOptions
driver = webdriver.Chrome(executable_path='/usr/local/share chromedriver',desired_capabilities=capabilities)

# Verify proxy ip
driver.get("http://www.whatsmyip.org/")
Reader answered 8/2, 2019 at 21:28 Comment(0)
L
8

Try setting up sock5 proxy too. I was facing the same problem and it is solved by using the socks proxy

def install_proxy(PROXY_HOST,PROXY_PORT):
        fp = webdriver.FirefoxProfile()
        print PROXY_PORT
        print PROXY_HOST
        fp.set_preference("network.proxy.type", 1)
        fp.set_preference("network.proxy.http",PROXY_HOST)
        fp.set_preference("network.proxy.http_port",int(PROXY_PORT))
        fp.set_preference("network.proxy.https",PROXY_HOST)
        fp.set_preference("network.proxy.https_port",int(PROXY_PORT))
        fp.set_preference("network.proxy.ssl",PROXY_HOST)
        fp.set_preference("network.proxy.ssl_port",int(PROXY_PORT))  
        fp.set_preference("network.proxy.ftp",PROXY_HOST)
        fp.set_preference("network.proxy.ftp_port",int(PROXY_PORT))   
        fp.set_preference("network.proxy.socks",PROXY_HOST)
        fp.set_preference("network.proxy.socks_port",int(PROXY_PORT))   
        fp.set_preference("general.useragent.override","Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.75.14 (KHTML, like Gecko) Version/7.0.3 Safari/7046A194A")
        fp.update_preferences()
        return webdriver.Firefox(firefox_profile=fp)

Then call install_proxy ( ip , port ) from your program.

Leprosy answered 1/6, 2015 at 6:43 Comment(1)
how would you modify this to accept proxy credentials?Henrietta
B
8

If anyone is looking for a solution here's how :

from selenium import webdriver
PROXY = "YOUR_PROXY_ADDRESS_HERE"
webdriver.DesiredCapabilities.FIREFOX['proxy']={
    "httpProxy":PROXY,
    "ftpProxy":PROXY,
    "sslProxy":PROXY,
    "noProxy":None,
    "proxyType":"MANUAL",
    "autodetect":False
}
driver = webdriver.Firefox()
driver.get('http://www.whatsmyip.org/')
Bathesda answered 15/6, 2015 at 6:11 Comment(0)
T
6

The result stated above may be correct, but isn't working with the latest webdriver. Here is my solution for the above question. Simple and sweet


        http_proxy  = "ip_addr:port"
        https_proxy = "ip_addr:port"

        webdriver.DesiredCapabilities.FIREFOX['proxy']={
            "httpProxy":http_proxy,
            "sslProxy":https_proxy,
            "proxyType":"MANUAL"
        }

        driver = webdriver.Firefox()

OR

    http_proxy  = "http://ip:port"
    https_proxy = "https://ip:port"

    proxyDict = {
                    "http"  : http_proxy,
                    "https" : https_proxy,
                }

    driver = webdriver.Firefox(proxy=proxyDict)
Threap answered 16/1, 2019 at 17:38 Comment(0)
V
5

This helps me in September 2022 - proxy for selenium with Auth user+password

import os
import zipfile

from selenium import webdriver

PROXY_HOST = '192.168.3.2'  # rotating proxy or host
PROXY_PORT = 8080 # port
PROXY_USER = 'proxy-user' # username
PROXY_PASS = 'proxy-password' # password

manifest_json = """
{
    "version": "1.0.0",
    "manifest_version": 2,
    "name": "Chrome Proxy",
    "permissions": [
        "proxy",
        "tabs",
        "unlimitedStorage",
        "storage",
        "<all_urls>",
        "webRequest",
        "webRequestBlocking"
    ],
    "background": {
        "scripts": ["background.js"]
    },
    "minimum_chrome_version":"22.0.0"
}
"""

background_js = """
var config = {
        mode: "fixed_servers",
        rules: {
        singleProxy: {
            scheme: "http",
            host: "%s",
            port: parseInt(%s)
        },
        bypassList: ["localhost"]
        }
    };
chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});
function callbackFn(details) {
    return {
        authCredentials: {
            username: "%s",
            password: "%s"
        }
    };
}
chrome.webRequest.onAuthRequired.addListener(
            callbackFn,
            {urls: ["<all_urls>"]},
            ['blocking']
);
""" % (PROXY_HOST, PROXY_PORT, PROXY_USER, PROXY_PASS)

def get_chromedriver(use_proxy=False, user_agent=None):
    path = os.path.dirname(os.path.abspath(__file__))
    chrome_options = webdriver.ChromeOptions()
    if use_proxy:
        pluginfile = 'proxy_auth_plugin.zip'

        with zipfile.ZipFile(pluginfile, 'w') as zp:
            zp.writestr("manifest.json", manifest_json)
            zp.writestr("background.js", background_js)
        chrome_options.add_extension(pluginfile)
    if user_agent:
        chrome_options.add_argument('--user-agent=%s' % user_agent)
    driver = webdriver.Chrome(
        os.path.join(path, 'chromedriver'),
        chrome_options=chrome_options)
    return driver

def main():
    driver = get_chromedriver(use_proxy=True)
    driver.get('https://ifconfig.me/)

if __name__ == '__main__':
    main()

source link

Vulcanology answered 16/9, 2022 at 11:52 Comment(0)
P
4

Try by Setting up FirefoxProfile

from selenium import webdriver
import time


"Define Both ProxyHost and ProxyPort as String"
ProxyHost = "54.84.95.51" 
ProxyPort = "8083"



def ChangeProxy(ProxyHost ,ProxyPort):
    "Define Firefox Profile with you ProxyHost and ProxyPort"
    profile = webdriver.FirefoxProfile()
    profile.set_preference("network.proxy.type", 1)
    profile.set_preference("network.proxy.http", ProxyHost )
    profile.set_preference("network.proxy.http_port", int(ProxyPort))
    profile.update_preferences()
    return webdriver.Firefox(firefox_profile=profile)


def FixProxy():
    ""Reset Firefox Profile""
    profile = webdriver.FirefoxProfile()
    profile.set_preference("network.proxy.type", 0)
    return webdriver.Firefox(firefox_profile=profile)


driver = ChangeProxy(ProxyHost ,ProxyPort)
driver.get("http://whatismyipaddress.com")

time.sleep(5)

driver = FixProxy()
driver.get("http://whatismyipaddress.com")

This program tested on both Windows 8 and Mac OSX. If you are using Mac OSX and if you don't have selenium updated then you may face selenium.common.exceptions.WebDriverException. If so, then try again after upgrading your selenium

pip install -U selenium
Permatron answered 3/7, 2016 at 10:10 Comment(0)
T
3

The answers above and on this question either didn't work for me with Selenium 3.14 and Firefox 68.9 on Linux, or are unnecessarily complex. I needed to use a WPAD configuration, sometimes behind a proxy (on a VPN), and sometimes not. After studying the code a bit, I came up with:

from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile

proxy = Proxy({'proxyAutoconfigUrl': 'http://wpad/wpad.dat'})
profile = FirefoxProfile()
profile.set_proxy(proxy)
driver = webdriver.Firefox(firefox_profile=profile)

The Proxy initialization sets proxyType to ProxyType.PAC (autoconfiguration from a URL) as a side-effect.

It also worked with Firefox's autodetect, using:

from selenium.webdriver.common.proxy import ProxyType

proxy = Proxy({'proxyType': ProxyType.AUTODETECT})

But I don't think this would work with both internal URLs (not proxied) and external (proxied) the way WPAD does. Similar proxy settings should work for manual configuration as well. The possible proxy settings can be seen in the code here.

Note that directly passing the Proxy object as proxy=proxy to the driver does NOT work--it's accepted but ignored (there should be a deprecation warning, but in my case I think Behave is swallowing it).

Tung answered 5/6, 2020 at 22:28 Comment(0)
T
3

This worked for me and allow to use an headless browser, you just need to call the method passing your proxy.

def setProxy(proxy):
        options = Options()
        options.headless = True
        #options.add_argument("--window-size=1920,1200")
        options.add_argument("--disable-dev-shm-usage")
        options.add_argument("--no-sandbox")
        prox = Proxy()
        prox.proxy_type = ProxyType.MANUAL
        prox.http_proxy = proxy
        prox.ssl_proxy = proxy
        capabilities = webdriver.DesiredCapabilities.CHROME
        prox.add_to_capabilities(capabilities)
        return webdriver.Chrome(desired_capabilities=capabilities, options=options, executable_path=DRIVER_PATH)
Trilingual answered 21/8, 2020 at 10:30 Comment(0)
F
3

Surprised to see no examples of using authenticated proxies.

October 2022 solution for authenticated proxies (Firefox & Chrome):

from selenium import webdriver

PROXY_HOST = "0.0.0.0";
PROXY_PORT = "0000"
PROXY_USERNAME = "user"
PROXY_PASS = "pass"

# If you're using Firefox
profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http",PROXY_HOST) 
profile.set_preference("network.proxy.http_port", PROXY_PORT) 
fp.set_preference('network.proxy.no_proxies_on', 'localhost, 127.0.0.1')
credentials = '%s:%s' % (PROXY_USERNAME, PROXY_PASS)
credentials = b64encode(credentials.encode('ascii')).decode('utf-8')
fp.set_preference('extensions.closeproxyauth.authtoken', credentials)
driver = webdriver.Firefox(firefox_profile=profile)

# If you're using Chrome
chrome_options = WebDriver.ChromeOptions()
options.add_argument('--proxy-server=http://%s:%s@%s:%s' % (PROXY_HOST, PROXY_PORT, PROXY_USERNAME, PROXY_PASS))
driver = webdriver.Chrome(executable_path='chromedriver.exe', chrome_options=chrome_options)

# Proxied request
driver.get("https://www.google.com")
Foeman answered 9/10, 2022 at 14:50 Comment(2)
options.add_argument('--proxy-server=http://%s:%s@%s:%s' % (PROXY_HOST, PROXY_PORT, PROXY_USERNAME, PROXY_PASS)) the username & password should appear before the hostnameServia
Also, the chrome arguements doesn't work. Chrome cannot handle proxy authentication via arguments, the above code results in ERR_NO_SUPPORTED_PROXIES error on chromeServia
I
2

As stated by @Dugini, some config entries have been removed. Maximal:

webdriver.DesiredCapabilities.FIREFOX['proxy'] = {
    "httpProxy":PROXY,
    "ftpProxy":PROXY,
    "sslProxy":PROXY,
    "noProxy":[],
    "proxyType":"MANUAL"
 }
Incense answered 27/4, 2019 at 15:20 Comment(0)
I
1

Scraping the data from any online source is quite easy when scraping APIs are used. You can try using scraper API to scrape the information from webpages and it automatically parses the web data. API can be integrated into your source code as well. Other than using API to scrape data, you can try the under-mentioned source code in beautiful soup to scrape data using CSS selectors. Before trying this code, please note that the select() method can be utilized to find numerous elements. Along with that, select_one() to be used search single element.

Source Code:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
PROXY = "177.46.141.143:59393" #your proxy  (ip address: port no)
chrome_options = WebDriverWait.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)
chrome = webdriver.Chrome(chrome_options=chrome_options)
chrome.get("https://www.ipchicken.com/")
Inelegance answered 29/6, 2021 at 18:51 Comment(0)
R
1

January 2023 solution with authentication and SSL for Firefox

from selenium import webdriver

PROXY_HOST = "ip"
PROXY_PORT = "port"
PROXY_USERNAME = "username"
PROXY_PASS = "password"

profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference('network.proxy.ssl_port', int(PROXY_PORT))
profile.set_preference('network.proxy.ssl', PROXY_HOST)
profile.set_preference("network.proxy.http", PROXY_HOST)
profile.set_preference("network.proxy.http_port", int(PROXY_PORT))
profile.set_preference("network.proxy.no_proxies_on", 'localhost, 127.0.0.1')
profile.set_preference("network.proxy.socks_username", PROXY_USERNAME)
profile.set_preference("network.proxy.socks_password", PROXY_PASS)
profile.update_preferences()

driver = webdriver.Firefox(firefox_profile=profile)

driver.get("https://2ip.ru")

Also there is a possibility you might need to lock in your ip address in proxy provider account settings. Otherwise you will be prompt to enter credentials on page load.

Roswald answered 29/1, 2023 at 13:14 Comment(0)
T
0

Authenticated proxy support. September 2023

I know of 2 ways to support authenticated proxies in selenium. I will showcase both.

Method 1: selenium-wire

If you're willing to download another library, which extends the base selenium library. Authenticated proxy support can be added via the selenium-wire library.

It should be noted, using this method will cause the browser to state the connection is 'unsecure' as seleniumwire uses it's own root certificate to intercept requests. It is possible to download seleniumwire's certificate and prevent this, however for this example i will not do that and will instead choose to ignore the unsecured warning. If this is not ideal, and you do not wish to download the root certificiate, see method 2. See here for more information regarding seleniumwire's certificate handling.

See the code below for an implementation utilising the selenium-wire library.

from seleniumwire import webdriver

proxy_host = "your_host"
proxy_port = 8888
proxy_user = "your_username"
proxy_pass = "your_password"

seleniumwire_options = {
    "proxy":{
        "http":f"http://{proxy_user}:{proxy_pass}@{proxy_host}:{proxy_port}",
        "verify_ssl":False,
    }
}

chrome_options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors') #ignore the 'unsecured' warning caused by seleniumwires root certificate (if certificate is not installed)

driver = webdriver.Chrome(
    options=chrome_options,
    seleniumwire_options=seleniumwire_options,
)

driver.get('http://httpbin.org/ip')
input()
driver.quit()

Method 2: Chrome-extension

Another way to use a proxy in selenium is by using a chrome extension that automatically loads the proxy to chrome, and authenticates it with the relative auth credentials when requested.

I will use a chrome extension, similar to that posed by @Jackssn , but rewritten to support manifest v3; as v2 has become deprecated in 2023.

See the code below:

from selenium import webdriver
from pathlib import Path
import shutil

## js strings to be written to file when creating chrome extension

ext_json_manifest = """{
    "manifest_version": 3,
    "name": "Proxy Loader",
    "version": "1.0.0",
    "background":{
        "service_worker":"service-worker.js"
    },
    "permissions":[
        "proxy",
        "webRequest",
        "webRequestAuthProvider"
    ],
    "host_permissions":[
        "<all_urls>"
    ],
    "minimum_chrome_version": "108"
}"""

ext_js_serviceworker = """host = "%s"
port = %s
username = "%s"
password = "%s"


var config = {
    mode:"fixed_servers",
    rules:{
        singleProxy:{
            host:host,
            port:port,
            scheme:"http",
        },
        bypassList:["localhost"] //wont use a proxy for these sites
    }
}

chrome.proxy.settings.set(
    {value:config,scope:"regular"},
    function(){}
)

// supply proxy auth credentials when prompted.
chrome.webRequest.onAuthRequired.addListener(
    function(details) { //
        return {
            authCredentials:{
                username:username,
                password:password
            }
        }
    },
    {
        urls:["<all_urls>"] //which urls to provide credentials for
    },
    ["blocking"] //block the request until auth details have been supplied
)"""

## your proxy details
proxy_host = "your_host"
proxy_port = 8888
proxy_user = "your_username"
proxy_pass = "your_password"

## create extension
ext_path = "./extension/"
Path(ext_path).mkdir(parents=True,exist_ok=True)

with open(str(Path(ext_path).joinpath("manifest.json")),"w+") as f:
    f.write(ext_json_manifest)
    f.close()

with open(str(Path(ext_path).joinpath("service-worker.js")),"w+") as f:
    f.write(ext_js_serviceworker % (proxy_host,proxy_port,proxy_user,proxy_pass))
    f.close()

## configure and create driver
options = webdriver.ChromeOptions()
options.add_argument(f"--load-extension={str(Path(ext_path).absolute())}")

driver = webdriver.Chrome(
    options=options
)

## delete extension once loaded.
shutil.rmtree(ext_path)

## check selenium is using the proxy.
driver.get('http://httpbin.org/ip')
input()
driver.quit()
Tamica answered 15/9, 2023 at 16:14 Comment(0)
N
-4

try running tor service, add the following function to your code.

def connect_tor(port):

socks.set_default_proxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', port, True)
socket.socket = socks.socksocket

def main():

connect_tor()
driver = webdriver.Firefox()
Namaqualand answered 10/7, 2018 at 9:6 Comment(2)
This post lacks information, connect_tor() and main() functions have no correct indentation, and connect_tor() call is missing the mandatory argument "port" on the example. Which tor port should I use? Where can I get Tor's port information?Honan
Tor network (even if adds an extra layer of anonymity) is very slow.Orthopteran

© 2022 - 2024 — McMap. All rights reserved.