Selenium using Python: enter/provide http proxy password for firefox

Question

I want to use selenium with a proxy which is password protected. The proxy is not fixed, but a variable. So this has to be done in the code (just setting up firefox on this particular machine to work with the proxy is less-than-ideal). So far I have the following code:

fp = webdriver.FirefoxProfile()
# Direct = 0, Manual = 1, PAC = 2, AUTODETECT = 4, SYSTEM = 5
fp.set_preference("network.proxy.type", 1)

fp.set_preference("network.proxy.http", PROXY_HOST)
fp.set_preference("network.proxy.http_port", PROXY_PORT)

driver = webdriver.Firefox(firefox_profile=fp)
driver.get("http://whatismyip.com")

At this point, the dialog pops up requesting the proxy user/pass.

Is there an easy way to either:

1. Type in the user/pass in the dialog box.
2. Provide the user/pass at an earlier stage.

Moll

hey mate, have you found solutions yet? – Wendel

@Wendel have you found solutions yet? – Engle

@ArsenIbragimov have you found solutions yet? – Entasis

to use firefox with an extension, see this: https://mcmap.net/q/67781/-how-to-set-proxy-authentication-user-amp-password-using-python-selenium – Petrinapetrine

Answer 1

Selenium can't do that by itself. The only way I found helpful is described here. To be short, you need to add a browser extension on fly that does the authentication. It's much simpler than may seem to be.

Here is how it works for Chrome (in my case):

1. Create a zip file proxy.zip containing two files:

background.js

var config = {
    mode: "fixed_servers",
    rules: {
      singleProxy: {
        scheme: "http",
        host: "YOU_PROXY_ADDRESS",
        port: parseInt(YOUR_PROXY_PORT)
      },
      bypassList: ["foobar.com"]
    }
  };

chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});

function callbackFn(details) {
    return {
        authCredentials: {
            username: "YOUR_PROXY_USERNAME",
            password: "YOUR_PROXY_PASSWORD"
        }
    };
}

chrome.webRequest.onAuthRequired.addListener(
        callbackFn,
        {urls: ["<all_urls>"]},
        ['blocking']
);

Don't forget to replace YOUR_PROXY_* to your settings.

manifest.json

{
    "version": "1.0.0",
    "manifest_version": 2,
    "name": "Chrome Proxy",
    "permissions": [
        "proxy",
        "tabs",
        "unlimitedStorage",
        "storage",
        "<all_urls>",
        "webRequest",
        "webRequestBlocking"
    ],
    "background": {
        "scripts": ["background.js"]
    },
    "minimum_chrome_version":"22.0.0"
}

2. Add the created proxy.zip as an extension

   from selenium import webdriver
   from selenium.webdriver.chrome.options import Options
   
   chrome_options = Options()
   chrome_options.add_extension("proxy.zip")
   
   driver = webdriver.Chrome(executable_path='chromedriver.exe', chrome_options=chrome_options)
   driver.get("http://google.com")
   driver.close()
   

That's it. For me that worked like a charm. If you need to create proxy.zip dynamically or need PHP example then go to the original post

Newport

Mike, it does not work with me - joxi.net/LmG3OVNTRbEMam The last line says WebDriverException: Message: unknown error: cannot process extension #1 from unknown error: cannot read manifest (Driver info: chromedriver=2.29.461591 (62ebf098771772160f391d75e589dc567915b233),platform=Windows NT 10.0.14393 x86_64) – Mountbatten

so how to communicate with the addition extension? support i want to change the proxy dynamiclly – Sophistic

I've tried this for a SOCKS5 and changed scheme: "http", to scheme: "socks5" but no luck. Any ideas how this changes for socks5 instead of HTTP? – Actinia

@IgorSavinkin Check the suffix of your manifest file -> .json and not .js – Bogeyman

Worked! Thank you! – Breeks

This solution doesn't work for me anymore, did something change? someone got an update on it? – Bogeyman

I love you ! Thx ! :3 this work always in 2019 and it's the only solution that I've see work for python + selenium + http proxy with basic auth (for chrome) and I really tried a lot... So, sincerely thank you very much – Eichmann

Thank you! Worked with C#, Selenium, ChromeDriver – Talkathon

Unbelievable how the question asks for Firefox and yet the Chrome answer has so many uppvotes... – Dexamethasone

Answer 2

Code worked for me

from selenium import webdriver

browser=webdriver.Firefox()

def login(browser):

    alert=browser.switch_to_alert()
    alert.send_keys("username"+webdriver.common.keys.Keys.TAB+"password")
    alert.accept() 

Inger

Ugly method, but the only one which worked for me, just a new way to switch to alert: driver.switch_to.alert.send_keys(user + TAB + pass) driver.switch_to.alert.accept() Important send all text together instead of call send_keys() three times. – Granoff

Answer 3

Selenium 4 has a built-in solution for Basic Auth

// This "HasAuthentication" interface is the key!
HasAuthentication authentication (HasAuthentication) driver;

// You can either register something for all sites
authentication.register(() -> new UsernameAndPassword("admin", "admin"));

// Or use something different for specific sites
authentication.register(
  uri -> uri.getHost().contains("mysite.com"),

new UsernameAndPassword("AzureDiamond", "hunter2"));

https://www.selenium.dev/blog/2021/a-tour-of-4-authentication/

Tomlinson

Answer 4

Did you try PROXY_HOST = "http://username:[email protected]"?

Also:

Starting with Selenium 2.0 beta 1, there is built in support for handling popup dialog boxes.

Peder

PROXY_HOST = "username:[email protected]" is not working. As to popup dialog handling, how do i locate user/pass elements to input? – Wendel

@Shane, did you try PROXY_HOST = "username:[email protected]" or PROXY_HOST = "http://username:[email protected]". The second should pass the basic auth. Regarding the elements location: if you can switch to the dialog, you always can navigate its elements sending tab key signal. – Peder

This works only for outdated versions of chrome and selenium – Azores