Setting up browsermob proxy with ChromeDriver

Question

I'm trying to set up browsermob to work in my selenium project. I was looking for a way to use ChromeOptions to set the proxy, but all sources tell me to use ChromeOptions for everything else, then convert it into DesiredCapabilities before instantiating a new ChromeDriver instance.

This is my code:

ChromeOptions options = new ChromeOptions();
// Setting some chrome features here

ProxyServer proxyServer = new ProxyServer(4444);
proxyServer.start();

Proxy proxy = proxyServer.seleniumProxy();

DesiredCapabilities capabilities = DesiredCapabilities.chrome();

capabilities.setCapability(ChromeOptions.CAPABILITY, options);
capabilities.setCapability(CapabilityType.PROXY, proxy);

WebDriver driver = new ChromeDriver(capabilities); // Error happens here

I'm using Webdriver version 2.44 from the maven repositories. This is the error I get:

java.lang.IllegalAccessError: tried to access field com.google.gson.JsonNull.INSTANCE from class org.openqa.selenium.remote.BeanToJsonConverter

Does anyone know the cause or any alternative solutions for hooking up a proxy to chromedriver?

Montsaintmichel

Try this blog - carlduevel.net/get-selenium-to-run-with-proxy-authentication – Muskrat

Answer 1

If you're using an older version of browsermob-proxy, there might be some conflicts between Selenium's dependencies and BMP's. I'd recommend using the latest Selenium + building the latest BrowserMob Proxy from master.

Once you have the latest versions, you should be able to use Chrome + BMP the "usual" way:

        BrowserMobProxy proxy = new BrowserMobProxyServer();
        proxy.start(); // can specify a port here if you like

        // get the selenium proxy object
        Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);

        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);

        // if chromedriver isn't on your system path, you'll need to set this system property
        System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver");
        WebDriver driver = new ChromeDriver(capabilities);

        driver.get("https://www.google.com/");

Sofar

Answer 2

ChromeDriver doesn't support the proxy caps directly. But it does support passing command line args to the chrome process. And setting the http proxy is one of chrome command line switches. It can be set as follows:

DesiredCapabilities caps = DesiredCapabilities.chrome();    
ArrayList<String> switches = new ArrayList<String>();    
switches.add("--proxy-server=localhost:8080");    
caps.setCapability("chrome.switches", switches);    
webDriver = new ChromeDriver(caps);    

Demirep

I test this way with selenium 2.52 and bmp 2.1.0-beta4, perfect. The solution raised by jekh only works for firefox driver! thank you, guy! – Margalit

Answer 3

Browsermob-Proxy is a reliable solution, But while working with the remote grid machine, Browsermob-proxy isn't really helpful. Alternatively, I found this as a working solution for my setup.

Hopefully, it will be useful for someone with a similar setup.

1. Add the ModHeader extension to the chrome browser

How to download the Modheader? Link

ChromeOptions options = new ChromeOptions();
options.addExtensions(new File(C://Downloads//modheader//modheader.crx));

// Set the Desired capabilities 
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(ChromeOptions.CAPABILITY, options);

// Instantiate the chrome driver with capabilities
WebDriver driver = new RemoteWebDriver(new URL(YOUR_HUB_URL), options);

2. Go to the browser extensions and capture the Local Storage context ID of the ModHeader

3. Navigate to the URL of the ModHeader to set the Local Storage Context

.

// set the context on the extension so the localStorage can be accessed
driver.get("chrome-extension://idgpnmonknjnojddfkpgkljpfnnfcklj/_generated_background_page.html");

Where `idgpnmonknjnojddfkpgkljpfnnfcklj` is the value captured from the Step# 2

4. Now add the headers to the request using Javascript

.

   ((Javascript)driver).executeScript(
         "localStorage.setItem('profiles', JSON.stringify([{  title: 'Selenium', hideComment: true, appendMode: '', 
             headers: [                        
               {enabled: true, name: 'token-1', value: 'value-1', comment: ''},
               {enabled: true, name: 'token-2', value: 'value-2', comment: ''}
             ],                          
             respHeaders: [],
             filters: []
          }]));");

Where token-1, value-1, token-2, value-2 are the request headers and values that are to be added.

5. Now navigate to the required web-application.

   driver.get("your-desired-website");

Ammadas