Browser-mob Proxy is not capturing the network traffic with chrome driver

Question

I am using this code to integrate the browser mob proxy with maven dependency net.lightbody.bmp browsermob-core 2.1.5

its not capturing the network requests at all, I am getting this kind of har file:

{
   "log":{
      "version":"1.2",
      "creator":{
         "name":"BrowserMob Proxy",
         "version":"2.1.0-beta-6-littleproxy",
         "comment":""
      },
      "pages":[
         {
            "id":"11",
            "startedDateTime":"2017-10-26T17:28:42.501+05:30",
            "title":"11",
            "pageTimings":{
               "comment":""
            },
            "comment":""
         }
      ],
      "entries":[],
      "comment":""
   }
}

---

package lenskart.tests;

import java.io.File;
import java.net.UnknownHostException;
import java.util.concurrent.TimeUnit;

import org.openqa.selenium.Proxy;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.testng.annotations.Test;

import net.lightbody.bmp.BrowserMobProxyServer;
import net.lightbody.bmp.client.ClientUtil;
import net.lightbody.bmp.mitm.manager.ImpersonatingMitmManager;
import net.lightbody.bmp.proxy.CaptureType;

public class ProxyTestClass {

    @Test
    public static void main() throws Exception {
        // TODO Auto-generated method stub

        BrowserMobProxyServer browserMobProxy = new BrowserMobProxyServer();
        browserMobProxy.setTrustAllServers(true);
        browserMobProxy.setMitmManager(ImpersonatingMitmManager.builder().trustAllServers(true).build());
        browserMobProxy.start(0);

        System.out.println("Port Started On: " + browserMobProxy.getPort());
        System.setProperty("webdriver.chrome.driver", "/Users/pankaj.katiyar/Desktop/Automation/Lenskart_Automation/tpt/drivers/chromedriver");

        browserMobProxy.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT, CaptureType.RESPONSE_HEADERS);

        WebDriver driver = getDriver_CapProxy(browserMobProxy);

        driver.get("http://www.lenskart.com");
        driver.navigate().to("http://www.google.com");

        driver.quit();

        browserMobProxy.stop();

        browserMobProxy.newHar("11");
        browserMobProxy.getHar().writeTo(new File("/Users/pankaj.katiyar/Desktop/Automation/Lenskart_Automation/har"));
        ;
        System.out.println("Loaded browser ");
    }


    public static WebDriver getDriver_CapProxy(BrowserMobProxyServer browserMobProxy) throws UnknownHostException {
        Proxy proxy = ClientUtil.createSeleniumProxy(browserMobProxy);
        proxy.setHttpProxy("localhost:" + browserMobProxy.getPort());

        DesiredCapabilities cap = new DesiredCapabilities();

        ChromeOptions options = new ChromeOptions();
        options.addArguments("--ignore-certificate-errors");

        cap.setCapability(ChromeOptions.CAPABILITY, options);
        cap.setCapability(CapabilityType.PROXY, proxy);

        WebDriver driver = new ChromeDriver(options);

        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
        return driver;
    }
}

Anagram

Answer 1

There's two issues with your code.

First the selenium proxy is assigned to the capabilities, but the instance is never used. So either assign the options to the capabilities, or directly assign the selenium proxy to the options:

Proxy seleniumProxy  = ClientUtil.createSeleniumProxy(browserMobProxy);

ChromeOptions options = new ChromeOptions();
options.addArguments("--ignore-certificate-errors");
options.setCapability(CapabilityType.PROXY, seleniumProxy);

WebDriver driver = new ChromeDriver(options);
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

Second, you are calling browserMobProxy.newHar at the end of the recording when it should be at the beginning:

browserMobProxy.newHar("11");

driver.get("http://www.lenskart.com");
driver.navigate().to("http://www.google.com");

browserMobProxy.getHar().writeTo(new File("/Users/pankaj.katiyar/Desktop/Automation/Lenskart_Automation/har"));

driver.quit();
browserMobProxy.stop();

Joettajoette

After proxy is applied - I am getting : This page isn’t working www.google.co.in didn’t send any data. ERR_EMPTY_RESPONSE – Anagram

It could be an issue with the redirection. Try with https://www.google.com instead of http://www.google.com – Joettajoette

Answer 2

I ran your code and only two changes you need are, change

WebDriver driver = new ChromeDriver(options);

to

WebDriver driver = new ChromeDriver(cap);

And move browserMobProxy.newHar("11"); before the navigate

browserMobProxy.newHar("11");
driver.get("http://www.lenskart.com");

Rest all is fine in your code. Once you do that Har is generated fine as shown in below screenshot

Shantae

After proxy is applied, I am getting - This page isn’t working www.google.co.in didn’t send any data. ERR_EMPTY_RESPONSE – Anagram

@PankajKumarKatiyar, let me check – Shantae

Try removing the setMitmManager and setTrustAllServers. It works for me after that on https links also – Shantae

Sorry, its not working, I tried after commenting these lines, browserMobProxy.setTrustAllServers(true); browserMobProxy.setMitmManager(ImpersonatingMitmManager.builder().trustAllServers(true).build()); Keep on getting - This page isn’t working – Anagram

I think I may have done some certificate setup for same in past, let me check – Shantae

Let us continue this discussion in chat. – Shantae

Hi, I am getting this same error, were you able to resolve this issue? – Peccary