Using Selenium RemoteWebDriver behind corporate proxy

Question

How can I connect to a selenium grid such as BrowserStack via RemoteWebDriver from behind a corporate proxy?

The application under test is outside the proxy and freely accessible from BrowserStack.

This Using Selenium RemoteWebDriver behind corporate proxy (Java) stackoverflow question asked the same question but I couldn't follow the accepted answer.

Karole

Answer 1

I managed to get something working based on the accepted answer in the linked stackoverflow question, here's my implementation in case anyone else is stuck on the same problem:

Example

import java.net.InetAddress;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.Map;

import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.NTCredentials;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.HttpClientBuilder;
import org.openqa.selenium.remote.CommandInfo;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.HttpCommandExecutor;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.http.HttpClient.Factory;

public class Example {
    public RemoteWebDriver connectViaProxy(DesiredCapabilities caps) {
        String proxyHost = "?";
        int proxyPort = 8080;
        String proxyUserDomain = "?";
        String proxyUser = "?";
        String proxyPassword = "?";

        URL url;

        try {
            url = new URL("http://bsuser:[email protected]/wd/hub");
        } catch (MalformedURLException e) {
            throw new RuntimeException(e.getMessage(), e);
        }

        HttpClientBuilder builder = HttpClientBuilder.create();

        HttpHost proxy = new HttpHost(proxyHost, proxyPort);

        CredentialsProvider credsProvider = new BasicCredentialsProvider();

        credsProvider.setCredentials(new AuthScope(proxyHost, proxyPort), new NTCredentials(proxyUser, proxyPassword, getWorkstation(), proxyUserDomain));

        if (url.getUserInfo() != null && !url.getUserInfo().isEmpty()) {
            credsProvider.setCredentials(new AuthScope(url.getHost(), (url.getPort() > 0 ? url.getPort() : url.getDefaultPort())), new UsernamePasswordCredentials(url.getUserInfo()));
        }

        builder.setProxy(proxy);
        builder.setDefaultCredentialsProvider(credsProvider);

        Factory factory = new MyHttpClientFactory(builder);

        HttpCommandExecutor executor = new HttpCommandExecutor(new HashMap<String, CommandInfo>(), url, factory);

        return new RemoteWebDriver(executor, caps);
    }

    private String getWorkstation() {
        Map<String, String> env = System.getenv();

        if (env.containsKey("COMPUTERNAME")) {
            // Windows
            return env.get("COMPUTERNAME");         
        } else if (env.containsKey("HOSTNAME")) {
            // Unix/Linux/MacOS
            return env.get("HOSTNAME");
        } else {
            // From DNS
            try
            {
                return InetAddress.getLocalHost().getHostName();
            }
            catch (UnknownHostException ex)
            {
                return "Unknown";
            }
        }
    }
}

MyHttpClientFactory

import java.net.URL;

import org.apache.http.impl.client.HttpClientBuilder;
import org.openqa.selenium.remote.internal.ApacheHttpClient;

public class MyHttpClientFactory implements org.openqa.selenium.remote.http.HttpClient.Factory {
    final HttpClientBuilder builder; 

    public MyHttpClientFactory(HttpClientBuilder builder) {
        this.builder = builder;
    }

    @Override
    public org.openqa.selenium.remote.http.HttpClient createClient(URL url) {
        return new ApacheHttpClient(builder.build(), url);
    }
}

Karole

This was extremely useful, thanks. I haven't found any other documentation talking about this issue. It worked well in our corporate network. – Darcidarcia

The Apache Http Client is removed in selenium any alternative ways to set the remote driver proxy – Stinky

I've been wondering myself, haven't had a chance to look into this yet. – Karole

Answer 2

Adding to the answer above by Andrew, to make this work with Appium change the

HttpCommandExecutor executor = new HttpCommandExecutor(new HashMap<String, CommandInfo>(), url, factory);

to

HttpCommandExecutor executor = new HttpCommandExecutor(MobileCommand.commandRepository, url, factory);

Burleson

The question doesn't ask for Appium. – Remissible

Answer 3

I've reworked Andrew Sumner's solution slightly and taken some out in case someone like me wants to just quickly funnel their WebDriver traffic through Fiddler to see the traffic.

import java.net.MalformedURLException;
import java.net.URL;
import java.util.HashMap;

import org.apache.http.HttpHost;
import org.apache.http.impl.client.HttpClientBuilder;
import org.openqa.selenium.ie.InternetExplorerOptions;
import org.openqa.selenium.remote.CommandInfo;
import org.openqa.selenium.remote.HttpCommandExecutor;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.remote.http.HttpClient;
import org.openqa.selenium.remote.http.HttpClient.Factory;
import org.openqa.selenium.remote.internal.ApacheHttpClient;

public class ProxiedRemoteExample {
    private static final String PROXY_HOST = "localhost";
    private static final int PROXY_PORT = 8888;

    public ProxiedRemoteExample() throws MalformedURLException {
        InternetExplorerOptions ieOptions = new InternetExplorerOptions();
        RemoteWebDriver driver = new RemoteWebDriver(new HttpCommandExecutor(new HashMap<String, CommandInfo>(),
                new URL("http://localhost:5555/"), new Factory() {
                    private HttpClientBuilder builder;
                    {
                        builder = HttpClientBuilder.create();
                        builder.setProxy(new HttpHost(PROXY_HOST, PROXY_PORT));
                    }
                    @Override
                    public HttpClient createClient(URL url) {
                        return new ApacheHttpClient(builder.build(), url);
                    }

                }), ieOptions);
    }

}

Furnace

Answer 4

With org.seleniumhq.selenium:selenium-java:4.0.0-beta-3 I had to apply proxy settings in the following way:

• configure async http client to use proxy settings
  • create ahc.properties file under org/asynchttpclient/config folder
  • file content: org.asynchttpclient.useProxyProperties = true
• configure JVM proxy properties
  • System.getProperties().setProperty("http.proxyHost", "yourProxyHost")
  • System.getProperties().setProperty("http.proxyPort", "yourProxyPort")

Foliolate

Answer 5

After v4.0.0-alpha-7, the OkHttp implementation of the http client has been removed.

But I found an easy way to set the proxy after rereading Andrew's answer:

var url = new URL("Your grid endpoint like browserstack or saucelabs");
var proxy = new java.net.Proxy(java.net.Proxy.Type.HTTP,
                    new InetSocketAddress("Your proxy", 8080));
var executor = new HttpCommandExecutor(ClientConfig.defaultConfig().baseUrl(url).proxy(proxy));
var driver = new RemoteWebDriver(executor, browserOptions); 

Rondon

Answer 6

For who using Selenium with Java and want to add a authentication proxy in RemoteWebdriver. I tested and it worked for me. Hope it saves your time.

DesiredCapabilities capabilities = new DesiredCapabilities();
    Proxy proxy = new Proxy();
    proxy.setHttpProxy(proxyItem);
    proxy.setSslProxy(proxyItem);
    proxy.setFtpProxy(proxyItem);
    proxy.setSocksUsername(userProx);
    proxy.setSocksPassword(passProx);
    proxy.setAutodetect(false);
    proxy.setProxyType(Proxy.ProxyType.MANUAL);
    capabilities.setCapability(CapabilityType.PROXY, proxy);
    RemoteWebDriver.builder();
    WebDriver driver = RemoteWebDriver.builder()
            .oneOf(chromeOptions)
            .address(new URI("http://localhost:4444/wd/hub"))
            .setCapability(CapabilityType.PROXY, proxy)
            .build();
    AddHasAuthentication addHasAuthentication = new AddHasAuthentication();
    addHasAuthentication.getImplementation(
                    capabilities, new RemoteExecuteMethod((RemoteWebDriver) driver))
            .register(UsernameAndPassword.of(userProxy, passProxy));
    driver.get("https://www.whatismyip.com");

Lenticel