Apache HTTP client 4.3 credentials per request

Question

I have been having a look to a digest authentication example at:

http://hc.apache.org/httpcomponents-client-4.3.x/examples.html

In my scenario the there are several threads issuing HTTP requests and each of them has to be authenticated with their own set of credentials. Additionally, please consider this question is probably very specific for the Apache HTTP client 4.3 onwards, 4.2 handles authentication probably in a different way, although I didn't check it myself. That said, there goes the actual question.

I want to use just one client instance (static member of the class, that is threadsafe) and give it a connection manager to support several concurrent requests. The point is that each request will provide different credentials and I am not seeing the way to assign credentials per request as the credentials provider is set when building the http client. From the link above:

[...]

    HttpHost targetHost = new HttpHost("localhost", 80, "http");
    CredentialsProvider credsProvider = new BasicCredentialsProvider();
    credsProvider.setCredentials(
            new AuthScope(targetHost.getHostName(), targetHost.getPort()),
            new UsernamePasswordCredentials("username", "password"));
    CloseableHttpClient httpclient = HttpClients.custom()
            .setDefaultCredentialsProvider(credsProvider).build();

[...]

Checking:

http://hc.apache.org/httpcomponents-client-ga/tutorial/html/authentication.html#d5e600

The code sample in point 4.4 (seek 4.4. HTTP authentication and execution context), seems to say that the HttpClientContext is given the auth cache and the credentials provider and then is passed to the HTTP request. Next to it the request is executed and it seems that the client will get credentials filtering by the host in the HTTP request. In other words: if the context (or the cache) has valid credentials for the target host of the current HTTP request, he will use them. The problem for me is that different threads will perform different requests to the same host.

Is there any way to provide custom credentials per HTTP request?

Thanks in advance for your time! :)

Sacrosanct

Possible duplicate of Authenticating a single request with httpclient 4.x – Degrease

Answer 1

The problem for me is that different threads will perform different requests to the same host.

Why should this be a problem? As long as you use a different HttpContext instance per thread, execution contexts of those threads are going to be completely indepenent

CloseableHttpClient httpclient = HttpClients.createDefault();
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("user:pass"));
HttpClientContext localContext = HttpClientContext.create();
localContext.setCredentialsProvider(credentialsProvider);

HttpGet httpget = new HttpGet("http://localhost/");

CloseableHttpResponse response = httpclient.execute(httpget, localContext);
try {
    EntityUtils.consume(response.getEntity());
} finally {
    response.close();
}

Leix

Sorry, my question was unprecise: The problem for me is that different threads with different credentials each will perform requests to the same host. So, having only one client and only one credentials provider instance for all the threads (each thread will place in it's credentials) and one context per thread, how does the (only) client know what credentials to use for each thread??? Thanks for your time in advance! – Sacrosanct

@FranciscoCarriedoScher: I am not sure I understand the difficulty you are having here. There is absolutely NOTHING that prevents you from having a separate credentials provider per thread / context – Leix

The difficulty was to hold a single CredentialsProvider for all the threads (as it seems to be internally prepared to manage a set of credentials), but in the end it seems that, at least the BasicCredentialsProvider does manage only one credential per instance. So, using a new credential provider instance per thread seems the way to go and your example works OK. – Sacrosanct

What happens when a single thread is going to make several requests. Some of the need basic auth, others don't. Is there a way to clear the context? I really don't understand why Apache doesn't provide a way to pass the credentials to the client for a particular request – Thermotensile

@ryber: Oh yeah, those idiots. And what if request execution involves multiple redirects to different auth realms, and proxy authentication on top of that, or any combination of several dozen parameters or customization strategies? – Leix

I don't know if I would go so far as to call them idiots but it's just weird that they don't support a method that's exposed in pretty much every other HTTP client I know of. Java or otherwise. – Thermotensile

In my example. I have a collection of URL's in a database. Some have credentials, some don't. I'm going to loop over them and call each one and collect the results. Each run could have a different collection of URLs. I don't think this is a weird scenario, – Thermotensile

@ryber: HTTP auth is massively more complex than a vulgar username / password combo could possibly represent. If Apache HttpClient is too complex to your liking you are welcome to use any other HTTP client, Java or otherwise – Leix

Answer 2

I have a similar issue.

I must call n-times a service with a single system user, authenticated with NTLM. I want to do this using multiple threads. What I came up with is creating a single HTTPClient with no default credential provider. When a request needs to be performed I use an injected CredentialProviderFactory into the method performing the request (in a specific thread). Using this I get a brand new CredentialsProvider and I put this into a Context (created in the thread). Then I call the execute method on the client using the overload execute(method, context).

class MilestoneBarClient implements IMilestoneBarClient {

private static final Logger log = LoggerFactory.getLogger(MilestoneBarClient.class);
private MilestoneBarBuilder builder;
private CloseableHttpClient httpclient;
private MilestoneBarUriBuilder uriBuilder;
private ICredentialsProviderFactory credsProviderFactory;


MilestoneBarClient(CloseableHttpClient client, ICredentialsProviderFactory credsProviderFactory, MilestoneBarUriBuilder uriBuilder) {
    this(client, credsProviderFactory, uriBuilder, new MilestoneBarBuilder());
}

MilestoneBarClient(CloseableHttpClient client, ICredentialsProviderFactory credsProviderFactory, MilestoneBarUriBuilder uriBuilder, MilestoneBarBuilder milestoneBarBuilder) {
    this.credsProviderFactory = credsProviderFactory;
    this.uriBuilder = uriBuilder;
    this.builder = milestoneBarBuilder;
    this.httpclient = client;
}

// This method is called by multiple threads
@Override
public MilestoneBar get(String npdNumber) {
    log.debug("Asking milestone bar info for {}", npdNumber);

    try {
        String url = uriBuilder.getPathFor(npdNumber);
        log.debug("Building request for URL {}", url);
        HttpClientContext localContext = HttpClientContext.create();
        localContext.setCredentialsProvider(credsProviderFactory.create());

        HttpGet httpGet = new HttpGet(url);

        long start = System.currentTimeMillis();
        try(CloseableHttpResponse resp = httpclient.execute(httpGet, localContext)){
[...]

For some reasons I sometimes get an error, but I guess it's an NTLMCredentials issue (not being thread-safe...).

In your case, you could probably pass the factory to the get methods instead of passing in creation.

Inotropic