Apache HttpClient Digest authentication

Question

Basically what I need to do is to perform digest authentication. First thing I tried is the official example available here. But when I try to execute it(with some small changes, Post instead of the the Get method) I get a

org.apache.http.auth.MalformedChallengeException: missing nonce in challange
at org.apache.http.impl.auth.DigestScheme.processChallenge(DigestScheme.java:132)

When this failed I tried using:

DefaultHttpClient client = new DefaultHttpClient();
client.getCredentialsProvider().setCredentials(new AuthScope(null, -1, null), new UsernamePasswordCredentials("<username>", "<password>"));

HttpPost post = new HttpPost(URI.create("http://<someaddress>"));
        List<NameValuePair> nvps = new ArrayList<NameValuePair>();
nvps.add(new BasicNameValuePair("domain", "<username>"));
post.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));

DigestScheme digestAuth = new DigestScheme();
digestAuth.overrideParamter("algorithm", "MD5");
digestAuth.overrideParamter("realm", "http://<someaddress>");
digestAuth.overrideParamter("nonce", Long.toString(new Random().nextLong(), 36));
digestAuth.overrideParamter("qop", "auth");
digestAuth.overrideParamter("nc", "0");
digestAuth.overrideParamter("cnonce", DigestScheme.createCnonce());

Header auth = digestAuth.authenticate(new
      UsernamePasswordCredentials("<username>", "<password>"), post);
System.out.println(auth.getName());
System.out.println(auth.getValue());
post.setHeader(auth);


HttpResponse ret = client.execute(post);
ByteArrayOutputStream v2 = new ByteArrayOutputStream();
ret.getEntity().writeTo(v2);
System.out.println("----------------------------------------");
System.out.println(v2.toString());
System.out.println("----------------------------------------");
System.out.println(ret.getStatusLine().getReasonPhrase());
System.out.println(ret.getStatusLine().getStatusCode());

At first I have only overridden "realm" and "nonce" DigestScheme parameters. But it turned out that PHP script running on the server requires all other params, but no matter if I specify them or not DigestScheme doesn't generate them in the Authorization RequestPreperty when I call its authenticate() method. And PHP script returns HTTP response code 200 with a message that PHP script requires cnonce, nc and qop parameters.

I've been struggling with this for two days, and no luck. Based on everything I think that the cause of the problem is the PHP script. It looks to me that it doesn't send a challenge when app tries to access it unauthorized.

Any ideas anyone?

Edit: One more thing, I've tried connecting with cURL and it works.

Sophisticated

Looks like the server is sending non-standard challenges. Can you try it in browser and get a header trace? – Ohaus

I've made a HttpURLConnection to server and printed out the response headers. Below is how a challenge looks like: Key = www-authenticate Value = Digest realm="REST API" qop="auth" nonce="4c063992df3dd" opaque="aba3d4b49c454e1974970e7b5514b001" – Sophisticated

Answer 1

This code snippet worked for me. You have to provide the realm which you can get by looking at the 401 response header you get from the host.

val credsProvider = new BasicCredentialsProvider();
credsProvider.setCredentials(AuthScope.ANY,
  new UsernamePasswordCredentials(user, password));
val authCache = new BasicAuthCache();
val digestScheme = new DigestScheme();

digestScheme.overrideParamter("realm", "**Name of the Realm**");
// Nonce value
digestScheme.overrideParamter("nonce", "whatever");

authCache.put(targetHost, digestScheme);

context.setCredentialsProvider(credsProvider);
context.setAuthCache(authCache);

val httpget = new HttpGet(url);

val response = httpClient.execute(targetHost, httpget, context);

Unlade

This answer was very helpful but it's really pseudo-code. Here's an actual working example that gets realm, nonce, and opaque values from the server: #53028503 – Grover

Answer 2

I managed to do a Digest login using digestScheme after verifying the code.

digestAuth.processChallenge(null);

Forces the previous input parameters to be interpreted. The null parameter is a header, based on the header sent, if any.

Now qop/nc is used and digestScheme works as required. Running it on android

digestAuth.overrideParamter("algorithm", "MD5");
digestAuth.overrideParamter("realm", serverRealm);
digestAuth.overrideParamter("nonce", Long.toString(new Random().nextLong(), 36));
digestAuth.overrideParamter("qop", "auth");//   not effective 
digestAuth.overrideParamter("nc",""+sequence);//nt effective 
digestAuth.overrideParamter("cnonce", DigestScheme.createCnonce());
digestAuth.overrideParamter("opaque","ba897c2f0f3de9c6f52d");
String err;
try
{
    digestAuth.processChallenge(null);
    //force  qop in use  chalange  on return header ????!!!!
}
catch (Exception e)
{ 
    err=e.getLocalizedMessage();
}

Saccharometer

How can I determine the realm programatically? This code is for a library that will run on many computers connecting to many servers. So I can't hard code it. – Hairsplitter

And how can I determine the 'opaque'? The service I'm trying to call seems to require that, and it changes with each new session. – Grover

Answer 3

private static byte[] downloadFileWithDigitAuth(String url, String username, String password) {
    byte[] bytes = null;
    CloseableHttpClient httpClient = HttpClientBuilder.create().build();
    HttpGet httpGet = new HttpGet(url);
    HttpContext httpContext = new BasicHttpContext();
    CloseableHttpResponse httpResponse = null;
    try {
        httpResponse = httpClient.execute(httpGet, httpContext);

        if (httpResponse.getStatusLine().getStatusCode() == HttpStatus.SC_UNAUTHORIZED) {
            Header authHeader = httpResponse.getFirstHeader(AUTH.WWW_AUTH);
            DigestScheme digestScheme = new DigestScheme();

            /*
            override values if need
            No need override values such as nonce, opaque, they are generated by server side
            */
            digestScheme.overrideParamter("realm", "User Login Required !!");
            digestScheme.processChallenge(authHeader);

            UsernamePasswordCredentials creds = new UsernamePasswordCredentials(username, password);
            httpGet.addHeader(digestScheme.authenticate(creds, httpGet, httpContext));

            httpResponse.close();
            httpResponse = httpClient.execute(httpGet);
        }
        bytes = IOUtils.toByteArray(httpResponse.getEntity().getContent());
    } catch (IOException | MalformedChallengeException | AuthenticationException e) {
        e.printStackTrace();
    }
    finally {
        if (httpResponse != null) {
            try {
                httpResponse.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
    return bytes;
}

Gradle :

compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.12'
compile group: 'commons-io', name: 'commons-io', version: '2.6'

Nesbitt

Answer 4

You guys make it so complicated. If you read the documentation of apache httpclient, it would be super easy.

protected static void downloadDigest(URL url, FileOutputStream fos)
    throws IOException {
    HttpHost targetHost = new HttpHost(url.getHost(), url.getPort(), url.getProtocol());
    CloseableHttpClient httpClient = HttpClients.createDefault();
    HttpClientContext context = HttpClientContext.create();

    String credential = url.getUserInfo();
    if (credential != null) {
        String user = credential.split(":")[0];
        String password = credential.split(":")[1];

        CredentialsProvider credsProvider = new BasicCredentialsProvider();
        credsProvider.setCredentials(AuthScope.ANY,
            new UsernamePasswordCredentials(user, password));
        AuthCache authCache = new BasicAuthCache();
        DigestScheme digestScheme = new DigestScheme();
        authCache.put(targetHost, digestScheme);

        context.setCredentialsProvider(credsProvider);
        context.setAuthCache(authCache);
    }

    HttpGet httpget = new HttpGet(url.getPath());

    CloseableHttpResponse response = httpClient.execute(targetHost, httpget, context);

    try {
        ReadableByteChannel rbc = Channels.newChannel(response.getEntity().getContent());
        fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
    } finally {
        response.close();
    }
}

Microphyte

Because DigestScheme needs to pull the WWW-Authenticate, you have to register it, rather than returning the cache. Cache is stating that it is pre-emptive and all the data is already there. – Aerospace