Digest authentication in Android using HttpURLConnection

Question

as the question allready says, I am trying to do digest authentication in android.
Until now i have used the DefaultHttpClient and it's authentication method (using UsernamePasswordCredentials and so on), but it is deprecated since Android 5 and will be removed in Android 6.
So i am about to switch from DefaultHttpClient to HttpUrlConnection.
Now i am trying to achieve digest authentication, which should work pretty simple as explained here:

Authenticator.setDefault(new Authenticator() {
    protected PasswordAuthentication getPasswordAuthentication() {
        return new PasswordAuthentication(username, password);
    }
});

But the getPasswordAuthentication gets never called for some reason.
During my search for this problem i found different posts, saying digest authentication is not supported by the HttpUrlConnection in android, but those posts are from 2010-2012, so i am not sure if this is still true. Also we are using HttpUrlConnection with digest authentication in our desktop java application, where it does work.

I also found some posts, talking about OkHttp. OkHttp seems to be used by Android under the hood (to be more specific the HttpUrlConnectionImpl). But this HttpUrlConnectionImpl is a bit strange, it is not even shown in the Eclipse type hierarchy and i am not able to debug it. Also it should be a com.squareup.okhttp.internal.huc.HttpUrlConnectionImpl, while in android it is a com.android.okhttp.internal.http.HttpUrlConnectionImpl.

So i am just not able to do digest authentication with this HttpUrlConnection in android.
Can anyone tell me how to do that without external libraries?

EDIT:
The server asks for digest authentication:

WWW-Authenticate: Digest realm="Realm Name",domain="/domain",nonce="nonce",algorithm=MD5,qop="auth"

So Basic-Authentication shouldn' work, as the server is asking for digest.

Amenable

Answer 1

The answer is, that HttpUrlConnection does not support digest.

You therefore have to implement RFC2617 by yourself.

You can use the following code as a baseline implementation: HTTP Digest Auth for Android.

The steps involve (see RFC2617 for reference):

• If you get a 401 response, iterate over all WWW-Authenticate headers and parse them:
  • Check if algorithm is MD5 or undefined, (optionally select the auth qop option), otherwise ignore the challenge and go to the next header.
  • Get the credentials using Authenticator.requestPasswordAuthentication.
  • Calculate H(A1) using the username, realm and password.
  • Store the canonical root URL, realm, HA1, username, nonce (+ optionally algorithm, opaque and the client selected qop option if present).
  • Retry the request.
• On each request, iterate over all realms you have session information stored for by canonical root URL:
  • Calculate H(A2) using the request method and path.
  • Calculate H(A3) using HA1, nonce (+ optionally nc, cnonce, qop) and HA2.
  • Build and add the Authorization header to your HttpUrlConnection.
• Implement some sort of session pruning.

By using Authenticator, you can make sure, that as soon as HttpUrlConnection supports digest natively, your code is not being used anymore (because you wont receive the 401 in the first place).

This is just a quick summary on how to implement it, for you to get an idea.

If you want to go further you would probably like to implement SHA256 as well: RFC7616

Sherlocke

Thanks for that answer. Is it only the Androids version of HttpUrlConnection, which does not support digest or also the default java.net.HttpURLConnection? – Amenable

@Springrbua Only on Android its not supported, at least to my knowledge. Maybe you will find sources for the JDK implementation. – Sherlocke

Okay thanks for your answer. I'll stick with the DefaultHttpClient for now, but it seems like i need to switch sooner or later so i'll have to implement digest myself. Thanks! – Amenable

Answer 2

It is correct that HttpUrlConnection does not support Digest authentication. If your client must authenticate using Digest, you have a few options:

• Write your own HTTP Digest implementation. This can be a good option if you know which servers that you need to authenticate with and can ignore the parts of the the digest specification that you do not need. Here is an example where a subset of digest is implemented: https://gist.github.com/slightfoot/5624590.
• Use the external lib bare-bones-digest, which is a Digest lib for Android. You can use it to parse Digest challenges and generate responses to them. It supports the common digest use cases and some of the rarely used ones and can be used on top of HttpURLConnection.
• Use OkHttp together with okhttp-digest, which is a plugin that adds Http Digest support to OkHttp. Supporting Digest with OkHttp is easy, just add okhttp-digest as an authenticator and you will have transparent Http digest support. If you already use OkHttp or are OK with switching to it this can be an attractive option.
• Use the Apache HttpClient which supports Digest. The question explicitly states that HttpClient is not an option so I include it mostly for completion's sake. Google does not recommend using HttpClient and has deprecated it.

Lyckman

I replaced the DefaultHttpClient with the HttpUrlConnection a few months ago and I implemented Digest authentication mayself, using (this)[gist.github.com/slightfoot/5624590] as a template. I might add my code as an answer to, it might help someone else! – Amenable

Answer 3

Did you try to set the header manually like:

String basic = "Basic " + new String(Base64.encode("username:password".getBytes(),Base64.NO_WRAP ));
connection.setRequestProperty ("Authorization", basic);

Also be aware of some issues in Jellybeans and a bug when you try to perform a post request: HTTP Basic Authentication issue on Android Jelly Bean 4.1 using HttpURLConnection

EDIT: For Digest authentication

Have a look here https://code.google.com/p/android/issues/detail?id=9579

Especially this might work:

try {   
        HttpClient client = new HttpClient(
                new MultiThreadedHttpConnectionManager());

        client.getParams().setAuthenticationPreemptive(true);
        Credentials credentials = new UsernamePasswordCredentials("username", "password");
        client.getState().setCredentials(AuthScope.ANY, credentials);
        List<String> authPrefs = new ArrayList<String>(2);
        authPrefs.add(AuthPolicy.DIGEST);
        authPrefs.add(AuthPolicy.BASIC);
        client.getParams().setParameter(AuthPolicy.AUTH_SCHEME_PRIORITY,
                authPrefs);
        GetMethod getMethod = new GetMethod("your_url");
        getMethod.setRequestHeader("Accept", "application/xml");
        client.executeMethod(getMethod);
        int status = getMethod.getStatusCode();
        getMethod.setDoAuthentication(true);
        System.out.println("status: " + status);
        if (status == HttpStatus.SC_OK) {
            String responseBody = getMethod.getResponseBodyAsString();
            String resp = responseBody.replaceAll("\n", " ");
            System.out.println("RESPONSE \n" + resp);
        }
    } catch (Exception e) {
        e.printStackTrace();
    }

Prestonprestress

I edit my question and added the WWW-Authenticate header, given by the server. It says digest, so i gguess it expect and requires digest authentication... – Amenable

Ceck, changed my answer, hope it helps. There seem to be a few bugs in the Authenticator on Android – Prestonprestress

Also check #2954934 – Prestonprestress

Thats acutally what i am using now (the apache HttpClient) and it works, but it is deprecated in Android 5 and seems to be removed in Android 6, so i want to replace it with the HttpUrlConnection as suggested... – Amenable

Did you also check the other answers on that question, there are quite a few ways to try. – Prestonprestress

All 3 answers to the linked question seem to be more or less the same and use the DigestScheme, which is part of org.apache.hppt. This is actually the package which is deprecated and will be removed. So i can't use any class form the org.apache.http.* packages, at least if i want to support android 6... – Amenable

It is not a nice solution but perhaps you can make an if else for android < 6 and android 6+. It seems that the new way has bugs in older android versions which do not make your life easy. – Prestonprestress

No i did not test it with android 6 yet, but i am working with an android 5.1.1 Tablet, target API is also Android 5.1.1 (Api Level 22), where it is allready deprecated. So also there it has to work with the HttpUrlConnection somehow. I mean you can't deprecate something if there is no alternative for right? Do you have any source where they say it works in android 6? – Amenable

I have read a lot now, but they all say (even Android staff) that Digest is not supported in the current implementation and will be fixed in a future release.. whatever that means. I fear your best bet is to stick with the deprecated code for now and switch once you can test Android 6. – Prestonprestress

Okay, i guess i'll stick with the HttpClient for now. I have a Nexus 9 Tablet, so i should get Android 6 soon. Then I'll test it and i really hope they fixed that bug then. If not, i guess, i have to import the apache-library... Thanks a lot! +1 for your help :) – Amenable

Answer uses HttpClient, and hence does not answer the question. – Sherlocke

@Sherlocke also read the comments please... I already said that Digest authentication is not supported as yet by httpurlconnection. So yes you are right, it is not nice to use httpclient but for now the only way to go. – Prestonprestress

The question is how to make digest work without the recently deprecated org.apache.http package, and the answer does not address this at all. The first part is about basic auth, and the second part is about HttpClient which is already known to support digest. So there is no point in providing this information as an "answer". – Sherlocke

Answer 4

I finally replaced the deprecated DefaultHttpClient with my own implementation of the HttpUrlConnection and I implemented digest atuhentication myself, using this as a template.
The finaly code looks something like this:

// requestMethod: "GET", "POST", "PUT" etc.
// Headers: A map with the HTTP-Headers for the request
// Data: Body-Data for Post/Put
int statusCode = this.requestImpl(requestMethod, headers, data);
if (statusCode == HttpURLConnection.HTTP_UNAUTHORIZED && hasUserNameAndPassword) {
    String auth = getResponseHeaderField("WWW-Authenticate");
    // Server needs Digest authetication
    if(auth.startsWith("Digest")){
          // Parse the auth Header
          HashMap<String, String> authFields = parseWWWAuthenticateHeader(auth);
          // Generate Auth-Value for request
          String requestAuth = generateDigestAuth(authFields);
          headers.put("Authorization", authStr);
          statusCode = this.requestImpl(requestMethod, headers, data);
    }
}

So basicly I make a request and if it returns 401, I look, if the server wants digest authentication and if I have username and password. If thats the case, I parse the auth header of the response, which contains all the necessary informations about the authentication.
To parse the auth header I use some kind of StateMachine which is described here.
After parsing the response auth header, I generate the request auth header using the informations from the response:

    String digestAuthStr = null;

    String uri = getURL().getPath();
    String nonce = authFields.get("nonce");
    String realm = authFields.get("realm");
    String qop = authFields.get("qop");
    String algorithm = authFields.get("algorithm");
    String cnonce = generateCNonce();
    String nc = "1";
    String ha1 = toMD5DigestString(concatWithSeparator(":", username, realm, password));
    String ha2 = toMD5DigestString(concatWithSeparator(":", requestMethod, uri));
    String response = null;
    if (!TextUtils.isEmpty(ha1) && !TextUtils.isEmpty(ha2))
        response = toMD5DigestString(concatWithSeparator(":", ha1, nonce, nc, cnonce, qop, ha2));

    if (response != null) {
        StringBuilder sb = new StringBuilder(128);
        sb.append("Digest ");
        sb.append("username").append("=\"").append(username).append("\", ");
        sb.append("realm").append("=\"").append(realm).append("\", ");
        sb.append("nonce").append("=\"").append(nonce).append("\", ");
        sb.append("uri").append("=\"").append(uri).append("\", ");
        sb.append("qop").append("=\"").append(qop).append("\", ");
        sb.append("nc").append("=\"").append(nc).append("\", ");
        sb.append("cnonce").append("=\"").append(cnonce).append("\"");
        sb.append("response").append("=\"").append(response).append("\"");
        sb.append("algorithm").append("=\"").append(algorithm).append("\"");
        digestAuthStr = sb.toString();
    }

To generate the Client-Nonce I am using the following code:

private static String generateCNonce() {
    String s = "";
    for (int i = 0; i < 8; i++)
        s += Integer.toHexString(new Random().nextInt(16));
    return s;
}

I hope this helps someone. If the code contains any errors, please let me know so I can fix it. But right now it seems to work.

Amenable

This is a very compact and efficient digest implementation, but it is good to be aware of that it only implements a small subset of digest. This is fine if you, for instance, only work with one particular server and can test with that. If you need to be compatible with many different servers and handle things like multiple challenges, escaped characters in quoted strings, SHA-256 digest, the opaque directive, reusing challenges, etc, then you are probably better of to use one of the libraries (bare-bones-digest, okhtt-digest, Apache HttpClient, etc) to cover the corner cases. – Lyckman

@Lyckman I totaly agree with you. If you need full support of digest you should definitely go with a known and therefore tested library. – Amenable

Answer 5

For Android, I found the bare-bones-digest library worked well: https://github.com/al-broco/bare-bones-digest

1. Add one line to build.gradle
2. Use the example code at the above url

Works!

Vasiliki