Getting new token on retry before retrying old request with Volley

Question

I have a simple authentication system implemented using Volley. It goes like this: Get a token from server on login -> an hour later, this token expires -> when it expires, we will find that out on a failed API call, so we should (on retry) -> fetch a new token when that call fails and then -> retry the original call.

I've implemented this, and the token is returning successfully, but because I think I'm doing something wrong with the Volley RequestQueue, the original request uses all it's retrys before the new and valid token is able to be used. Please see the following code:

public class GeneralAPICall extends Request<JSONObject> {
public static String LOG_TAG = GeneralAPICall.class.getSimpleName();

SessionManager sessionManager; //instance of sessionManager needed to get user's credentials
private Response.Listener<JSONObject> listener; //the response listener used to deliver the response
private Map<String, String> headers = new HashMap<>(); //the headers used to authenticate
private Map<String, String> params; //the params to pass with API call, can be null

public GeneralAPICall(int method, String url, Map<String, String> params, Context context, Response.Listener<JSONObject> responseListener, Response.ErrorListener errorListener) {
    super(method, url, errorListener);
    sessionManager = new SessionManager(context); //instantiate
    HashMap<String, String> credentials = sessionManager.getUserDetails(); //get the user's credentials for authentication
    this.listener = responseListener;
    this.params = params;
    //encode the user's username and token
    String loginEncoded = new String(Base64.encode((credentials.get(Constants.SessionManagerConstants.KEY_USERNAME)
            + Constants.APIConstants.Characters.CHAR_COLON
            + credentials.get(Constants.SessionManagerConstants.KEY_TOKEN)).getBytes(), Base64.NO_WRAP));
    Log.v(LOG_TAG, loginEncoded); //TODO: remove
    this.headers.put(Constants.APIConstants.BasicAuth.AUTHORIZATION, Constants.APIConstants.BasicAuth.BASIC + loginEncoded); //set the encoded information as the header
    setRetryPolicy(new TokenRetryPolicy(context)); //**THE RETRY POLICY**
}

The retry policy I set is defined as default, but I implement my own retry method as such:

@Override
public void retry(VolleyError error) throws VolleyError {
    Log.v(LOG_TAG, "Initiating a retry");
    mCurrentRetryCount++; //increment our retry count
    mCurrentTimeoutMs += (mCurrentTimeoutMs * mBackoffMultiplier);
    if (error instanceof AuthFailureError) { //we got a 401, and need a new token
        Log.v(LOG_TAG, "AuthFailureError found!");
        VolleyUser.refreshTokenTask(context, this); //**GET A NEW TOKEN**
    }
    if (!hasAttemptRemaining()) {
        Log.v(LOG_TAG, "No attempt remaining, ERROR");
        throw error;
    }
}

The refresh token task defines a RefreshAPICall

public static void refreshTokenTask(Context context, IRefreshTokenReturn listener) {
    Log.v(LOG_TAG, "refresh token task called");
    final IRefreshTokenReturn callBack = listener;

    RefreshAPICall request = new RefreshAPICall(Request.Method.GET, Constants.APIConstants.URL.GET_TOKEN_URL, context, new Response.Listener<JSONObject>() {

        @Override
        public void onResponse(JSONObject response) {
            try {
                String token = response.getString(Constants.APIConstants.Returns.RETURN_TOKEN);
                Log.v(LOG_TAG, "Token from return is: " + token);
                callBack.onTokenRefreshComplete(token);
            } catch (JSONException e) {
                callBack.onTokenRefreshComplete(null); //TODO: log this
                e.printStackTrace();
            }
        }
    }, new Response.ErrorListener() {
        @Override
        public void onErrorResponse(VolleyError error) {
            Log.v(LOG_TAG, "Error with RETRY : " + error.toString());
        }
    });

    VolleySingleton.getInstance(context).addToRequestQueue(request);
}

Our RefreshAPICall definition:

public RefreshAPICall(int method, String url, Context context, Response.Listener<JSONObject> responseListener, Response.ErrorListener errorListener) {
    super(method, url, errorListener);
    sessionManager = new SessionManager(context); //instantiate
    HashMap<String, String> credentials = sessionManager.getRefreshUserDetails(); //get the user's credentials for authentication
    this.listener = responseListener;
    //encode the user's username and token
    String loginEncoded = new String(Base64.encode((credentials.get(Constants.SessionManagerConstants.KEY_USERNAME)
            + Constants.APIConstants.Characters.CHAR_COLON
            + credentials.get(Constants.SessionManagerConstants.KEY_PASSWORD)).getBytes(), Base64.NO_WRAP));
    this.headers.put(Constants.APIConstants.BasicAuth.AUTHORIZATION, Constants.APIConstants.BasicAuth.BASIC + loginEncoded); //set the encoded information as the header
    setTag(Constants.VolleyConstants.RETRY_TAG); //mark the retry calls with a tag so we can delete any others once we get a new token
    setPriority(Priority.IMMEDIATE); //set priority as immediate because this needs to be done before anything else

    //debug lines
    Log.v(LOG_TAG, "RefreshAPICall made with " + credentials.get(Constants.SessionManagerConstants.KEY_USERNAME) + " " +
            credentials.get(Constants.SessionManagerConstants.KEY_PASSWORD));
    Log.v(LOG_TAG, "Priority set on refresh call is " + getPriority());
    Log.v(LOG_TAG, "Tag for Call is " + getTag());
}

I set the priority of this request as high so that it gets triggered before the one that failed, so once we get a token the original call can then fire with the valid token.

Finally, on response I delete any other tasks with the retry tag (in case multiple API calls failed and made multiple retry calls, we don't want to overwrite the new token multiple times)

@Override
public void onTokenRefreshComplete(String token) {
    VolleySingleton.getInstance(context).getRequestQueue().cancelAll(Constants.VolleyConstants.RETRY_TAG);
    Log.v(LOG_TAG, "Cancelled all retry calls");
    SessionManager sessionManager = new SessionManager(context);
    sessionManager.setStoredToken(token);
    Log.v(LOG_TAG, "Logged new token");
}

Unfortunately, the LogCat is showing me that all the retries are happening before we use the token. The token is coming back successfully, but it's obvious that the IMMEDIATE priority is having no effect on the order that the queue dispatches the calls.

Any help on how to ensure my RefreshAPICall is fired before the other tasks would be greatly appreciated. I'm wondering if Volley considers the RefreshAPICall as a subtask of the original failed task, and so it attempts to call that original task for its number of retrys until those are out, and then fires off the RefreshAPICall.

LogCat (not sure how to make this look pretty):

05-05 16:12:07.145: E/Volley(1972): [137] BasicNetwork.performRequest: 
Unexpected response code **401 for https://url.me/api/get_friends**
05-05 16:12:07.145: V/TokenRetryPolicy(1972): Initiating a retry
05-05 16:12:07.145: V/TokenRetryPolicy(1972): AuthFailureError found!
05-05 16:12:07.146: V/VolleyUser(1972): refresh token task called
05-05 16:12:07.146: V/RefreshAPICall(1972): RefreshAPICall made with username user_password

05-05 16:12:07.147: V/RefreshAPICall(1972): Priority set on refresh call is HIGH
05-05 16:12:07.147: V/RefreshAPICall(1972): Tag for Call is retry
05-05 16:12:07.265: E/Volley(1972): [137] BasicNetwork.performRequest: Unexpected response code **401 for https://url.me/api/get_friends**
05-05 16:12:07.265: V/TokenRetryPolicy(1972): Initiating a retry
05-05 16:12:07.265: V/TokenRetryPolicy(1972): AuthFailureError found!
05-05 16:12:07.265: V/VolleyUser(1972): refresh token task called
05-05 16:12:07.265: V/RefreshAPICall(1972): RefreshAPICall made with user user_password

05-05 16:12:07.265: V/RefreshAPICall(1972): Priority set on refresh call is HIGH
05-05 16:12:07.265: V/RefreshAPICall(1972): Tag for Call is retry
05-05 16:12:07.265: V/TokenRetryPolicy(1972): No attempt remaining, ERROR

05-05 16:12:08.219: I/Choreographer(1972): Skipped 324 frames!  The application may be doing too much work on its main thread.
05-05 16:12:08.230: V/RefreshAPICall(1972): Response from server on refresh is: {"status":"success","token":"d5792e18c0e1acb3ad507dbae854eb2cdc5962a2c1b610a6b77e3bc3033c7f64"}
05-05 16:12:08.230: V/VolleyUser(1972): Token from return is: d5792e18c0e1acb3ad507dbae854eb2cdc5962a2c1b610a6b77e3bc3033c7f64
05-05 16:12:08.231: V/TokenRetryPolicy(1972): Cancelled all retry calls
05-05 16:12:08.257: V/SessionManager(1972): New Token In SharedPref is: d5792e18c0e1acb3ad507dbae854eb2cdc5962a2c1b610a6b77e3bc3033c7f64
05-05 16:12:08.257: V/TokenRetryPolicy(1972): Logged new token

Hochstetler

Did you get any fix for this? Or any workarounds? I am stuck with something similar. – Flirt

@sushantkunal please see below, hope it helps! – Hochstetler

Answer 1

Posting an answer now that I found a half-decent way to handle token refreshing on retry.

When I create my general (most common) API call with Volley, I save a reference to the call in case it fails, and pass it to my retry policy.

public GeneralAPICall(int method, String url, Map<String, String> params, Context context, Response.Listener<JSONObject> responseListener, Response.ErrorListener errorListener) {
    super(method, url, errorListener);
    sessionManager = SessionManager.getmInstance(context);
    HashMap<String, String> credentials = sessionManager.getUserDetails(); // Get the user's credentials for authentication
    this.listener = responseListener;
    this.params = params;
    // Encode the user's username and token
    String loginEncoded = new String(Base64.encode((credentials.get(Constants.SessionManagerConstants.KEY_USERNAME)
            + Constants.APIConstants.Characters.CHAR_COLON
            + credentials.get(Constants.SessionManagerConstants.KEY_TOKEN)).getBytes(), Base64.NO_WRAP));
    this.headers.put(Constants.APIConstants.BasicAuth.AUTHORIZATION, Constants.APIConstants.BasicAuth.BASIC + loginEncoded); // Set the encoded information as the header

    setRetryPolicy(new TokenRetryPolicy(context, this)); //passing "this" saves the reference
}

Then, in my retry policy class (which simply extends the DefaultRetryPolicy, when I receive a 401 error telling me I need a new token, I shoot off a refreshToken call to get a new one.

public class TokenRetryPolicy extends DefaultRetryPolicy implements IRefreshTokenReturn{
...

@Override
public void retry(VolleyError error) throws VolleyError {
    mCurrentRetryCount++; //increment our retry count
    mCurrentTimeoutMs += (mCurrentTimeoutMs * mBackoffMultiplier);
    if (error instanceof AuthFailureError && sessionManager.isLoggedIn()) {
        mCurrentRetryCount = mMaxNumRetries + 1; // Don't retry anymore, it's pointless
        VolleyUser.refreshTokenTask(context, this); // Get new token
    } if (!hasAttemptRemaining()) {
        Log.v(LOG_TAG, "No attempt remaining, ERROR");
        throw error;
    }
}
...

}

Once that call returns, I handle the response in my retry policy class. I modify the call that failed, giving it the new token (after storing the token in SharedPrefs) to authenticate itself, and then fire it off again!

@Override
public void onTokenRefreshComplete(String token, String expiration) {
    sessionManager.setStoredToken(token, expiration);

    HashMap<String, String> credentials = sessionManager.getUserDetails(); //get the user's credentials for authentication

    //encode the user's username and token
    String loginEncoded = new String(Base64.encode((credentials.get(Constants.SessionManagerConstants.KEY_USERNAME)
            + Constants.APIConstants.Characters.CHAR_COLON
            + credentials.get(Constants.SessionManagerConstants.KEY_TOKEN)).getBytes(), Base64.NO_WRAP));
    Log.v(LOG_TAG, loginEncoded); //TODO: remove
    callThatFailed.setHeaders(Constants.APIConstants.BasicAuth.AUTHORIZATION, Constants.APIConstants.BasicAuth.BASIC + loginEncoded); //modify "old, failed" call - set the encoded information as the header

    VolleySingleton.getInstance(context).getRequestQueue().add(callThatFailed);
    Log.v(LOG_TAG, "fired off new call");
}

This implementation works great for me.

However, I should note that this situation shouldn't happen much because I learned that I should check if my token has expired before making any API call. This is possible by storing an expiration time (returned from the server) in SharedPrefs, and seeing if current_time - expiration time < some_time, with some_time being the amount of time you would like to get a new token prior to it expiring, for me 10 seconds.

Hope this helps somebody out there, and if I'm wrong about anything, please comment!

Hochstetler

could you share your code?, i have to implement this, your code is like i want to do. Thanks – Kevon

@MaxPinto, what code are you looking for? The above code gives the basics of how to implement it. – Hochstetler

well yes i understand that your coude give the basics, but specifically how you do this file: IRefreshTokenReturn interface – Kevon

The IRefreshTokenReturn interface is simply a one-method interface: public void onTokenRefreshComplete(String token, String expiration).. The implementation of it sits in the TokenRetryPolicy, and is the last chunk of code above. – Hochstetler

@user1621629 , SessionManager is just a class I built that handles everything related to the users session. It's a singleton class that stores token, retrieves token, returns the users information such as username, etc. It also handles SharedPrefs modifications and retrievals. Let me know if you need more info. – Hochstetler

@Hochstetler In case of multi-thread multiple volley request will be processed in parallel, so multiple refresh token call will be made with older expiry token. How this situation can be handled? – Lemonade

@Lemonade I have since moved away from Volley in favor of Retrofit as this was one of the issues I often ran into. While I used Volley, I ensured that only one call was in flight at a time, which is admittedly far worse than optimal. I never really feel like I got a good hold on Volley, so I'm sorry I can't better answer your question. – Hochstetler

Answer 2

The strategy I am using now is to add a refreshToken to the failed retry. This is a custom failure retry.

public class CustomRetryPolicy implements RetryPolicy
{
    private static final String TAG = "Refresh";
    private Request request;
    /**
     * The current timeout in milliseconds.
     */
    private int mCurrentTimeoutMs;

    /**
     * The current retry count.
     */
    private int mCurrentRetryCount;

    /**
     * The maximum number of attempts.
     */
    private final int mMaxNumRetries;

    /**
     * The backoff multiplier for the policy.
     */
    private final float mBackoffMultiplier;

    /**
     * The default socket timeout in milliseconds
     */
    public static final int DEFAULT_TIMEOUT_MS = 2500;

    /**
     * The default number of retries
     */
    public static final int DEFAULT_MAX_RETRIES = 1;

    /**
     * The default backoff multiplier
     */
    public static final float DEFAULT_BACKOFF_MULT = 1f;

    /**
     * Constructs a new retry policy using the default timeouts.
     */
    public CustomRetryPolicy() {
        this(DEFAULT_TIMEOUT_MS, DEFAULT_MAX_RETRIES, DEFAULT_BACKOFF_MULT);
    }

    /**
     * Constructs a new retry policy.
     *
     * @param initialTimeoutMs  The initial timeout for the policy.
     * @param maxNumRetries     The maximum number of retries.
     * @param backoffMultiplier Backoff multiplier for the policy.
     */
    public CustomRetryPolicy(int initialTimeoutMs, int maxNumRetries, float backoffMultiplier) {
        mCurrentTimeoutMs = initialTimeoutMs;
        mMaxNumRetries = maxNumRetries;
        mBackoffMultiplier = backoffMultiplier;
    }

    /**
     * Returns the current timeout.
     */
    @Override
    public int getCurrentTimeout() {
        return mCurrentTimeoutMs;
    }

    /**
     * Returns the current retry count.
     */
    @Override
    public int getCurrentRetryCount() {
        return mCurrentRetryCount;
    }

    /**
     * Returns the backoff multiplier for the policy.
     */
    public float getBackoffMultiplier() {
        return mBackoffMultiplier;
    }

    /**
     * Prepares for the next retry by applying a backoff to the timeout.
     *
     * @param error The error code of the last attempt.
     */
    @SuppressWarnings("unchecked")
    @Override
    public void retry(VolleyError error) throws VolleyError {
        mCurrentRetryCount++;
        mCurrentTimeoutMs += (mCurrentTimeoutMs * mBackoffMultiplier);
        if (!hasAttemptRemaining()) {
            throw error;
        }
        //401 and 403 
        if (error instanceof AuthFailureError) {//Just token invalid,refresh token
            AuthFailureError er = (AuthFailureError) error;
            if (er.networkResponse != null && er.networkResponse.statusCode == HttpURLConnection.HTTP_UNAUTHORIZED) {
                //Count is used to reset the flag
                RefreshTokenManager instance = RefreshTokenManager.getInstance();
                instance.increaseCount();
                CUtils.logD(TAG, "come retry count: " + instance.getCount());
                boolean ok = instance.refreshToken();
                if (ok) {
                    Map<String, String> headers = request.getHeaders();
                    String[] tokens = instance.getTokens();
                    headers.put("token", tokens[0]);
                    Log.d(TAG, "retry:success");
                } else {
                    throw error;
                }
            }
        }
    }

    /**
     * Returns true if this policy has attempts remaining, false otherwise.
     */
    protected boolean hasAttemptRemaining() {
        return mCurrentRetryCount <= mMaxNumRetries;
    }

    public Request getRequest() {
        return request;
    }

    public void setRequest(Request request) {
        this.request = request;
    }
}

RefreshToken

public class RefreshTokenManager {
private static final String TAG = "Refresh";
private static RefreshTokenManager instance;
private final RefreshFlag flag;
/**
 *retry count
 */
private AtomicInteger count = new AtomicInteger();

public int getCount() {
    return count.get();
}

public  int increaseCount() {
    return count.getAndIncrement();
}

public void resetCount() {
    this.count.set(0);
}

/**
 * 锁
 */
private Lock lock;

public static RefreshTokenManager getInstance() {
    synchronized (RefreshTokenManager.class) {
        if (instance == null) {
            synchronized (RefreshTokenManager.class) {
                instance = new RefreshTokenManager();
            }
        }
    }
    return instance;
}

private RefreshTokenManager() {
    flag = new RefreshFlag();
    lock = new ReentrantLock();
}

public void resetFlag() {
    lock.lock();
    RefreshFlag flag = getFlag();
    flag.resetFlag();
    lock.unlock();
}

protected boolean refreshToken() {
   lock.lock();
    RefreshFlag flag = getFlag();
    //Reset the flag so that the next time the token fails, it can enter normally.
    if (flag.isFailure()) {
        if (count.decrementAndGet() == 0) {
            resetFlag();
        }
        lock.unlock();
        return false;
    } else if (flag.isSuccess()) {
        CUtils.logD(TAG, "decrease retry count: " + instance.getCount());
        if (count.decrementAndGet() == 0) {
            count.incrementAndGet();
            flag.resetFlag();
        } else {
            lock.unlock();
            return true;
        }
    }
    // refreshToken is doing.
    flag.setDoing();
    //Upload refresh_token and get the response from the server
    String response = postRefreshTokenRequest();
    CUtils.logD(TAG, "refreshToken: response " + response);
    if (!TextUtils.isEmpty(response)) {
        try {
            JSONObject jsonObject = new JSONObject(response);
            JSONObject data = jsonObject.optJSONObject("data");
            if (data != null) {
                String token = data.optString("token");
                String refreshToken = data.optString("refresh_token");
                CUtils.logD(TAG, "refreshToken: token : " + token + "\n" + "refresh_token : " + refreshToken);
                if (!TextUtils.isEmpty(token) && !TextUtils.isEmpty(refreshToken)) {
                    //success,save token and refresh_token
                    saveTokens(token, refreshToken);
                    CUtils.logD(TAG, "run: success  notify ");
                    flag.setSuccess();
                    if (count.decrementAndGet() == 0) {
                        resetFlag();
                    }
                    CUtils.logD(TAG, "decrease retry count: " + instance.getCount());
                    lock.unlock();
                    return true;
                }
            }
        } catch (Exception e) {
            CUtils.logE(e);
        }
    }
    //delete local token and refresh_token
    removeTokens();
    flag.setFailure();
    count.decrementAndGet();
    CUtils.logD(TAG, "decrease retry count: " + instance.getCount());
    lock.unlock();
    CUtils.logD(TAG, "run: fail  notify ");
    return false;

}

private RefreshFlag getFlag() {
    return flag;
}

}

This is the flag

public final class RefreshFlag {
private static final int FLAG_SUCCESS = 0x01;
private static final int FLAG_DOING = 0x11;
private static final int FLAG_FAILURE = 0x10;
private static final int FLAG_INIT = 0x00;
/**
 * flag 标志位
 */
private int flag = FLAG_INIT;

public boolean isDoingLocked() {
    return flag == FLAG_DOING;
}

public void setDoing() {
    flag = FLAG_DOING;
}

public void setSuccess() {
    flag = FLAG_SUCCESS;
}

public void setFailure() {
    flag = FLAG_FAILURE;
}

public boolean isSuccess() {
    return flag == FLAG_SUCCESS;
}

public boolean isFailure() {
    return flag == FLAG_FAILURE;
}

public void resetFlag() {
    flag = FLAG_INIT;
}
}

Beabeach

Answer 3

I know this post this old, but posting my solution after other solutions suggested didn't help me.

Note - I did try Brandon's method given above, i.e., extending DefaultRetryPolicy. But it's fields are private, so didn't want to implement the whole class, there had to be a better way.

So I write the code in the CustomRequest class extending Request. Here are relevant snippets -

Store tokens in login response -

@Override
protected Response<T> parseNetworkResponse(NetworkResponse response) {
    ...
    //if oauth data is sent with response, store in SharedPrefs
    ...
}

If access token has expired -

@Override
protected VolleyError parseNetworkError(VolleyError volleyError) {
...
if (volleyError instanceof NoConnectionError) {
        //i know, there has to be a better way than checking this. 
        //will work on it later
        if(volleyError.getMessage().equalsIgnoreCase("java.io.IOException: No authentication challenges found")) {

            String accessToken = getNewAccessToken();//synchronous call

            //retry
            if(accessToken != null) {
                //IMP: this is the statement which will retry the request manually
                NetworkHelper.get(mContext).getRequestQueue().add(this);
            }
        }
    }
    ...
 }

Attach access token to request -

@Override
public Map<String, String> getHeaders() throws AuthFailureError {
    ...
    String accesssToken = //get from SharedPrefs
    headers.put("Authorization", "Bearer " +accessToken);
    ...
}

Going to login screen if refresh token is invalid -

private void showLogin(){
    //stop all current requests
    //cancelAllRequests();

    Intent intent = new Intent(mContext, LoginActivity.class);
    intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK | Intent.FLAG_ACTIVITY_CLEAR_TOP);
    mContext.startActivity(intent);
}

Getting new access token using refresh token. This has to be a synchronous method using RequestFuture -

private String getNewAccessToken(){
    ...
    //get new access token from server and store in SharedPrefs
    ...
    //also return the new token so that we know if we need to retry or not
    return newAccessToken;
}

HTH

Thenar

I'll be honest, shortly after battling through Volley a bit I went right to Retrofit. It, paired with OkHttp is way better than Volley and you can use an Authenticator to do all the retries and everything. It's a way more seamless experience than having something like this. – Hochstetler

OkHttp was definitely better for me for handling multipart files. I was wary about using Authenticators as the OAuth code on server is a custom one. I'll try Retrofit next time. – Thenar