How to use NSURLSession to determine if resource has changed?

Question

I'm using NSURLSession to request a JSON resource from an HTTP server. The server uses Cache-Control to limit the time the resource is cached on clients.

This works great, but I'd also like to cache a deserialized JSON object in memory as it is accessed quite often, while continuing to leverage the HTTP caching mechanisms built into NSURLSession.

I'm thinking I can save a few HTTP response headers: Content-MD5, Etag, and Last-Modified along with the deserialized JSON object (I'm using those 3 fields since I've noticed not all HTTP servers return Content-MD5, otherwise that'd be sufficient by itself). The next time I receive a response for the JSON object, if those 3 fields are the same then I can reuse the previously deserialized JSON object.

Is this a robust way to determine the deserizlied JSON is still valid. If not, how do I determine if the deserialized object is up to date?

Hyperspace

Answer 1

I created a HTTPEntityFingerprint structure which stores some of the entity headers: Content-MD5, Etag, and Last-Modified.

import Foundation

struct HTTPEntityFingerprint {
    let contentMD5 : String?
    let etag : String?
    let lastModified : String?
}

extension HTTPEntityFingerprint {
    init?(response : NSURLResponse) {
        if let httpResponse = response as? NSHTTPURLResponse {
            let h = httpResponse.allHeaderFields
            contentMD5 = h["Content-MD5"] as? String
            etag = h["Etag"] as? String
            lastModified = h["Last-Modified"] as? String

            if contentMD5 == nil && etag == nil && lastModified == nil {
                return nil
            }
        } else {
            return nil
        }
    }

    static func match(first : HTTPEntityFingerprint?, second : HTTPEntityFingerprint?) -> Bool {
        if let a = first, b = second {
            if let md5A = a.contentMD5, md5B = b.contentMD5 {
                return md5A == md5B
            }
            if let etagA = a.etag, etagB = b.etag {
                return etagA == etagB
            }
            if let lastA = a.lastModified, lastB = b.lastModified {
                return lastA == lastB
            }
        }

        return false
    }
}

When I get an NSHTTPURLResponse from an NSURLSession, I create an HTTPEntityFingerprint from it and compare it against a previously stored fingerprint using HTTPEntityFingerprint.match. If the fingerprints match, then the HTTP resource hasn't changed and thus I do not need to deserialized the JSON response again; however, if the fingerprints do not match, then I deserialize the JSON response and save the new fingerprint.

This mechanism only works if your server returns at least one of the 3 entity headers: Content-MD5, Etag, or Last-Modified.

More Details on NSURLSession and NSURLCache Behavior

The caching provided by NSURLSession via NSURLCache is transparent, meaning when you request a previously cached resource NSURLSession will call the completion handlers/delegates as if a 200 response occurred.

If the cached response has expired then NSURLSession will send a new request to the origin server, but will include the If-Modified-Since and If-None-Match headers using the Last-Modified and Etag entity headers in the cached (though expired) result; this behavior is built in, you don't have to do anything besides enable caching. If the origin server returns a 304 (Not Modified), then NSURLSession will transform this to a 200 response the application (making it look like you fetched a new copy of the resource, even though it was still served from the cache).

Hyperspace

What type of NSURLSession are you referring too: default, ephemeral, or background? – Staceestacey

It seems that you can only enable caching for default session – Mendy

Answer 2

This could be done with simple HTTP standard response.

Assume previous response is something like below:

{ status code: 200, headers {
    "Accept-Ranges" = bytes;
    Connection = "Keep-Alive";
    "Content-Length" = 47616;
    Date = "Thu, 23 Jul 2015 10:47:56 GMT";
    "Keep-Alive" = "timeout=5, max=100";
    "Last-Modified" = "Tue, 07 Jul 2015 11:28:46 GMT";
    Server = Apache;
} }

Now use below to tell server not to send date if it is not modified since.

NSURLSession is a configurable container, you would probably need to use http option "IF-Modified-Since"

Use below configuration kind before downloading the resource,

    NSURLSessionConfiguration *backgroundConfigurationObject = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:@"myBackgroundSessionIdentifier"];
    [backgroundConfigurationObject setHTTPAdditionalHeaders:
     @{@"If-Modified-Since": @"Tue, 07 Jul 2015 11:28:46 GMT"}];

if resource for example doesn't change from above set date then below delegate will be called

    - (void)URLSession:(NSURLSession *)session downloadTask:(NSURLSessionDownloadTask *)downloadTask didFinishDownloadingToURL:(NSURL *)location
    {

    NSHTTPURLResponse *httpResponse = (NSHTTPURLResponse *) downloadTask.response;
        if([httpResponse statusCode] == 304)
                //resource is not modified since last download date
    }

Check the downloadTask.response status code is 304 .. then resource is not modified and the resource is not downloaded.

Note save the previous success full download date in some NSUserDefaults to set it in if-modifed-since

Soemba

I tried this out with an NSURLSessionDataTask (not download task like your example) but, to my surprise, the 304 not modified is never returned. I tried using the completion handler and also a NSURLSessionDataDelegate. In both cases, I'd always get a 200 returned. I even ran tcpdump to verify the server was returning 304 and indeed it was, but something in NSURLSession (perhaps the part that accesses NSURLCache) transforms it into a 200 by the time the completion handler/delegate methods are called. – Hyperspace

using NSURLSessionDataTask is no problem, i guess the issue here might me the GMT date that could have been passed. Can you check the complete response.. the date which should be used must be from last modified of successful response of code 200 { status code: 200, headers { "Accept-Ranges" = bytes; Connection = "Keep-Alive"; "Content-Length" = 47616; Date = "Thu, 23 Jul 2015 10:47:56 GMT"; "Keep-Alive" = "timeout=5, max=100"; "Last-Modified" = "Tue, 07 Jul 2015 11:28:46 GMT"; Server = Apache; } } – Soemba

The date hasn't passed. I know this because I've verified using tcpdump that the server does indeed return a 304 response code. But even when the server returns the 304 without a body, something in NSURLSession transforms it into a 200 response with the body. I'm guessing this is a feature of NSURLSession so that developers don't have to take into account 304 responses which get served from the NSURLCache. – Hyperspace

I should also mention that you don't have to set the HTTP cache validation headers like Etag and/or If-Modified-Since. NSURLSession does this for you (perhaps only if you have caching configured like I do). – Hyperspace

i can assume the behaviour of NSURLSession is an expected one, as ios kit engine itself does this job on our behalf so it is good to set to 200. As you are accessing a resource data, why don't we use NSURLSessionDownloadTask which is designed for such case. – Soemba

i can assume the behaviour of NSURLSession is an expected one, as ios kit engine itself does this job on our behalf so it is good to set to 200. As you are accessing a resource data, why don't we use NSURLSessionDownloadTask which is designed for such case and use http response header withit. Using a http with if-modified-since is common design followed in any language/brower kit design. This design wouldn't break any time and would hold good if in feature we need to do the same for many resources. – Soemba