Why is the HTTPBody of a request inside an NSURLProtocol Subclass always nil?

Question

I have my NSURLProtocol MyGreatProtocol. I add it to the URL Loading system,

NSURLProtocol.registerClass(MyGreatProtocol)

I then start receiving events inside MyGreatProtocol during network sessions.

Suppose I create the session after registering my protocol,

let session = NSURLSession.sharedSession()
    let request = NSMutableURLRequest(URL: NSURL(string:"http://example.com")!, cachePolicy: .ReloadIgnoringLocalCacheData, timeoutInterval: 2.0) //arbitrary 404
    request.HTTPBody = ([1, 3, 3, 7] as [UInt8]).toNSData
    print("body data: \(request.HTTPBody)") //prints `Optional(<01030307>)`
    request.HTTPMethod = "POST"
    session.dataTaskWithRequest(request) { (data, response, err) in
        print("data: \(data)\nresponse: \(response)\nerror\(err)")
    }.resume()

I expect the request's HTTP body 1/3/3/7 to be present inside MyGreatProtocol, where it is not.

Inside MyGreatProtocol, I override the following methods.

override class func canonicalRequestForRequest(request: NSURLRequest) -> NSURLRequest {
    print(request.HTTPBody) //prints nil
    return request
}

override class func canInitWithRequest(request: NSURLRequest) -> Bool {
    print(request.HTTPBody) //prints nil
    return true
}

override func startLoading() {
    print(self.request.HTTPBody) //prints nil
}

The other properties of the NSURLRequest seem to be retained. The URL, HTTP verb, headers, etc, are all there. Something specific about the nature of the body remains elusive.

Why is the HTTP Body Nil inside a custom NSURLProtocol?

There seems to be some similar discussion on radars previously filed (i.e. https://bugs.webkit.org/show_bug.cgi?id=137299)

Prospect

Answer 1

Here's a sample code for reading the httpBody:

Swift 5

extension URLRequest {

    func bodySteamAsJSON() -> Any? {

        guard let bodyStream = self.httpBodyStream else { return nil }

        bodyStream.open()

        // Will read 16 chars per iteration. Can use bigger buffer if needed
        let bufferSize: Int = 16

        let buffer = UnsafeMutablePointer<UInt8>.allocate(capacity: bufferSize)

        var dat = Data()

        while bodyStream.hasBytesAvailable {

            let readDat = bodyStream.read(buffer, maxLength: bufferSize)
            dat.append(buffer, count: readDat)
        }

        buffer.deallocate()

        bodyStream.close()

        do {
            return try JSONSerialization.jsonObject(with: dat, options: JSONSerialization.ReadingOptions.allowFragments)
        } catch {

            print(error.localizedDescription)

            return nil
        }
    }
}

Then, in URLProtocol:

override func startLoading() {

    ...

    if let jsonBody = self.request.bodySteamAsJSON() {

        print("JSON \(jsonBody)")

    }

    ...
}

Smokechaser

Some request, from amplitude for example, has nil httpBodyStream eventhough it should have body request – Discovert

Answer 2

IIRC, body data objects get transparently converted into streaming-style bodies by the URL loading system before they reach you. If you need to read the data:

• Open the HTTPBodyStream object
• Read the body data from it

There is one caveat: the stream may not be rewindable, so don't pass that request object on to any other code that would need to access the body afterwards. Unfortunately, there is no mechanism for requesting a new body stream, either (see the README file from the CustomHTTPProtocol sample code project on Apple's website for other limitations).

Mephitis

HTTPBodyStream also has no data in it – Cia

Then you've found a bug. That should not be possible. – Mephitis

I copied the code from this question into canInitWithRequest: and got a -1 return value for inputStream.read #25477270 – Cia

Did you open the stream first? – Mephitis

Brilliant answer! I didn't realise I needed to open the stream before reading from it. Thanks a lot :) – Jahdol

Please upvote or accept this answer. I'd like to be able to dup another question to this one, and can't because the answer is neither accepted nor upvoted. :-) Thanks. – Mephitis

Can any help me with the code for opening httpBodyStream? – Mchale

First, set a delegate on the stream, then call open on the stream, and return (to let the run loop run). When your delegate's stream:handleEvent: method is called with NSStreamEventHasBytesAvailable, read an arbitrary amount of data from the stream and accumulate it somewhere. The read call may return fewer bytes than you asked for, and you may seen this event more than once. When your delegate's stream:handleEvent: method gets called with NSStreamEventEndEncountered, process the accumulated data. Note that NSStream does not retain its delegate; you must retain it yourself. – Mephitis

Is there any way to rewind the stream? I want to read the HTTPBodyStream in multiple NSURLProtocol subclasses, but it seems that only the first one can read the stream. – Glycogen

There's no guarantee that the stream can be rewound. The usual approach is to create a bound pair of streams, and change the request's stream to the read end of that pair when you send it back out, and then write the data into the write end of the pair as you read it from the original stream. Either that or write it to disk as you read it, and then replace it with a file stream. Either way. – Mephitis