Difficulty Writing Pixel Buffer To Asset Writer on Certain Devices

Question

I am working on a function in my app to write images from my sample buffer to an AVAssetWriter. Curiously, this works fine on a 10.5" iPad Pro, but causes a crash on a 7.9" iPad Mini 2. I can't fathom how the same code could be problematic on two different devices. But here's my code;

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

    // Setup the pixel buffer image
    let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!

    // Setup the format description
    let formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer)!

    // Setup the current video dimensions
    self.currentVideoDimensions = CMVideoFormatDescriptionGetDimensions(formatDescription)

    // Setup the current sample time
    self.currentSampleTime = CMSampleBufferGetOutputPresentationTimeStamp(sampleBuffer)

    // Handle record
    if self.isCapturing {

        // Setup auto release pool
        autoreleasepool {

            // Setup the output image
            let outputImage = CIImage(cvPixelBuffer: pixelBuffer)

            // Ensure the video writer is ready for more data
            if self.videoWriter?.assetWriterPixelBufferInput?.assetWriterInput.isReadyForMoreMediaData == true {

                // Setup the new pixel buffer (THIS IS WHERE THE ERROR OCCURS)
                var newPixelBuffer: CVPixelBuffer? = nil

                // Setup the pixel buffer pool
                CVPixelBufferPoolCreatePixelBuffer(nil, (self.videoWriter?.assetWriterPixelBufferInput!.pixelBufferPool!)!, &newPixelBuffer)

                // Render the image to context
                self.context.render(outputImage, to: newPixelBuffer!, bounds: outputImage.extent, colorSpace: nil)

                // Setup a success case
                let success = self.videoWriter?.assetWriterPixelBufferInput?.append(newPixelBuffer!, withPresentationTime: self.currentSampleTime!)

                // Ensure the success case exists
                guard let mySuccess = success else { return }

                // If unsuccessful, log
                if !mySuccess {
                    print("Error with the sample buffer.  Check for dropped frames.")
                }
            }
        }
    }
}

I receive an error that newPixelBuffer is nil, but again, only on a 7.9" iPad. The iPad Pro functions without any errors. Any thoughts? Thanks!

Panchromatic

What is the return value of CVPixelBufferPoolCreatePixelBuffer's call ? – Quietly

All I'm receiving is Thread 3: Unexpectedly found nil while unwrapping an Optional value when running this on the problematic device. – Panchromatic

Gonna have to ask the same question Valérian asked. What does CFPixelBufferPoolCreatePixelBuffer return? According to the documentation, that function returns a CVReturn containing an error value which would presumably explain why it's failing. What is it? – Ism

Not sure if I'm doing this right, but I set var status:CVReturn = CVPixelBufferPoolCreatePixelBuffer(nil, (self.videoWriter?.assetWriterPixelBufferInput!.pixelBufferPool!)!, &newPixelBuffer). When crashing, the return is 65552448. Let me know if that may be incorrect. – Panchromatic

Answer 1

I eventually resolved this issue by tracing the problem back to my chosen codec in my Asset Writer's video output settings. I had my codec set to:

let codec: AVVideoCodecType = AVVideoCodecType.hevc

In doing some research, I found this article, which indicates that only certain devices can capture media in HEVC. As my first device was a 10.5" iPad Pro, it captured media with no problem. My second device was an iPad Mini, which resulted in the original problem occurring each time I tried to capture.

I have since changed my codec choice to:

let codec: AVVideoCodecType = AVVideoCodecType.h264, and the issue has now disappeared.

Panchromatic