Get RAW Bayer pixel data from CVPixelBufferRef

Question

I am using AVCaptureSession & AVCapturePhotoOutput to capture RAW photo data from device's camera in the kCVPixelFormatType_14Bayer_RGGB format.

I have got as far as getting the raw photo sample buffer in the AVCapturePhotoCaptureDelegate callback:

func capture(captureOutput: AVCapturePhotoOutput,
             didFinishProcessingRawPhotoSampleBuffer rawSampleBuffer: CMSampleBuffer?,
             previewPhotoSampleBuffer: CMSampleBuffer?,
             resolvedSettings: AVCaptureResolvedPhotoSettings,
             bracketSettings: AVCaptureBracketedStillImageSettings?,
             error: Error?) {

  guard let rawSampleBuffer = rawSampleBuffer else { return }

  guard let pixelBuffer = CMSampleBufferGetImageBuffer(rawSampleBuffer) else { return }
}

I am now attempting to follow the answers to this question to obtain pixel values from the CVPixelBufferRef but I cannot work out how to do this when using the 14 bit Bayer RGGB pixel format as opposed to the 32 bit RGB format mentioned in the answers.

Hughie

Did you find an answer to this? – Bremer

I did not get any answer – Hughie

Answer 1

First you need to find out the "Pixel format type" of the buffer. This is done with the CVPixelBufferGetPixelFormatType function.

The available types are listed here. In your case I'm guessing kCVPixelFormatType_14Bayer_BGGR.

Then you need to know how the data is stored. According to What is a CVPixelBuffer in iOS? it is

"Bayer 14-bit Little-Endian, packed in 16-bits, ordered R G R G... alternating with G B G B..."

Then you extract the pixels for Uint16 the same way they did for Uint8 in Get pixel value from CVPixelBufferRef in Swift.

Settera