What is plane in a CVPixelbuffer?

Question

In CVPixelBuffer object, have one or many planes. (reference) We have methods to get number, heigh, the base address of plane.

So what exactly the plane is? And how it constructed inside a CVPixelBuffer?

Sample:

<CVPixelBuffer 0x1465f8b30 width=1280 height=720 pixelFormat=420v iosurface=0x14a000008 planes=2>
    <Plane 0 width=1280 height=720 bytesPerRow=1280>
    <Plane 1 width=640 height=360 bytesPerRow=1280>

Elah

Answer 1

Video formats are an incredibly complex subject.

Some video streams have the pixels stored in bytes RGBA, ARGB, ABGR, or several other variants (with or without an alpha channel)

(In RGBA format, you'd have the red, green, blue, and alpha values of a pixel one right after each other in memory, followed by another set of 4 bytes with the color values of the next pixel, etc.) This is interlaced color information.

Some video streams separate out the color channels so all the red channel, blue, green, and alpha are sent as separate "planes". You'd get a buffer with all the red information, then all the blue data, then all the green, and then alpha, if alpha is included. (Think of color negatives, where there are separate layers of emulsion to capture the different colors. The layers of emulsion are planes of color information. It's the same idea with digital.)

There are formats where the color data is in one or 2 planes, and then the luminance is in a separate plane. That's how old analog color TV works. It started out as black and white (luminance) and then broadcasters added side-band signals to convey the color information. (Chroma)

I don't muck around with CVPixelBuffers often enough to know the gory details of what you are asking, and have to invest large amounts of time and copious amounts of coffee before I can "spin up" my brain enough to grasp those gory details.

Edit:

Since your debug information shows 2 planes, it seems likely that this pixel buffer has a luminance channel and a chroma channel, as mentioned in @zeh's answer.

Naamana

Answer 2

Although the existing and accepted answer is rich of important information when dealing with CVPixelBuffers, in this particular case the answer is wrong. The two planes that the question refers to are the luminance and chrominance planes

Luminance refers to brightness and chrominance refers to color - From Quora

The following code snippet from Apple makes it more clear:

let lumaBaseAddress = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0)
let lumaWidth = CVPixelBufferGetWidthOfPlane(pixelBuffer, 0)
let lumaHeight = CVPixelBufferGetHeightOfPlane(pixelBuffer, 0)
let lumaRowBytes = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 0)

var sourceLumaBuffer = vImage_Buffer(data: lumaBaseAddress,
                                     height: vImagePixelCount(lumaHeight),
                                     width: vImagePixelCount(lumaWidth),
                                     rowBytes: lumaRowBytes)

let chromaBaseAddress = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 1)
let chromaWidth = CVPixelBufferGetWidthOfPlane(pixelBuffer, 1)
let chromaHeight = CVPixelBufferGetHeightOfPlane(pixelBuffer, 1)
let chromaRowBytes = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 1)

var sourceChromaBuffer = vImage_Buffer(data: chromaBaseAddress,
                                       height: vImagePixelCount(chromaHeight),
                                       width: vImagePixelCount(chromaWidth),
                                       rowBytes: chromaRowBytes)

See full reference here.

Millwater

I covered luminance and chroma as one of the possibilities for the format of a CVPixelBuffer. Based on the logged data in the OP's question (it shows 2 planes) it looks like you might be right. Is that what you are using to determine that that is the format of the OPs pixel buffer data? – Naamana