CVPixelBuffer to CIImage always returning nil

Question

I am trying to convert a pixelBuffer extracted from AVPlayerItemVideoOutput to CIImage but always getting nil.

The Code

if([videoOutput_ hasNewPixelBufferForItemTime:player_.internalPlayer.currentItem.currentTime])
{
    CVPixelBufferRef pixelBuffer = [videoOutput_ copyPixelBufferForItemTime:player_.internalPlayer.currentItem.currentTime
                                                         itemTimeForDisplay:nil];

    CIImage *image = [CIImage imageWithCVPixelBuffer:pixelBuffer]; // Always image === nil
    CIFilter *filter = [FilterCollection filterSepiaForImage:image];
    image = filter.outputImage;

    CIContext *context = [CIContext contextWithOptions:nil];
    CGImageRef cgimg = [context createCGImage:image fromRect:[image extent]];
    [pipLayer_ setContents:(id)CFBridgingRelease(cgimg)];

} 

Below are details of a pixelBuffer used in order to create a CIImage (which always results in nil):

$0 = 0x09b48720 <CVPixelBuffer 0x9b48720 width=624 height=352 bytesPerRow=2496 pixelFormat=BGRA iosurface=0x0 attributes=<CFBasicHash 0x98241d0 [0x1d244d8]>{type = immutable dict, count = 3,
entries =>
    0 : <CFString 0x174cf4 [0x1d244d8]>{contents = "Height"} = <CFNumber 0x9a16e70 [0x1d244d8]>{value = +352, type = kCFNumberSInt32Type}
    1 : <CFString 0x174ce4 [0x1d244d8]>{contents = "Width"} = <CFNumber 0x9a109d0 [0x1d244d8]>{value = +624, type = kCFNumberSInt32Type}
    2 : <CFString 0x1750e4 [0x1d244d8]>{contents = "PixelFormatType"} = <CFArray 0x1090ddd0 [0x1d244d8]>{type = mutable-small, count = 1, values = (
    0 : <CFNumber 0x7a28050 [0x1d244d8]>{value = [phone], type = kCFNumberSInt32Type}
)}
}
 propagatedAttachments=<CFBasicHash 0x9b485c0 [0x1d244d8]>{type = mutable dict, count = 6,
entries =>
    0 : <CFString 0x174ff4 [0x1d244d8]>{contents = "CVImageBufferTransferFunction"} = <CFString 0x174f84 [0x1d244d8]>{contents = "ITU_R_709_2"}
    2 : <CFString 0x174f74 [0x1d244d8]>{contents = "CVImageBufferYCbCrMatrix"} = <CFString 0x174f94 [0x1d244d8]>{contents = "ITU_R_601_4"}
    9 : <CFString 0x174f14 [0x1d244d8]>{contents = "CVPixelAspectRatio"} = <CFBasicHash 0x9b1bc30 [0x1d244d8]>{type = immutable dict, count = 2,
entries =>
    0 : <CFString 0x174f34 [0x1d244d8]>{contents = "VerticalSpacing"} = <CFNumber 0x9b0f730 [0x1d244d8]>{value = +1, type = kCFNumberSInt32Type}
    2 : <CFString 0x174f24 [0x1d244d8]>{contents = "HorizontalSpacing"} = <CFNumber 0x9b0f730 [0x1d244d8]>{value = +1, type = kCFNumberSInt32Type}
}

    10 : <CFString 0x174fb4 [0x1d244d8]>{contents = "CVImageBufferColorPrimaries"} = <CFString 0x174fd4 [0x1d244d8]>{contents = "SMPTE_C"}
    11 : <CFString 0x174e24 [0x1d244d8]>{contents = "QTMovieTime"} = <CFBasicHash 0x7a47940 [0x1d244d8]>{type = immutable dict, count = 2,
entries =>
    0 : <CFString 0x174e44 [0x1d244d8]>{contents = "TimeScale"} = <CFNumber 0x7a443d0 [0x1d244d8]>{value = +90000, type = kCFNumberSInt32Type}
    2 : <CFString 0x174e34 [0x1d244d8]>{contents = "TimeValue"} = <CFNumber 0x7a476e0 [0x1d244d8]>{value = +1047297, type = kCFNumberSInt64Type}
}

    12 : <CFString 0x174eb4 [0x1d244d8]>{contents = "CVFieldCount"} = <CFNumber 0x9b0f730 [0x1d244d8]>{value = +1, type = kCFNumberSInt32Type}
}
 nonPropagatedAttachments=<CFBasicHash 0x9b44b40 [0x1d244d8]>{type = mutable dict, count = 0,
entries =>
}
>

Retool

Answer 1

Solved this problem - I was trying on simulator, while it seems that is is only supported on devices.

Retool

OMG... is that documented somewhere? – Agential

This "solved" my problem as well. In Swift, my let inputImage = CIImage(CVPixelBuffer: pixelBuffer) always returns nil on the simulator and an object on the device. :( – Hunley

I am getting nil even on device. Can any one tell the reason? – Handsome

@shaun, I am checking on device. still it get nil. can you please provide any solution? - Punita – Alexi

Answer 2

It seems that you do not have iosurface properties key defined. Not 100% it will solve your issue, but try this:

CFDictionaryRef attrs = (CFDictionaryRef)@{
    (id)kCVPixelBufferWidthKey: @(WIDTH_OF_YOUR_VIDEO_GOES_HERE),
    (id)kCVPixelBufferHeightKey: @(HEIGHT_OF_YOUR_VIDEO_GOES_HERE),
    (id)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_420YpCbCr8BiPlanarFullRange;), // kCVPixelFormatType_32BGRA
    (id)kCVPixelBufferIOSurfacePropertiesKey: @{},
}

And pass this dictionary along with other keys during creation of your pixelbuffer

Hands

In macos v10.10，must set kCVPixelBufferPixelFormatTypeKey＝kCVPixelFormatType_32ARGB，then +[CIImage imageWithCVPixelBuffer:] could be success – Bursarial

Answer 3

That's not the way you set the content of a layer; this is:

(__bridge id)uiImage.CGImage;

There's no way you got what you had working on a device or a simulator.

Hindman