FairPlay Streaming: Calling copyPixelBufferForItemTime on AVPlayerItemVideoOutput returns NULL

Question

Has anybody had experience using HLS with Fairplay and succeeded in retrieving the pixel buffer?

I'm using an AVURLAsset with its resourceLoader delegate set. My AVAssetResourceLoaderDelegate takes care of dealing with the Fairplay process.

It displays fine on an AVPlayerLayer, however, when I try to use an AVPlayerItemVideoOutput that is attached to the AVPlayerItem and use copyPixelBufferForItemTime on it, the pixelBuffer returned is always NULL.

On the other hand, when I use a non-encrypted stream and not use the resourceLoader copyPixelBufferForItemTime returns a pixelbuffer as expected.

Is this by working as intended? (i.e. Streams that are protected using FairPlay don't expose their pixel buffer to prevent misuse.)

And if not, is there any way to get the pixel buffer from a stream that is protected using FairPlay on iOS?

Cutaway

Answer 1

After researching into this issue further, I have come to the conclusion that Apple engineered their AVPlayer in such a way that once you use FairPlay protected HLS the only exit point (see A:) for the decrypted data copyPixelBufferForItemTime always returns nil.

• I have tested the exact same stream without encryption and copyPixelBufferForItemTime returns a reference to the pixel buffer as expected.
• I have tested the exact same stream with encryption on an AVPlayerLayer and it displays the video as excepted.

It appears that once you use FairPlay, the only way to display your protected video content is by using an AVPlayerLayer. There appears to be no way as of today to retrieve FairPlay protected HLS media from Apple's APIs in order to display it on an OpenGL texture in 3D space for example.

A: copyPixelBufferForItemTime being the only exit point since calling renderInContext on an AVPlayerLayer doesn't work

Cutaway

Answer 2

i met the same problem when using an AVURLAsset with its resourceLoader delegate set. Different from your situation, there are two kind of live streaming decrypted content, and one of them can retrieve the pixel buffer another can't. I am not sure if it is related to the content of the m3u8 file.

The content that can retrieve the pixel buffer, and get thumbnail.

#EXTM3U
#EXT-X-VERSION:2
#EXT-X-KEY:METHOD=AES-128,URI="http://xxx.xxxx.net/key/xxx",IV=0xxxx
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:11601809
#EXTINF:10, no desc
#EXT-X-PROGRAM-DATE-TIME:2019-05-28T11:00:10Z
stream_index_20190331T232744_1_11601809.ts
#EXTINF:10, no desc
#EXT-X-PROGRAM-DATE-TIME:2019-05-28T11:00:20Z
stream_index_20190331T232744_1_11601810.ts

The content that can't retrieve the pixel buffer

#EXTM3U
#EXT-X-VERSION:8
#EXT-X-INDEPENDENT-SEGMENTS

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audiod",NAME="en",DEFAULT=YES,AUTOSELECT=YES,LANGUAGE="eng",URI="Audio3_2.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=3000000,AVERAGE-BANDWIDTH=3000000,VIDEO-RANGE=PQ,CODECS="dvh1.05.06",RESOLUTION=1920x1080,FRAME-RATE=50.000,AUDIO="audiod",CLOSED-CAPTIONS=NONE,HDCP-LEVEL=TYPE-1
Video1_1.m3u8

Jolynjolynn