errors when decode H.264 frames using ffmpeg

Question

I am getting the following errors when decoding H.264 frames received from the remote end of a H.264 based SIP video call. Appreciate any help in understanding there errors.

non-existing PPS 0 referenced
decode_slice_header error
non-existing PPS 0 referenced
decode_slice_header error
no frame!

non-existing PPS 0 referenced
decode_slice_header error
non-existing PPS 0 referenced
decode_slice_header error
no frame!

Lidia

Answer 1

That just means that ffmpeg has not seen a keyframe yet, which carries SPS and PPS information. SPS and PPS are crucial in decoding an incoming frame/slice. Keyframes are sent periodically (i.e. every 5-10 seconds or more); so if it turns out that you joined a stream before the keyframe arrived; you will see this warning for every frame until a keyframe shows up.

As soon as the keyframe shows up from the wire, ffmpeg will have enough information to decode that frame (and any subsequent frames until the next keyframe), so those warnings will go away.

Hickson

That seems to be the problem. The remote end is not sending a keyframe even when my client requests for it using a SIP INFO. Any ideas how this can be solved? I am using an old version of Bria at the remote end. – Lidia

The remote side is not sending any keyframes? That is very odd. Not sure if Bria has an option to configure the keyframe interval, but it would be worth looking into. So in general, you are not seeing any video from the remote side at all, or it takes a while to see it? – Hickson

Your are most likely having general decoding issues. The errors that you are seeing, do they ever stop? Can we see some of your ffmpeg decoding code? – Hickson

Or you could be having rendering issues of the decoded frames(i.e. mismatched colorspaces in between the decoder and renderer). Anyhow, there is a lot of moving parts here. If you could specify the platform, the way you are rendering video; that would also give me a better idea on what's happening. – Hickson

Try to set key frame interval to few seconds on remote coder. If it set in automatic mode and camera is static you can receive frames through very long periods of time – Stew

@Hickson See this. If missing of keyframe is the issue, how can i solve it? – According

A Raspberry Pi hardware encoded stream with keyframes set to every second will still generate this error for every frame, and never sync up to a keyframe. What else can generate this error? – Shirtmaker

Answer 2

You need to add frames sps and pps information. ffmpeg needs these frames to decode. You can find these values in the SDP file.

In the SDP file, you should look for NAL units, you can see something like that:

z0IAHukCwS1xIADbugAzf5GdyGQl, aM4xUg

these values, base64 encoded, should be converted to binary. I am using wireshark and wireshark converts these values automatically for you. After that you have sps and pps values.

Now you have to add these NAL blocks before the data frame:

00 00 00 01 sps 00 00 00 01 pps 00 00 00 01 data 

for h264 these block is what I have been using to decode.

Aspectual

Answer 3

To decode a frame or a slice, sliceHeader is decoded, which refers to a PPS or "Picture Parameter Set". It has information regarding the specifics of the frame like width, height etc.

I guess your data is coming through a streaming input channel, in which case SPS and PPS would have been sent earlier in the stream.

You may have to concatenate the same to your stream.

Internuncio