VideoToolbox can't decode MFT encoded H264 properly

Question

Did anyone experience an issue where VideoToolbox can't decode Media Foundation Transform (MFT) encoded H264 properly? The decoded frame has green block distortion more than half of the frame. I tried Livy Stork's example for decoding MFT encoded H264, which is transmitted over network from PC video endpoint running on Windows 7 desktop. I'm using Xcode 6.3 and running it on iOS 8 devices.

Clue: Same algorithm works fine when MFT encoded H264 comes from PC video endpoint running on Windows 8 or 8.1 desktop/tablet. According to MSDN, most codec properties aren't supported in Windows 7, but I can't figure it out which property makes the difference...

Supersonic

Answer 1

But if you replace all the start code of all the NALUs, it should work. I have successfully solved a problem that using Video Toolbox to decode a data array with multiple I slices or multiple p slice when they start with 4 bytes start code then following 8 slice with 3 bytes start code. Just replace every start code with slice length respectively. Note that slice length doesn't include start code length, you should ignore them.

Fungistat

Answer 2

It turned out MFT H264 encoder on Windows 7 returns a frame with multiple NALs. VideoToolbox decoder requires the NAL start code (0x00 00 00 01) to be replaced with the NAL size; when it finds the second NAL start code in the frame, it stops decoding. Again MFT H264 encoder on Windows 8 or higher doesn't do this.

Supersonic