Access StageFright.so directly to decode H.264 stream from JNIlayer in Android

Question

Is there a way to access libstagefright.so directly to decode H.264 stream from JNI layer on Android 2.3 or above?

Onagraceous

Answer 1

If your objective is to decode an elementary H.264 stream, then your code will have to ensure that the stream is extracted, the codec-specific-data is provided to the codec which is primarily SPS and PPS data and frame data along with time-stamps is provided to the codec. Across all Android versions, the most common interface would be OMXCodec which is an abstraction over an underlying OMX component.

In Gingerbread (Android 2.3) and ICS (Android 4.0.0), if you would like to create a decoder, the best method would be to create an OMXCodec component and abstract your code through a MediaSource interface i.e. your wrapper code is modeled as MediaSource and OMXCodec reads from this source and performs the decoding.

Link to Android 2.3 Video decoder creation: http://androidxref.com/2.3.6/xref/frameworks/base/media/libstagefright/AwesomePlayer.cpp#1094

Link to Android 4.0.0 Video decoder creation: http://androidxref.com/4.0.4/xref/frameworks/base/media/libstagefright/AwesomePlayer.cpp#1474

The main challenges would be the following:

1. Model the input as a MediaSource.

2. Read a wrapper code to read the buffer from the codec and handle the same and release it back to the codec.

For simplification, you could look stagefright command line executable code as in http://androidxref.com/4.0.4/xref/frameworks/base/cmds/stagefright/stagefright.cpp#233

However, if your program is based on JellyBean (Android 4.1.x, 4.2.x) onwards, then these are slightly more simplified. From your JNI code, you could create a MediaCodec component and employ the same for decoding. To integrate the same into your program, you could refer to the SimplePlayer implementation as in http://androidxref.com/4.2.2_r1/xref/frameworks/av/cmds/stagefright/SimplePlayer.cpp#316

Gatling

I used libstagefright hardware decode video on Android, main imitate AwesomePlayer.cpp, invoke OMXCodec,it's ok on Android 4.0, but on Android 4.1, it's black screen, Have you some suguest? – Chuckhole

@CrossleSong In Android 4.1 i.e. JellyBean the display pipeline underwent some significant improvements. Can you please share some logs for further analysis? I would recommend logging from SurfaceTexture, HwComposer and Layer modules. One side note: Is your layer marked as secure for some reason? – Gatling

Thanks your answer.I has resolved this problem, because ANativeWindow problem. lack native_window_set_scaling_mode(mNativeWindow.get(),mVideoScalingMode); – Chuckhole