How to create a stagefright plugin

Question

I have a task which involves integration of a video decoder into Stagefright(Android's multimedia framework). I searched and found the following about creating a new plugin for Stagefright:

To add support for a new format, you need to:

• Develop a new Extractor class, if the container is not supported yet.

• Develop a new Decoder class, that implements the interface needed by the StageFright core to read the data.

• Associate the mime-type of the files to read to your new Decoder in the OMXCodec.cpp file, in the kDecoderInfo array.

    static const CodecInfo kDecoderInfo[] = {  
        {MEDIA_MIMETYPE_AUDIO_AAC, "OMX.TI.AAC.decode"},
        {MEDIA_MIMETYPE_AUDIO_AAC, "AACDecoder"},
    };

The above data is all i could find out on net. Right now i have a simple app that will take a file as an input and render it on the screen using native API's in android. Can anyone please tell me how to proceed further. And from where does all these OMXCodec.cpp and others come into picture and which directory of my project should i have them in. Please provide solutions regarding the same. Thanks in advance.

Impeller

Answer 1

From your question, it appears that you are looking at a recommendation which is specific for Ice-Cream Sandwich or earlier versions of Android. The first thing you should be clear about is the version of the android i.e. Ice-Cream Sandwich or before or JellyBean and after. The integration of codecs is different across different releases of Android.

I have already commented on your other question which is specific for JellyBean and later (Reference: Android: How to integrate a decoder to multimedia framework)

If you would like to integrate your codec in Ice-Cream Sandwich or before, the steps are already available in your question. In addition to adding the decoder into kDecoderInfo list, you may like to setup certain quirks as shown here.

For the question on OMXCodec.cpp, you can find this file at frameworks/base/media/libstagefright/ in case of Ice-Cream Sandwich and frameworks/av/media/libstagefright/ in case of JellyBean.

If you have followed all the steps to integrate the video decoder into the Stagefright framework, then the easiest test would be to perform the following:

1. Copy a media file into SD-Card

2. In OMXCodec.cpp, enable logs by removing the comment in this statement //#define LOG_NDEBUG 0 and run a mm in the directory. Copy the rebuilt libstagefright.so to /system/lib on your device.

3. Enable logcat and start capturing logs.

4. Goto gallery, select your file and allow the standard player to play your file.

5. Check your log file if the player has selected your OMX component by searching for your component name. If found, your integration of codec into Stagefright is successful. Else, you will have to debug and find out what is the problem.

Postscript:

1. Based on your queries, I presume you aren't familiar with Android sources. Please refer to androidxref site to become familiar with AOSP distributions.

2. Unless you are planning to support a new media file-format, you will not require to support Extractor class. MediaExtractor abstracts a file-format parser and helps to de-multiplex the different tracks in a media file.

I hope with this information, you should be able to get your codec integrated and functional in Android.

Zaidazailer

Thanks a lot... With this info i'm sure i can proceed. If possible can you please provide a specific link where i can know from the basics about android sources. In the above given link androidxref.com i'm not able to find the basics for how to start using android sources in my project. I would be really greatful to you if you could provide a source for how to use android sources in any application. Also a small query: are we modifying the android source code here or just creating a standalone plugin for the existing source code.? – Impeller

@Impeller If you are looking for a Java level introduction, you could start here: developer.android.com/training/index.html. However, if you are looking at integration notes, then you could read more in stackoverflow archives. For integrating the codec in Ice-Cream Sandwich you will have to modify the Android Sources. For integration into JellyBean, you are updating a platform specific file and not the sources, which is the expected mode of integration – Zaidazailer

Finally i understood that i need to rebuild a part of the android source code and create a libstagefright.so using an Android.mk and replace it with the existing one in /system/libs in my device. Now inorder to create this *.so file do i need to download the entire android source code or i just need to download frameworks/base/media/libstagefright/ contents? – Impeller

@Impeller I think you may require more folders under frameworks/ directory. It would be best if you can sync an entire android forest and keep the same ready. – Zaidazailer

You seem well versed in Android media pipeline, Ganesh. I would like to ask you more questions outside Stack Overflow, related to this topic. Are you interested in some consulting work on this topic? Please let me know at [email protected] if so. – Grime

@Zaidazailer : I was checking your posts on stagefright and really informational. Any possible way in stage fright to access all native video buffers? I was checking for your contact but couldn't find any where. – Roomy

@Ayyappa..You can access the AndroidNativeBuffers from CPU by locking the same. When the buffers are created, you should indicate that you plan read/write from CPU through GRALLOC_USAGE_SW_WRITE_OFTEN or GRALLOC_USAGE_SW_READ_OFTEN flags as part of the usage flags. With this change, you should be able to lock and access the buffers through the vptr returned from lock. – Zaidazailer