Decode video using CoreMedia.framework on iOS

Question

I need to decode mp4 file and draw it using OpenGL in ios app. I need to extract and decode h264 frames from mp4 file and I heard what it posible to do using CoreMedia. Anybody has any idea how to do it? Any examples of CoreMedia using?

Biochemistry

Answer 1

It's not Core Media you're looking for, it's AVFoundation. In particular, you'd use an AVAssetReader to load from your movie and iterate through the frames. You then can upload these frames as OpenGL ES textures either by using glTexImage2D() or (on iOS 5.0) by using the much faster texture caches.

If you don't want to roll your own implementation of this, I have working AVFoundation-based movie loading and processing via OpenGL ES within my GPUImage framework. The GPUImageMovie class encapsulates movie reading and the process of uploading to a texture. If you want to extract that texture for use in your own scene, you can chain a GPUImageTextureOutput to it. Examples of both of these classes can be found in the SimpleVideoFileFilter and CubeExample sample applications within the framework distribution.

You can use this directly, or just look at the code I wrote to perform these same actions within the GPUImageMovie class.

Wampumpeag

I tried to user AVAssetReader and it works fine when I try to play one long file, but I need to play sequence of small video files downloaded from server. It's something my own implementation of HTTP Live streaming. I tried to play one file create new reader and play next file, but AVAsset/AVAssetReader initialization require some time and I have delay between segments. I thought about appending new mp4 files to first, but I can't find right way to do it. Do you have any idea how to do it? Or how to optimize AVAsset/AVAssetReader start? – Biochemistry

Also I use your library GPUImage for some video effect. Thanks for good job, man! – Biochemistry

Sorry, haven't played around with multiple movie inputs like you describe. I have observed that much of AVFoundation seems geared around playing or recording single media files, not multiple ones. – Wampumpeag

Take a look at AVComposition. It's a subclass of AVAsset that has support for multiple sequenced files. – Flagitious