In Android, how to pass a predefined Surface to MediaCodec for encoding?

Question

I have an app that manages its own GLSurfaceView and now I want to use Android 4.3's new MediaCodec feature that takes a Surface as input.

In all the examples I've seen, the Surface is created using MediaCodec.createInputSurface(), then the GL context is created for this Surface. This feels monolithic and incredibly disruptive to retrofit into a code base that is already stable.

Is it possible to use MediaCodec.configure(format, a_predefined_Surface, null, MediaCodec.CONFIGURE_FLAG_ENCODE) instead? This allows me to use MediaCodec in a plug-and-play and on-demand way. The fact that MediaCodec.configure() takes a Surface parameter indicates that this should be possible. However, the API states that 'Specify a surface on which to render the output of this decoder' http://developer.android.com/reference/android/media/MediaCodec.html#configure(android.media.MediaFormat, android.view.Surface, android.media.MediaCrypto, int) does this mean this is only meant for decode and not encode?. If so, is there any way to make MediaCodec use a predefined Surface for encoding?

The Surface I'm passing in is already created with EGL_RECORDABLE_ANDROID set to true and the returned GL context is verified to contain the required EGL_RECORDABLE_ANDROID attribute. Despite this, MediaCodec.configure() fails with an unhelpful exception 'native_window_api_connect returned an error: Invalid argument (-22)':

I/ACodec(32383):  Now uninitialized
I/OMXClient(32383): Using client-side OMX mux.
I/ACodec(32383): [OMX.qcom.video.encoder.avc] Now Loaded
E/MediaCodec(32383): native_window_api_connect returned an error: Invalid argument (-22)
W/System.err(32383): java.lang.IllegalStateException
W/System.err(32383):    at android.media.MediaCodec.native_configure(Native Method)
W/System.err(32383):    at android.media.MediaCodec.configure(MediaCodec.java:259)
[...]
W/System.err(32383):    at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1520)
W/System.err(32383):    at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1248)

This is from a Samsung Galaxy S4 with Android 4.3.

Landaulet

Answer 1

No, it doesn't work that way. The input surface must be created with createInputSurface().

Bear in mind that a "surface" isn't a buffer of data, it's a queue of buffers for which the producer and consumer endpoints are often in different processes. There are a lot of moving pieces that need to be set up. Also note that Surface and EGLSurface are two different things that, while often used together, aren't closely related.

The API seems lumpy and weird because... it is. The implementation of Surface has changed quite a bit over the years -- the underpinnings used to be far less general, so most of the APIs for altering endpoints aren't exposed. The (underspecified) MediaCodec API is still evolving.

There's an example of presenting the same content (camera preview) to GLSurfaceView and MediaCodec in Grafika. It sounds like you're trying to do something similar. (If not, update your question, and I'll update the answer.)

Xylem

can I record video putting bitmap in Surface of MediaCodec like with MediaRecorder? see it #51332886 – Semidome

Answer 2

It seems that in API 23+ there is an API called setInputSurface(Surface):

Reneta

Note that it takes a "persistent input surface", created with the MediaCodec createPersistentInputSurface() call. So the API is more flexible, but you still have to use a Surface that was created by MediaCodec. – Xylem