MediaCodec Encoded video has green bar at bottom and chrominance screwed up

Question

I started a project from Grafika and modified it. I've got a framework (not too different from the original) which captures a preview from the Camera and continuously encodes it to a video at the same time both at different resolutions.

MediaCodec (for encoding) is configured to use COLOR_FormatSurface in order for us to be able to render to the Surface queried using getInputSurface() using GLES.

The media format is set to MIME Type video/avc

For most phones, this setup works perfectly.

But for a few phones, the encoded video has the chrominance values slightly skewed and a green bar at the bottom.

Currently the encoder surface is of 640x360 resolution. EGLConfig choosen to render to the surface supports 32 bit RGBA format.

The preview is perfect in all phones.

So I am assuming there's some thing wrong with the encoder parameters either on the application side or the OS framework side.

It's happening on Android 4.4. - Not sure, if it's reproducible on 5.*.

]2

Ostler

That sure looks like a color format problem, which shouldn't be happening when encoding from a Surface. Are you able to try other resolutions? I'm curious if the codec on these devices has a bug at that resolution. Do you know if all affected devices have the same video encoder chip? (You can usually tell by looking at logcat -- there's usually some hardware-specific babbling.) – Indifference

Usually, the preview resolution and the encoding resolution are different. Is 640x360 the encoding resolution also? Are you encoding 1080p? If so, then I think your issue is due to alignment vertically. The encoder may be reading 1088 instead of 1080. If so, the camera output should be aligned similarly. – Inextirpable

The preview is 720p. It doesn't matter. Because I'm rendering it as a texture (Using OpenGLES) to the encoder surface. @Indifference has already pointed out the issue. Looks like the codec issue. I'm now using a Surface with height as the next multiple of 16. – Ostler

@Ganesh: Sorry, I think I didn't explain the question correctly. Camera preview: 720p, Device Preview resolution: Device resolution, Encoder resolution: 360p, The thing is youtube livestreaming API supports only these resolutions: Livestreaming supported resolutions Should I just use height 368 instead of 360? The problem with that is the aspect ratio is spoiled and youtube will add a bit of blanking on either side. – Ostler

Were you able to solve this issue? – Manlove

I've added an answer, because of word limit in comments. Check it out – Ostler

@fadden: Are you aware of any fix/workaround for this. I want to encode at 1080, but I am getting same issue on a MediaTek chipset device. I have tried encoding at 1072, and that seems fine. But I don't want to go with this solution, for several reasons - 1) Non standard resolution, 2) How to decide which device encodes at 1080 and which of them require 1072, 3) Other issues may crop up (exporting at 1088 fixed this issue, but added other issues) Thanks a lot in advance! – Manlove

You're not the only one having MediaTek issues (e.g. stackoverflow.com/questions/34891420). Have you checked the MediaCodecInfo.CodecProfileLevel to confirm that the device advertises support for 1080p? Many devices have a "secret" multiple-of-16 requirement but correctly handle 1080 anyway. – Indifference

Answer 1

I'll answer that for myself just in case anyone else find it useful.

Apparently the issue only occurs on a few select chipsets such as MediaTek ones, which don't support non multiple of 16 resolutions. In my case it was 640x360 (where height is a multiple of 8). To partially solve it, I added a check in the App code, if the encoder is Omx.mtk, then reconfigure the encoder with width/height as the next multiple of 16, which comes out to be 640x368 in my case (closest to the requirement i.e. 360p).

For me, YouTube player adds a slight blanking on either side, which is the best we can do with those encoders.

In case you don't have dependency on any third party APIs/players etc

Ostler