Will all phones support YUV 420 (Semi) Planar color format in h.264 encoder?

Question

Preambule: This may sound like a very specific question, but this is actually a go / no go to build an API 16+ Android application using MediaCodec that is compatible with most phone.

I have an application with a h.264 MediaCodec that receives data from a buffer - and not a surface since I'm doing a lot of manipulations on the image. When creating the Encoder, I iterate through the list of possible encoders from the phone to make sure I'm using a proprietary encoder if any. This part is not a problem.

The problem is that each encoder has its color format preference. This may lead to color format conversion before encoding. In my proof-of-concept, for example, I included method to convert to NV12, NV21 and YV12, along with following very strict rules, like placing the start of some planes / interleaved plane at a precise offset in the buffer, etc. Just being able to make an encoded video look good may be a long story.

So, if I was certain that there is a standard in what Android h.264 MediaCodec Encoders are accepting, then that would limit a lot the amount of customization I have to do to get my API 16 MediaCodec proof-of-concept working on all devices.

[EDIT]

Oh. I just saw that it's impossible to create an input surface before Android 18. I will have to rely on detecting proprietary codecs and color formats for each case, random crashes, slow FPS, etc. Bwe... With a bit of chance, in 2017 or 2018, there will be enough devices with relevant API features to write a decent app using MediaCodec.

Lindo

I assume you've found this source.android.com/compatibility/downloads.html which is where you can find the older Android Compatibility Definition Document. Also developer.android.com/reference/android/hardware/… may answer your question. – Enormous

Really interesting, I didn't know from where these CTS did come. Though I'm actually searching for a confirmation that the majority of phones running a MediaCodec h.264 encoder on API 16+ are able to receive either YUV 420 planar or YUV 420 semi planar frames. – Lunitidal

Answer 1

The really short answer is: No.

From Android 4.3 there is a CTS test that verifies that devices support either 420 planar or 420 semiplanar input - and, most importantly, that it interprets it in the same way as the others.

In practice, most devices from Android 4.1 do support either planar or semiplanar in one form or another, but there's a lot of quirks you need to work around, which are specific to different devices and chipsets. Some encoders (a bunch of Samsung Exynos devices) advertise semiplanar but interpret it with chroma components swapped compared to other devices (and the reference). Some encoders (also Samsung Exynos) advertise planar but interpret it as a proprietary tiled format (or crash). Some encoders (Qualcomm) assume extra alignment between the luma and chroma planes.

See https://code.google.com/p/android/issues/detail?id=37769 for more details on the issues you can encounter. In practice you can probably get it working on a large number of devices, but you'll need to more or less verify it per device and enable different quirk workarounds per device.

Linotype

Thanks, this is exactly what I needed to hear. Then is encoding using Surface a safe - cross-device - solution? I don't do anything on my image outside of using different shaders. – Lunitidal

Surface should be safe AFAIK (but it appeared in 4.3, and is properly CTS tested), but I haven't used it much personally so I don't have experience to share, only an assumption that the CTS test hopefully have avoided broken devices from shipping. – Linotype

I'm using BigFlake custom OutputCodecSurface class used for compatibility with API <18. I hope it will be compatible with the InputSurface/OutputSurface pattern from API 18+, that's where I am now. I've used a GLSurfaceView as the screen renderer, and now I'll see if I'll be able to use a surface as a transcoder or encoder. – Lunitidal