MPEG-4 and alpha transparency - the ongoing saga

Question

I'd like to reach some sort of definitive answer for the following questions:

1. Is alpha transparency supported in MPEG-4-based codecs?
2. Is there any way to be reasonably certain that there is no alpha-channel from the output of ffprobe?

Some links I've found on the subject:

• https://mcmap.net/q/323757/-how-to-create-an-h264-video-with-an-alpha-channel-for-use-with-html5-canvas

Also, transparent alpha layers, an optional part of the H.264 specification, are now supported in H.264-based QuickTime movies.

• http://software.intel.com/en-us/articles/alpha-channel-support-in-the-h264-decoder-in-ipp/

The support for alpha channel in the H.264 decoder is now available in Intel® Integrated Performance Primitives 6.0 version.

• http://forums.creativecow.net/readpost/335/30475

The MPEG4 Simple Studio Profile supports auxiliary channels, which could be used for alpha or depth -- and presumably even displacement, motion vectors, normals, etc.

However, I've never seen a concrete example, or indeed any details on how you use these announced features.

Olibanum

I can not see any conflict. All three say, that MPEG-4 supports it. – Domain

The first says that it doesn't, and that claims that it does are a miss-undertanding of Apple's layer composition tool. The second mentions a decoding opcode, but no follow-up to the reasonable question of what it's decoding. And the third is not backed-up with any evidence and the only reference to "Simple Studio Profile" I can find makes no mention of alpha channels or transparency. – Olibanum

The fist quotes the following description: docs.info.apple.com/article.html?artnum=42599 which does not look entirely ambiguous. So all three sources say that there is alpha support. And the sources say that it is an optional feature. This paper explains it a bit more: icme2008.org/papers/data/377/377_1.pdf – Domain

If I could see an example of such a video I'd be more convinced. Even so, part two of my question is still to be answered. – Olibanum

Answer 1

The short answer is that h.264 does not support an alpha channel since no decoder/encoder support the functionality. See this creativecow forum post for more info. While an alpha channel does not work in the h.264 video itself, it is possible to use 2 different h.264 videos to get the same effect. See my post to the creativecow forum for more info.

Rasbora

I would believe you, but see the second link in the question. – Olibanum

Feel free to demonstrate where you can find both an encoder and a decoder that supports h.264 with an alpha channel. I have read most everything that can be found online about h.264 and alpha channel support (including this ipp stuff), basically this functionality does not exist. You can read about my own implementation that works around this problem if you like, it is linked in the forum post above. – Rasbora

Answer 2

H.264 has an auxillary channel, set in place in codec and format specification as useable in providing transparency. I have not seen an encoder capable of using it.

(PDF download only) https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-H.264-201106-S!!PDF-E&type=items you can read about it here. It's a bit dense, but the International Telecommunication Union's spec for H.264 encoding explicitly states transparency(and additional luma maps) as being a use for the specification's auxiliary channel. However it also notes "The... decoding of auxiliary coded pictures is not required" and I don't think it was ever used in an official capacity.

The confusion is likely because mp4 and quicktime can both use the h.264 codec, but quicktime, (both the container and player) supports transparency, so theoretically mp4 can be encoded with it as well, though it would require a dedicated decoder. I have seen a hack that simply encodes an alpha layer to another video or at the end of the file. Adobe video export can default to uncompressed RGBA in the case that quicktime alpha output is requested, meaning you can ask for h.264 quicktime, request alpha, and get a quicktime with transparency that is not h.264. This confused me in testing and took a couple tries to figure out what had actually happened to my file. Combined with confusion over quicktime player and wrapper(they support both H.264 and alpha but not simultaneously), all of this could be a source of public confusion.

This is incredibly late, I understand, but these videos, I believe, primarily come from After Effects' and Adobe's encoders and decoders, which are Apple-blessed, and therefore take advantage of Apple quicktime player's RGB+A decoding update which was not related to its h.264 characteristics. Adobe removed them when Apple deprecated both quicktime and the de4dicated player, but the wrapper was added back after years of public protest because there was no functionality to replace the workflow. It is indeed a saga.

Apple has deprecated quicktime, and it's been deprecated for years, but is still in active use in VFX because to my knowledge neither Apple nor Adobe supplied a replacement for video that was both heavily compressed and supported alpha. Webm is looking like a modern replacement, but its support by third parties is spotty, and AV1, touted as an h.264 replacement does not seem to officially support it at all.

(edited for correctness- mostly the bold stuff. sorry people who noted I had it a little backwards! quicktime being deprecated and prores and uncompressed RGBA using the same extension as quicktime really obfuscates things... premiere had the codec ommitted because it was uncompressed, which turned me around as I hadn't requested it.)

Incunabulum

That video uses a ProRes codec, not H.264 – Olibanum

It's also an MPEG-4 .mov container, not an old QT one. – Olibanum

Thanks- corrected. I also added some additional information I found, including an official release of information regarding alpha channels in h.264. – Incunabulum

You're still confusing container formats and codecs. And the uncompressed RGBA from Apple isn't ProRes (as your previous example) but a PNG codec. – Olibanum

Perhaps you should show some screenshots from Premier where you can select both H.264 and RGBA at the same time. – Olibanum