How to create an h264 video with an alpha channel for use with HTML5 Canvas?

Question

I've been interested in this demo: http://jakearchibald.com/scratch/alphavid/

I also saw this question on here:

Can I have a video with transparent background using HTML5 video tag?

But I can't seem to figure out: How do you create h264, ogg and webm video files with alpha channels?

Sabian

Answer 1

If you open this video from your demo with QuickTime you will see a video with separate RGB and A regions:

If you then look at the code for the demo, you will see that it is using one offscreen HTML5 Canvas to draw each video frame to, reading the alpha pixels from the second half of that canvas to set explicit per-pixel alpha values in the first half, and then using putImageData to shove the result to another HTML5 Canvas which is actually displaying the video.

function processFrame() {
  buffer.drawImage(video, 0, 0);
                
  var image = buffer.getImageData(0, 0, width, height),
      imageData = image.data,
      alphaData = buffer.getImageData(0, height, width, height).data;
                
  for (var i=3, len=imageData.length; i<len; i += 4){
    imageData[i] = alphaData[i-1];
  }
                
  output.putImageData(image, 0, 0, 0, 0, width, height);
}

Coupled with various statements throughout the web that H.264 does not support alpha and I almost feel confident that H.264 cannot have alpha. I say this because I find multiple references to this 2006 quote from Apple computer regarding updated QuickTime in their "Leopard" release of OS X:

QuickTime’s plumbing is receiving significant upgrades in Leopard. There have been significant enhancements in handling the H.264 encoding. Also, transparent alpha layers, an optional part of the H.264 specification, are now supported in H.264-based QuickTime movies. And finally, QuickTime supports 64-bit.

The only thing I have found related to this marketing quote, however, is this document showing how to change the overall opacity of a video layer in QuickTime Pro.

Patrilineal

Hi @Phrogz, thank you I'm starting to understand. But here is another question. I'm anything but a JavaScript expert, so I apologize if this is obvious, but this is what I thought what was happening: The for-loop goes through every 4th element of the image data array, which apparently represents the alpha channel and the previous three being R, G and B. But you are saying that this demo creates alpha data without an actual alpha channel, is that right? – Sabian

@Sabian Correct: the data attribute of an ImageData object is a flat array-like structure called a CanvasPixelArray that has all RGBA values in series. The script above is taking the values from the blue channel in the lower half of the image and using them as the alpha values in the upper half. Then it copies this new image-with-alpha created each frame to the output. – Patrilineal

Thank you so much for your help, I finally understand what's happening here. – Sabian

Answer 2

WebM does support transparency now, and Chrome 29+ can play it back without a flag.

http://updates.html5rocks.com/2013/07/Alpha-transparency-in-Chrome-video

But it does not work on Chrome for Android.

Unman