HTML5 Canvas drawImage from video not showing on first draw

Question

I have a video element with a canvas overlay. Then I have a drawing tool setup to draw over the video and a save button that does a drawImage from both the video and then the canvas to save a comped frame. However, the first time I press Save I only get the result from the canvas drawImage, the video does not show. On subsequent Saves I receive both images properly layered. I thought this might be an issue with loading of the video image, but the video is fully loaded before I hit save and can even advance frames and have it work properly on the 2nd Save.

Here is the code...

<div style="width:960px; height:540px; display:inline-block;">
    <video id="video" src="media/_tmp/AA_017_COMP_v37.mov" width="960" height="540" ></video>
</div>
<canvas id="canvas" width="960" height="540" style="position:absolute; top:40px; left:9px; z-index:100;"></canvas>
<input type="button" value="save" id="btn" size="30" onclick="save()" style="float:left; padding:4px; margin-right:4px;" >
<div id="saved" style="border:1px solid #000; position:absolute; top:626px; left:10px; bottom:40px; width:958px; overflow:auto;">SAVED:</div>


function save() {

    //COMP CANVAS OVER VIDEOFRAME
    var video = document.getElementById("video");
    var currentFrame = Math.floor((<?php echo $mov_frames ?> / video.duration) * video.currentTime);

    var compCanvas = document.createElement('canvas');
    compCanvas.width = video.width;
    compCanvas.height = video.height;
    compContext = compCanvas.getContext('2d');
    compContext.drawImage(video, 0, 0);
    compContext.drawImage(canvas, 0, 0);
    var dataURL = compCanvas.toDataURL();

    $("#saved").append('<div style="width:954px; border-bottom:1px solid #000; padding:2px 2px 0 2px;"><img id="compFrame_'+currentFrame+'" width="180" height="90" src="'+dataURL+'" />Frame: '+currentFrame+'</div>');
}

Amazing

Can you provide a jsfiddle showing the problem? – Cryptoanalysis

Here is a jsFiddle link The only problem I don't know for the jsFiddle is the cross browser video link won't draw to a canvas... but never the less the code is there. – Amazing

is your code launched only after DOM loaded ? – Sneakers

The save function lives outside of $(document).ready but the function is only ever called after the page is fully loaded. – Amazing

1) Maybe it's an off-by-one error due to some buffer swap issue. Can you be sure the image 'printed' is the very one seen, and not the previous ? And also, an idea you have had already : 2) what about playing also the video in a canvas ? this way you would be 100% sure of the images you have at hand. – Sneakers

The video I'm actually testing with does have a frame number burn-in so I'll double check I'm saving the right frame. Also I thought about playing the video through the same canvas and then writing directly on that, but I would loose my ability to erase any drawing. I haven't tried running it through it's own canvas but I'm not sure why that would be any different. Worst case till I figure this out I can hack it so the first time it tries to call the image it actually runs through the process twice. The overhead is pretty low so while horribly inelegant I don't think the user would notice. – Amazing

If you draw on another canvas, you have no wonder about the image being shown vs the image in buffer AND you can still do a clear on topmost canvas with no question. // For the 'read-twice-first' solution : yes... as long as it works for all (supported) video formats/browsers/platform. I fear the definite answer here lies in testing.... – Sneakers

So the read-twice-first solution actually yielded the same results! Now I am throughly confused as to why it would not be reading the video. Seems like it might be time to explore the dual canvas idea. – Amazing

Also, can you see if this fiddle produces the frames as expected (it seem to work for me): jsfiddle.net/AbdiasSoftware/zL8KC/1 – Phasis

Hi Ken.. thanks for the question. I did try another format and received the same results. I will try other browsers and see if anything changes. Currently I am testing with Safari on OSX Mavericks. I tried your jsFiddle and used snaps.appendChild(compCanvas); instead of the toDataURL and it was the same result. – Amazing

Answer 1

As this is on OSX with Safari there is some bad news:

Note: Video as a source for the canvas drawImage() method is not currently supported on iOS.

Source

If this is outdated information I don't know but in either case the current implementation has issues. As established (in comments) the problem is not likely related to codecs as this occur with other formats as well. The fiddle eliminates the toDataURL() method as source so what is left is the drawImage() method.

As this works on other platform with the provided code, and from the look of it, this is neither the source of the problem so you are here looking at a very possible issue (bug) with the Safari browser on the OSX platform.

The official documentation quoted above also states this is not supported so this is probably a very early implementation that still has some issues.

There is not much you can do in that regard but to wait and also report it as an issue (I looked for a reported issue but couldn't find any so I recommend you do this).

Phasis

I believe you are correct. Trying this on firefox on OSX as well as chrome and firefox on Win7 all seem to produce the correct behavior. – Amazing

Actually going back and reading the source link you provided I realized that iOS specifically is not supported. Mavericks on a desktop system is not iOS. However it does appear to be directly related to Safari so it may still be a bug. – Amazing

Answer 2

I recently ran into the same issue myself, and worked around it by drawing the video onto a "dummy" canvas before doing "real" work with it. This works well as all of the methods in my code that attempt to parse the video wait until the "canplaythrough" event has been triggered on the video. I wrap the "canplaythrough" event handler in a Promise, and only resolve the promise when the "canplaythrough" event has been received AND an attempt has been made to draw the video onto a dummy canvas. This appears to be an effective workaround for the Safari bug.

For example:

var readyPromise = new Promise(function(resolve) {
    video.addEventListener("canplaythrough", function() {
        var canvas = document.createElement("canvas"),
            context = canvas.getContext("2d");

        context.drawImage(video, 0, 0);

        resolve(video);
    });
});

readyPromise.then(function() {
    // NOW manipulate the video, draw it onto a canvas, etc
});

Hylo

It doesn't work anymore in chrome. I see canplaythrough fires, but drawImage draws nothing. Setting timeout to 100ms inside canplaythrough helps, but maybe there's less hacky way. – Transmarine

Answer 3

I met the same issue and tried out the solution in the last comment. It didn't work for me quite yet.

I figured later and found a workaround:

in my code I have some other drawing along with drawImage from video like

context.rect(0, 0, 50, 50);
context.fillStyle = 'black';
context.fill();

and with these, the second draw still won't work even when redraw it after 'canplaythrough' event.

So I removed the above three lines, and just did a second draw after event 'canplaythrough'. Then it worked.

Simply, the code that worked for me is like this:

context.drawImage(video, 0, 0);
video.addEventListener('canplaythrough', function () {
    context.drawImage(video, 0, 0);
});

Illuminate

Answer 4

None of these worked for me. It seems like chrome (atm it's version 97) is doing some nasty optimization in background.

• canplaythrough seems to be lazy, and doesn't guaranty that 1st frame is loaded
• if video has style display: none it won't be loaded as well, thus drawing won't work either.
• video element should be attached to DOM
• canvas element should not have any css related to its width or size.

The only thing worked is setTimeout, it's a nasty hack, but it is like it is.

fileinput.addEventListener('change', function (e) {
    var ctx = canvas.getContext("2d");
    video.addEventListener('loadedmetadata', function () {
        canvas.width = video.videoWidth;
        canvas.height = video.videoHeight;
        setTimeout(() => {
            ctx.drawImage(video, 0, 0);
        }, 100)
    });
    video.querySelector("source").setAttribute('src', URL.createObjectURL(fileinput.files[0]));
    video.load();
});

video {
    opacity: 0;
    width: 0;
    height: 0;
} 

<input type="file" id="fileinput" name="file">
<video id="video" controls>
  <source type="video/mp4">
</video>
<canvas id="canvas"></canvas>

Transmarine