HTML5 Video buffered attribute features

Question

I am designing a custom HTML5 video player. Thus, it will have its own custom slider to mimic the video progress, so I need to understand the entire buffering shebang of a HTML5 video.

I came across this article: Video Buffering. It says that the buffered object consists of several time ranges in linear order of start time. But I couldn't find out the following:

1. Say the video starts. It continues upto 1:45 on its own (occasionally stalling perhaps, waiting for further data), after which I suddenly jump to 32:45. Now after some time, if I jump back to 1:27 (within the time range initially loaded and played through, before I made the jump), will it start playing immediately as it was already loaded before? Or is it that since I made a jump, that portion is lost and will have to be fetched again? Either way, is the behavior consistent for all such scenarios?

2. Say I make 5 or 6 such jumps, each time waiting for a few seconds for some data to load after the jump. Does that mean the buffered object will have all those time ranges stored? Or might some get lost? Is it a stack kind of thing, where the earlier ranges will get popped off as more ranges get loaded due to further jumps?

3. Will checking whether the buffered object has one time range starting at 0 (forget live streaming) and ending at the video duration length ensure that the entire video resource has been loaded fully? If not, is there some way to know that the entire video has been downloaded, and any portion is seekable, from which video can play continuously upto end without a moment's delay?

The W3C specs are not very clear on this, and I also can't find a suitably large (say more than an hour) remote video resource to test.

Ti

Answer 1

How video is buffered is browser implementation-dependent and therefore may vary from browser to browser.

Various browsers can use different factors to determine to keep or to discard a part of the buffer. Old segments, disk space, memory, and performance are typical factors.

The only way to know is to "see" what the browser has or is loading.

For example - in Chrome I played a few seconds then I skipped to about 30 seconds and you can see that it starts to load another part starting from that position.

(The buffer also seem to be bounded to key-frames so it is possible to decode the n-frames in that buffer. This means the buffer can start to load data a little before the actual position).

I supplied a demo video about 1 minute long - however, this is not long enough to do proper testing. Free free to supply video links that contain longer video (or please share if you want me to update the demo with this).

The main function will iterate through the buffered object on the video element. It will render all parts that exist to the canvas right below the video showing in red.

You can click (bit not drag) on this viewer to move the video to different positions.

/// buffer viewer loop (updates about every 2nd frame)
function loop() {

    var b = vid.buffered,  /// get buffer object
        i = b.length,      /// counter for loop
        w = canvas.width,  /// cache canvas width and height
        h = canvas.height,
        vl = vid.duration, /// total video duration in seconds
        x1, x2;            /// buffer segment mark positions

    /// clear canvas with black
    ctx.fillStyle = '#000';
    ctx.fillRect(0, 0, w, h);

    /// red color for loaded buffer(s)
    ctx.fillStyle = '#d00';

    /// iterate through buffers
    while (i--) {
        x1 = b.start(i) / vl * w;
        x2 = b.end(i) / vl * w;
        ctx.fillRect(x1, 0, x2 - x1, h);
    }

    /// draw info
    ctx.fillStyle = '#fff';

    ctx.textBaseline = 'top';
    ctx.textAlign = 'left';
    ctx.fillText(vid.currentTime.toFixed(1), 4, 4);

    ctx.textAlign = 'right';
    ctx.fillText(vl.toFixed(1), w - 4, 4);

    /// draw cursor for position
    x1 = vid.currentTime / vl * w;

    ctx.beginPath();
    ctx.arc(x1, h * 0.5, 7, 0, 2 * Math.PI);
    ctx.fill();

    setTimeout(loop, 29);
}

Calyptrogen

When you pause the video and run buffered.end(index) it returns 1 and stop working. Is that a common behaviour ? couldn't find related problems. – Shippee

Answer 2

According to

• https://developer.mozilla.org/en-US/docs/Web/HTML/Element/video
• https://developer.mozilla.org/en-US/docs/Web/API/TimeRanges
• https://developer.mozilla.org/en-US/docs/Web/API/TimeRanges.start

the buffered attribute holds information about all currently buffered time ranges. To my understanding, if a buffered portion is lost, it is removed from the object (in case that ever happens).

Esepcially the last link seems to be very useful for understanding the matter (since it offers a code sample) but keep in mind these are mozilla documents and support might be different in other browsers.

To answer your questions

Say the video starts. It continues upto 1:45 on its own (occasionally stalling perhaps, waiting for further data), after which I suddenly jump to 32:45. Now after some time, if I jump back to 1:27 (within the time range initially loaded and played through, before I made the jump), will it start playing immediately as it was already loaded before?

It should play immediately when jumping back unless the buffer of that portion was unloaded. I think it's very reasonable to assume that buffers or buffer ranges are unloaded at some point if the overall buffersize exceeds a certain volume.

Say I make 5 or 6 such jumps, each time waiting for a few seconds for some data to load after the jump. Does that mean the buffered object will have all those time ranges stored?

Yes, all buffered ranges should be readable through the attribute.

Will checking whether the buffered object has one time range starting at 0 (forget live streaming) and ending at the video duration length ensure tht the entire video resource has been loaded fully?

Yes, this is the code example in the last link. Apparently this is an applicable method of determining if the entire video has been loaded.

if (buf.start(0) == 0 && buf.end(0) == v.duration)

Amino

The problem is that I tried to experiment with the buffer. I let the video stream for 2-3 minutes, then jumped to 10 odd, then let it stream for 2 minutes, then to 20 and the same. Then I expected there will be 3 ranges in buffered, whereas I find one, the one currently buffering. – Ti

@Cupidvogel Well, either you've spent enough time at each position that it had enough time to download a large continuous portion of the video, or the unloading I was talking of is actually happening there. Either way, the buffered attribute should reflect what's happening. – Amino

Yeah, I thought the same too. But I am seeking only 3 portions, it seems too small a number to start clearing the cache. And no, the resulting single buffer doesn't combine the previous one and the current one (they are separated by a time span of 10 minutes, with the intermittent portion still not loaded, so no way can they be combined), it just refers to the starting point where I seeked, and the ending point gets updated as more data is loaded. – Ti

@Cupidvogel I have just dug through some of the Firefox source code and they indeed have a very elaborate buffer management system that will drop buffered segments under certain circumstances. For example, a buffered segment before the current playing position will be dropped if it isn't part of the current segement and playback wasn't interrupted by the user for some time. Most of this is in content/media/ in the source tree. – Amino

That's pretty bad, isn't it? You say Firefox, I observed the same in Chrome, Safari also.. – Ti

@Cupidvogel That depends on how you define "bad" I guess. To some people it might appear bad that the entire video isn't buffering. To other it could be bad that their memory and cache is being clogged with video data... I think the browser vendors are just trying to find the sweet spot in between. – Amino

In my design, I need to implement a custom progress-bar to mimic the progress. When the user jumps back to a portion he has downloaded before, I wanted to set the progress bar of that portion to the amount downloaded already, so that the user, apart from seeing that the video is not stalling, also sees the progress bar shoot rightwards immediately, assuring him that the content is not lost. How do I do it then? – Ti

@Cupidvogel YouTube for example has the same problem. When you've watched a video almost to the end and jump back to the beginning it will start downloading all over again, although the progressbar indicates that that portion was downloaded before. I don't see any real options for you but to live with it. – Amino

No, that's the problem. In Flash, the video is really lost. In HTML5 it is not lost, if you see in the native controls, if you jump, that portion will immediately shoot to show that this had been downloaded before, it's that there is no info to translate that info to custom controls. – Ti

Answer 3

1. Almost every browser saves the buffered data in cache for that session. The cache expires after the user goes away from that page. I don't think that the user will have to load the page each time he loads the video from a point where the video has been loaded. The user will face this issue only when the server is clearing out all the cache data. HTML5 video tag will support this, and will save the video upto the point till where it has been loaded.

2. It doesnot mean that the session has been lost, it means that either the object (if you are using flash player) is looking for some data from that particular point or the html5 video tag is having some issues either because of the INTERNET connection failure, or some other server errors.

3. The metadata is automatically loaded, untill you use this <audio preload="none"... this will make the browser not to download anything from server, you can use it as:
   <audio preload="auto|metadata|none"... If you use none, nothing is downloaded unless the user clicks play button, and metadata will download name, timing and other meta data from server, but not the file somehow, auto will start downloading as soon as the page loads.

I will always refer you to read some documentations by jQuery. As the jQuery will let you change and update the content using ajax API and will be helpfull too. Hope you succeed! Cheers.

Bemba

Answer 4

Although the accepted answer's description is excellent, I decided to update its code sample, for several reasons:

• The progress render task should really be fired only on a progress event.
• The progress render task is mixed up with some other tasks like drawing the timestamp and the playhead position.
• The code refers to several DOM elements by their IDs without using document.getElementById().
• The variable names were all obscured.
• I thought a forward for() loop was more elegant than a backward while() loop.

Note that I have removed the playhead and timestamp to keep the code clean, as this answer focusses purely on visualisation of the video buffer.

LINK TO ONLINE VIDEO BUFFER VISUALISER

Rewrite of accepted answer's loop() function:

function drawProgress(canvas, buffered, duration){
    // I've turned off anti-aliasing since we're just drawing rectangles.
    var context = canvas.getContext('2d', { antialias: false });
    context.fillStyle = 'blue';

    var width = canvas.width;
    var height = canvas.height;
    if(!width || !height) throw "Canvas's width or height weren't set!";
    context.clearRect(0, 0, width, height); // clear canvas

    for(var i = 0; i < buffered.length; i++){
        var leadingEdge = buffered.start(i) / duration * width;
        var trailingEdge = buffered.end(i) / duration * width;
        context.fillRect(leadingEdge, 0, trailingEdge - leadingEdge, height);
    }
}

Antecedent

Answer 5

This is just a variation of this excellent answer https://mcmap.net/q/527765/-html5-video-buffered-attribute-features

I only made it work without any work required and added few perks. Everything is automatic.

• currently intended for full-screen video playback such as netflix or hbogo
• automatically creates the canvas
• auto-updates width to 100% viewport width
• works as a bookmarklet
• does not obstruct the view much (transparent, 2px tall)

function prepare() {
    console.log('prepare');

    _v = $('video')[0];
    _v.insertAdjacentHTML('afterend',
    `<canvas
        id="WowSuchName"
        height="1"
        style="
            position: absolute;
            bottom: 0;
            left: 0;
            opacity: 0.4;
        "></canvas>`);

    _c = WowSuchName
    _cx = _c.getContext('2d');

    window.addEventListener('resize', resizeCanvas, false);

    function resizeCanvas() {
        console.log('resize');
        _c.width = window.innerWidth;
    }
    resizeCanvas();
}

/// buffer viewer loop (updates about every 2nd frame)
function loop() {
    if (!window.WowSuchName) { prepare(); }

    var b = _v.buffered,  /// get buffer object
        i = b.length,     /// counter for loop
        w = _c.width,     /// cache canvas width and height
        h = _c.height,
        vl = _v.duration, /// total video duration in seconds
        x1, x2;           /// buffer segment mark positions

    /// clear canvas
    _cx.clearRect(0, 0, w, h);

    /// color for loaded buffer(s)
    _cx.fillStyle = '#888';

    /// iterate through buffers
    while (i--) {
        x1 = b.start(i) / vl * w;
        x2 = b.end(i) / vl * w;
        _cx.fillRect(x1, 0, x2 - x1, h);
    }

    /// draw cursor for position
    _cx.fillStyle = '#fff';
    x1 = _v.currentTime / vl * w;
    _cx.fillRect(x1-1, 0, 2, h);

    setTimeout(loop, 29);
}

loop();

And the code for bookmarklet is here

javascript:eval(atob("CmZ1bmN0aW9uIHByZXBhcmUoKSB7CiAgICBjb25zb2xlLmxvZygncHJlcGFyZScpOwoKICAgIF92ID0gJCgndmlkZW8nKVswXTsKICAgIF92Lmluc2VydEFkamFjZW50SFRNTCgnYWZ0ZXJlbmQnLAogICAgYDxjYW52YXMKICAgICAgICBpZD0iV293U3VjaE5hbWUiCiAgICAgICAgaGVpZ2h0PSIxIgogICAgICAgIHN0eWxlPSIKICAgICAgICAgICAgcG9zaXRpb246IGFic29sdXRlOwogICAgICAgICAgICBib3R0b206IDA7CiAgICAgICAgICAgIGxlZnQ6IDA7CiAgICAgICAgICAgIG9wYWNpdHk6IDAuNDsKICAgICAgICAiPjwvY2FudmFzPmApOwogICAgCiAgICBfYyA9IFdvd1N1Y2hOYW1lCiAgICBfY3ggPSBfYy5nZXRDb250ZXh0KCcyZCcpOwoKICAgIHdpbmRvdy5hZGRFdmVudExpc3RlbmVyKCdyZXNpemUnLCByZXNpemVDYW52YXMsIGZhbHNlKTsKCiAgICBmdW5jdGlvbiByZXNpemVDYW52YXMoKSB7CiAgICAgICAgY29uc29sZS5sb2coJ3Jlc2l6ZScpOwogICAgICAgIF9jLndpZHRoID0gd2luZG93LmlubmVyV2lkdGg7CiAgICB9CiAgICByZXNpemVDYW52YXMoKTsKfQoKLy8vIGJ1ZmZlciB2aWV3ZXIgbG9vcCAodXBkYXRlcyBhYm91dCBldmVyeSAybmQgZnJhbWUpCmZ1bmN0aW9uIGxvb3AoKSB7CiAgICBpZiAoIXdpbmRvdy5Xb3dTdWNoTmFtZSkgeyBwcmVwYXJlKCk7IH0KCiAgICB2YXIgYiA9IF92LmJ1ZmZlcmVkLCAgLy8vIGdldCBidWZmZXIgb2JqZWN0CiAgICAgICAgaSA9IGIubGVuZ3RoLCAgICAgLy8vIGNvdW50ZXIgZm9yIGxvb3AKICAgICAgICB3ID0gX2Mud2lkdGgsICAgICAvLy8gY2FjaGUgY2FudmFzIHdpZHRoIGFuZCBoZWlnaHQKICAgICAgICBoID0gX2MuaGVpZ2h0LAogICAgICAgIHZsID0gX3YuZHVyYXRpb24sIC8vLyB0b3RhbCB2aWRlbyBkdXJhdGlvbiBpbiBzZWNvbmRzCiAgICAgICAgeDEsIHgyOyAgICAgICAgICAgLy8vIGJ1ZmZlciBzZWdtZW50IG1hcmsgcG9zaXRpb25zCgogICAgLy8vIGNsZWFyIGNhbnZhcwovLyAgICAgX2N4LmZpbGxTdHlsZSA9ICcjMDAwJzsKLy8gICAgIF9jeC5maWxsUmVjdCgwLCAwLCB3LCBoKTsKICAgIF9jeC5jbGVhclJlY3QoMCwgMCwgdywgaCk7CgogICAgLy8vIGNvbG9yIGZvciBsb2FkZWQgYnVmZmVyKHMpCiAgICBfY3guZmlsbFN0eWxlID0gJyM4ODgnOwoKICAgIC8vLyBpdGVyYXRlIHRocm91Z2ggYnVmZmVycwogICAgd2hpbGUgKGktLSkgewogICAgICAgIHgxID0gYi5zdGFydChpKSAvIHZsICogdzsKICAgICAgICB4MiA9IGIuZW5kKGkpIC8gdmwgKiB3OwogICAgICAgIF9jeC5maWxsUmVjdCh4MSwgMCwgeDIgLSB4MSwgaCk7CiAgICB9CgogICAgLy8vIGRyYXcgY3Vyc29yIGZvciBwb3NpdGlvbgogICAgX2N4LmZpbGxTdHlsZSA9ICcjZmZmJzsKICAgIHgxID0gX3YuY3VycmVudFRpbWUgLyB2bCAqIHc7CiAgICBfY3guZmlsbFJlY3QoeDEtMSwgMCwgMiwgaCk7CgogICAgc2V0VGltZW91dChsb29wLCAyOSk7Cn0KCmxvb3AoKTsK"))

Goldie