requestAnimationFrame JavaScript: Constant Frame Rate / Smooth Graphics
Asked Answered
C

2

7

According to several developers (link1, link2) the proper way to have a constant frame rate with requestAnimationFrame is to adjust the "last rendered" time within the game loop as follows:

function gameLoop() {

    requestAnimationFrame(gameLoop);

    now = Date.now();
    delta = now - then;

    if (delta > interval) {
        then = now - (delta % interval); // This weird stuff
        doGameUpdate(delta);
        doGameRender();
    }
}

Where interval is 1000/fps (i.e. 16.667ms).

The following line makes no sense to me:

then = now - (delta % interval);

Indeed if I try it I don't get smooth graphics at all but fast then slow depending on the CPU: https://jsfiddle.net/6u82gpdn/

If I just let then = now (which makes sense) everything works smoothly: https://jsfiddle.net/4v302mt3/

Which way is "correct"? Or what are the tradeoffs I am missing?

Chong answered 19/9, 2017 at 12:50 Comment(0)
D
29

Delta time is bad animation.

It seems just about anyone will post a blog about the right way to do this and that, and be totally wrong.

Both articles are flawed as they do not understand how requestAnimationFrame is called and how it should be used in relation to frame rate and time.

When you use delta time to correct animation positions via requestAnimationFrame you have already presented the frame, it's too late to correct it.

  • requestAnimationFrame's callback function is passed an argument that holds the high precision time in ms (1/1000th) accurate to microseconds (1/1,000,000th) second. You should use that time not the Date objects time.

  • The callback is called as soon as possible after the last frame was presented to the display, there is no consistency in the interval between calls to the callback.

  • Methods that use delta time need to predict when the next frame is presented so that object can be render at the correct position for the upcoming frame. If your frame rendering load is high and variable, you can not predict at the start of frame, when the next frame will be presented.

  • The rendered frame is always presented during the vertical display refresh and is always on a 1/60th second time. The time between frames will always be integers multiples of 1/60th giving only frame rates of 1/60, 1/30, 1/20, 1/15 and so on

  • When you exit the callback function the rendered content is held in the backbuffer until the next vertical display refresh. Only then is it moved to display RAM.

  • The frame rate (vertical refresh) is tied to the device hardware and is perfect.

  • If you exit the callback late, so that the browser does not have time to move the canvas content to the display, the back buffer is held until the next vertical refresh. Your next frame will not be called until after the buffer has been presented.

  • Slow renders do not reduce frame rate, they cause frame rate oscillations between 60/30 frames per second. See example snippet using mouse button to add render load and see dropped frames.

Use the time supplied to the callback.

There is only one time value you should use and that is the time passed by the browser to the requestAnimationFrame callback function

eg

function mainLoop(time){  // time in ms accurate to 1 micro second 1/1,000,000th second
   requestAnimationFrame(mainLoop);
}
requestAnimationFrame(mainLoop);

Post frame correction error.

Don't use delta time based animation unless you must. Just let the frames drop or you will introduce animation noise in the attempt to reduce it.

I call this post frame correction error (PFCE). You are attempting to correct a position in time for an upcoming and uncertain time, based on the past frame time, which may have been in error.

Each frame you are rendering will appear some time from now (hopefully in the next 1/60th second). If you base the position on the previouse rendered frame time and you dropped a frame and this frame is on time you will render the next frame ahead of time by one frame, and the same applies to the previous frame which would have been rendered a frame behind as a frame was skipped. Thus with only a frame dropped you render 2 frames out of time. A total of 3 bad frames rather than 1.

If you want better delta time, count frames via the following method.

var frameRate = 1000/60;
var lastFrame = 0;
var startTime;
function mainLoop(time){  // time in ms accurate to 1 micro second 1/1,000,000th second
   var deltaTime = 0
   if(startTime === undefined){
       startTime = time;
   }else{
       const currentFrame = Math.round((time - startTime) / frameRate);
       deltaTime = (currentFrame - lastFrame) * frameRate;
   }
    lastFrame = currentFrame;
   requestAnimationFrame(mainLoop);
}
requestAnimationFrame(mainLoop);

This does not eliminate PFCE but is better than irregular interval time if you use delta time as timeNow - lastTime.

Frames are always presented at a constant rate, requestAnimationFrame will drop frames if it can not keep up, but it will never present mid frame. The frame rates will be at fixed intervals of 1/60, 1/30, 1/20, or 1/15 and so on. Using a delta time that does not match these rates will incorrectly position your animation.

A snapshot of animation request frames

enter image description here

This is a timeline of requestAnimationframe for a simple animation function. I have annotated the results to show when the callback is called. During this time the frame rate was constant at a perfect 60fps no frames where dropped.

Yet the times between callbacks is all over the place.

Frame render timing

The example shows frame timing. Running in SO sandbox is not the ideal solution and to get good results you should run this in a dedicated page.

What it shows (though hard to see for small pixels) is the various time error from ideal times.

  • Red is frame time error from the callback argument. It will be stable near 0ms from 1/60th second ideal frame time.
  • Yellow is the frame time error calculated using performance.now(). It varies about 2 ms in total with the occasional peek outside the range.
  • Cyan is the frame time error calculated using Date.now(). You can clearly see the aliasing due to the poor resolution of the date's ms accuracy
  • Green dots are the difference in time between the callback time argument and the time reported by performance.now() and on my systems is about 1-2ms out.
  • Magenta is the last frame's render time calculated using performance now. If you hold the mouse button you can add a load and see this value climb.
  • Green vertical lines indicate that a frame has been dropped / skipped
  • The dark blue and black background marks seconds.

The primary purpose of this demo is to show how frames are dropped as render load increase. Hold the mouse button down and the render load will start to increase.

When the frame time gets close to 16 ms you will start to see frames dropped. Until the render load reaches about 32ms you will get frames between 1/60 and 1/30, first more at 1/60th for every one at 1/30th.

This is very problematic if you use delta time and post frame correction as you will constantly be over and under correcting the animation position.

const ctx = canvas.getContext("2d");
canvas.width = 512;
canvas.height = 380;

const mouse  = {x : 0, y : 0, button : false}
function mouseEvents(e){
	mouse.x = e.pageX;
	mouse.y = e.pageY;
	mouse.button = e.type === "mousedown" ? true : e.type === "mouseup" ? false : mouse.button;
}
["down","up","move"].forEach(name => document.addEventListener("mouse"+name,mouseEvents));

var lastTime;   // callback time
var lastPTime;  // performance time
var lastDTime;  // date time
var lastFrameRenderTime = 0; // Last frames render time
var renderLoadMs = 0;  // When mouse button down this slowly adds a load to the render
var pTimeErrorTotal = 0;
var totalFrameTime = 0;
var totalFrameCount = 0;
var startTime;
var clearToY = 0;
const frameRate = 1000/60;
ctx.font = "14px arial";
var w = canvas.width;
var h = canvas.height;
var cw = w / 2;  // center 
var ch = h / 2;
var globalTime;  // global to this 
ctx.clearRect(0,0,w,h);

const graph = (()=>{
    var posx = 0;
    const legendW = 30;
    const posy = canvas.height - 266;
    const w = canvas.width - legendW;
    const range = 6;
    const gridAt = 1;
    const subGridAt = 0.2;
    const graph = ctx.getImageData(0,0,1,256);
    const graph32 = new Uint32Array(graph.data.buffer);
    const graphClearA = new Uint32Array(ctx.getImageData(0,0,1,256).data.buffer);
    const graphClearB = new Uint32Array(ctx.getImageData(0,0,1,256).data.buffer);
    const graphClearGrid = new Uint32Array(ctx.getImageData(0,0,1,256).data.buffer);
    const graphFrameDropped = ctx.getImageData(0,0,1,256);
    const graphFrameDropped32 = new Uint32Array(graphFrameDropped.data.buffer);
    graphClearA.fill(0xFF000000);
    graphClearB.fill(0xFF440000);
    graphClearGrid.fill(0xFF888888);
    graphFrameDropped32.fill(0xFF008800);
    const gridYCol = 0xFF444444;  // ms marks
    const gridYColMaj = 0xFF888888;  // 4 ms marks
    const centerCol = 0xFF00AAAA;
    ctx.save();
    ctx.fillStyle = "black";
    ctx.textAlign = "right";
    ctx.textBaseline = "middle";
    ctx.font = "10px arial";
    for(var i = -range; i < range; i += subGridAt){
        var p = (i / range) * 128 + 128 | 0;
        i = Number(i.toFixed(1));
        graphFrameDropped32[p] = graphClearB[p] = graphClearA[p] = graphClearGrid[p] = i === 0 ? centerCol : (i % gridAt === 0) ? gridYColMaj : gridYCol;
        if(i % gridAt === 0){
            ctx.fillText(i + "ms",legendW - 2, p + posy);
            ctx.fillText(i + "ms",legendW - 2, p + posy);
        }
    }
    ctx.restore();
    var lastFrame;
    return {
        step(frame){
            if(lastFrame === undefined){
                lastFrame = frame;
            }else{
                
                while(frame - lastFrame > 1){
                    if(frame - lastFrame > w){ lastFrame = frame - w - 1 } 
                    lastFrame ++;
                    ctx.putImageData(graphFrameDropped,legendW + (posx++) % w, posy);
                }
                lastFrame = frame;
                ctx.putImageData(graph,legendW + (posx++) % w, posy);
                ctx.fillStyle = "red";
                ctx.fillRect(legendW + posx % w,posy,1,256);
                if((frame / 60 | 0) % 2){
                    graph32.set(graphClearA)
                }else{
                    graph32.set(graphClearB)
                    
                }
            }
        },
        mark(ms,col){
            const p = (ms / range) * 128 + 128 | 0;
            graph32[p] = col;
            graph32[p+1] = col;
            graph32[p-1] = col;
        }
    }
        
})();


function loop(time){
    var pTime = performance.now();
    var dTime = Date.now();
    var frameTime = 0;
    var framePTime = 0;
    var frameDTime = 0;
    if(lastTime !== undefined){
        frameTime = time - lastTime;
        framePTime = pTime - lastPTime;
        frameDTime = dTime - lastDTime;
        graph.mark(frameRate - framePTime,0xFF00FFFF);
        graph.mark(frameRate - frameDTime,0xFFFFFF00);
        graph.mark(frameRate - frameTime,0xFF0000FF);
        graph.mark(time-pTime,0xFF00FF00);
        graph.mark(lastFrameRenderTime,0xFFFF00FF);
        
        pTimeErrorTotal += Math.abs(frameTime - framePTime);
        totalFrameTime += frameTime;
        totalFrameCount ++;
    }else{
        startTime = time;
    }
    
    lastPTime = pTime;
    lastDTime = dTime;
    lastTime = globalTime = time;
    var atFrame = Math.round((time -startTime) /  frameRate);
    ctx.setTransform(1,0,0,1,0,0); // reset transform
    ctx.clearRect(0,0,w,clearToY);
    ctx.fillStyle = "black";
    var y = 0;
    var step = 16;
    ctx.fillText("Frame time : " + frameTime.toFixed(3)+"ms",10,y += step);
    ctx.fillText("Rendered frames : " + totalFrameCount,10,y += step);
    ctx.fillText("Mean frame time : " + (totalFrameTime / totalFrameCount).toFixed(3)+"ms",10,y += step);
    ctx.fillText("Frames dropped : " + Math.round(((time -startTime)- (totalFrameCount * frameRate)) / frameRate),10,y += step);
    ctx.fillText("RenderLoad : " + lastFrameRenderTime.toFixed(3)+"ms Hold mouse into increase",10,y += step);
    clearToY = y;
    graph.step(atFrame);

    requestAnimationFrame(loop);

    if(mouse.button ){
        renderLoadMs += 0.1;
        var pt = performance.now();
        while(performance.now() - pt < renderLoadMs);
    }else{
        renderLoadMs = 0;
    }
    
    lastFrameRenderTime = performance.now() - pTime;
}
requestAnimationFrame(loop);
canvas { border : 2px solid black; }
body { font-family : arial; font-size : 12px;}
<canvas id="canvas"></canvas>
<ul>
<li><span style="color:red">Red</span> is frame time error from the callback argument.</li>
<li><span style="color:yellow">Yellow</span> is the frame time error calculated using performance.now().</li>
<li><span style="color:cyan">Cyan</span> is the frame time error calculated using Date.now().</li>
<li><span style="color:#0F0">Green</span> dots are the difference in time between the callback time argument and the time reported by performance.now()</li>
<li><span style="color:magenta">Magenta</span> is the last frame's render time calculated using performance.now().</li>
<li><span style="color:green">Green</span> vertical lines indicate that a frame has been dropped / skipped</li>
<li>The dark blue and black background marks seconds.</li>
</ul>

For me I never use delta time for animation, and I accept that some frames will be lost. But overall you get a smoother animation using a fixed interval than attempting to correct the time post render.

The best way to get smooth animation is to reduce the render time to under 16ms, if you can't get that then use deltat time not to set animation frame but to selectively drop frames and maintain a rate of 30 frames per second.

Dietz answered 21/9, 2017 at 14:21 Comment(8)
Great answer, but what to do with screen refresh rates different than 60 Hz? Even my current desktop display is not perfect 60 Hz, but 59.997 Hz, what about that, any ideas?Underwaist
@Underwaist The down shift to 59.997Hz by the hardware is associated with video sync hardware that supports NTSC encoding. I am unsure how the browsers handle the missing frame (every ~5 min), but to prevent shearing they will refresh during the VSync, which is a hardware interrupt, thus in perfect sync with the display. Using the browser API's there is no reliable way to determine if frames are consistently ~0.05ms late, counting dropped frames will not work as browsers drop frames for many many reasons, 1 frame every ~20,000 frames.not a chance.Dietz
Thanks for more comment, but what about screens with completely different refresh rate, like 144 Hz? In your answer you're using hardcoded frameRate = 1000/60, so I assume it will work smoothly only at 60 Hz.Underwaist
@Underwaist if refresh rate is way out from 60Hz, the browser will ignore the hardware and present content every 60th second. The main reason for the browser to use the device refresh is to sync with VSync this stops shearing (most people don't notice it) and flickering (can be prevented using back buffers) . The 1000/60 is only a reference, There is but one speed on the browser 60Hz..and depending on the machine the browser will cruse with ease or struggle to keep up. If you want super smooth uHD HDI 240Hz best to not use the browser and JSDietz
OK, thank you again for answer. Do you have maybe any links or source that says more about browsers sticking to 60 Hz in different screen refresh rates?Underwaist
@AxelF I have read MDN rAF page developer.mozilla.org/en-US/docs/Web/API/window/… I see no mention of error associated with using the current DOMHighResTimeStamp.now (w3c.github.io/hr-time/#dom-domhighrestimestamp) , as passed to rAF callback, as specified in html.spec.whatwg.org/multipage/… , which can result in an incorrectly calculated time for the upcoming frame presentation, and why this can reduce animation quality. Please elaborate on how this answer is incongruous with MDN docs on rAFDietz
Sorry, should have specified. This sentence "The number of callbacks is usually 60 times per second, but will generally match the display refresh rate in most web browsers as per W3C recommendation." hints at frame rates higher than 60Hz. I don't know which is correct, but what is implied differs from your previous comment that 60Hz is as high as it goes.Morpho
@AxelF The "...usually 60..." does not hint at higher, rather it indicates that frame rates can be lower. Chrome (Webkit) limits frame rates to 60fps irrespective of the device frame rate. On chrome this may be turned off with --disable-frame-rate-limit and --disable-gpu-vsync Though on this device this does not seam to work (Win10 Current Chrome 120Hz display) still refuses to go over 60fps. Firefox can have vsync disabled by device driver switch, however the result is terrible shearing and a frame rate that is still 60fpsDietz
P
1

The point of a delta time is to keep the frame-rate stable by compensating for time taken by computations.

Think of this code:

var framerate = 1000 / 60;
var exampleOne = function () {
    /* computation that takes 10 ms */
    setTimeout(exampleOne, framerate);
}
var exampleTwo = function () {
    setTimeout(exampleTwo, framerate);
    /* computation that takes 30 ms */
}

In example one the function would calculate for 10 ms and then wait the frame-rate before painting the next frame. This will inevitably lead to a frame-rate lower than the expected.

In example two the function would start the timer for the next iteration immediately and then calculate for 30 ms. This will lead to the next frame being painted before the previous is done calculating, bottle necking your application.

With delta-time you get the best of both worlds:

var framerate = 1000 / 60;
var exampleThree = function () {
    var delta = Date.now();
    /* computation that takes 10 to 30 ms */
    var deltaTime = Date.now() - delta;
    if (deltaTime >= framerate) {
        requestAnimationFrame(exampleThree);
    }
    else {
        setTimeout(function () { requestAnimationFrame(exampleThree); }, framerate - deltaTime);
    }
};

With delta-time, which represents the calculation time, we know how much time we have left before the next frame needs to be painting.

We don't have the sliding performance from example one and we don't have a bunch of frames trying to draw at the same time as in example two.

Preteritive answered 19/9, 2017 at 14:3 Comment(3)
Yep that's it. I'd like to add that our game's update() function will still need to know the actual delta time between calls and this delta will indeed be wrong as applied above. I solved it by keeping track of 2 different deltas: one for rendering and one for updating.Chong
This does not take in account the display rate or backbuffer presentation time. Nor does it have anything to do with requestAnimationFrame And why I have to give a -1 is because example 3 by calling itself when late blocks the browser and results in the last rendered frame never being presented to the display hardware.Dietz
@Dietz I never called exampleThree either as it was just pseudo code. I have added the requestAnimationFrame calls. As for display rate and backbuffer, the question was more about what delta-time meant in the linked tutorials so that's what i focused on.Zimbabwe

© 2022 - 2024 — McMap. All rights reserved.