How can I tell when the browser stops repainting DOM layers/nodes because they are obscured?

Question

Identification: I'm looking at all possible solutions for detecting when the FPS drop or the repaints/reflows stop happening in a browser due to the browser optimizing performance because an area of the document is out of view.

Observation: So far most native HTML/JS solutions I have researched do not take into account the portion of the DOM, which has been obscured by other pieces of the page, or DOM which is below the fold. While sometimes content will be directly embedded in the parent document, other times it may be in a frame or iframe.

Test results: mozPaintCount is an accurate representation of the dom reflows for a window, and responds correctly when a window or a document in a frame is obscured. It also appears that Firefox is correctly identifying parts of the dom and not repainting them when they are obscured which can be visually checked through the firefox inspector tool when something is out of view, however the inspector may not take into account the view hierarchy when drawing the highlights on the repainted regions. Kudos to you, Mozilla!

Reference: mozPaintCount

Flash also shows a slow down in FPS when it is off screen and the newest Flash Player 11.2 has a dedicated event which fires when it has been throttled by the browser for optimization reasons. Flash is also not a trustworthy solution because it is disabled by default in a number of browsers.

Reference: Flash Player 11.2 Throttle Events

The current browser implementation of the page visibility change event is capable of broadcasting when the window has been changed to another tab or the window becomes inactive, but does not take into account scroll position for elements/windows/documents below the fold or obscured by other dom. In the second draft of this spec however, it has been proposed the browser take into account the viewport and DOM hierarchy to give more accurate results.

References: Recommendation Page Visibility, 2011 and Draft Page Visibility 2, 2013

CSS transitions do not appear to be optimized away when their elements are obscured, and so they cannot be measured correctly. I tried a number of different properties which cause reflows (height, width, box shadow, etc) and was not able to gain any meaningful metric from these.

requestAnimationFrame appears to throttle animations only when the tab is changed and the browser window becomes blurred, but does not handle content below the fold. Also, because raf is only attached at the window level, it is unable to be tied to specific dom nodes (which would seem to be shortsighted in its implementation).

Reference: requestAnimationFrame

I have not finished testing WebGL, however that will be limited to a small number of modern browsers (an acceptable result given flash solutions for older browsers).

I have not finished testing canvas, but this could be a good solution as it is supported by a wider array of browsers than WebGL.

I have not even begun thinking about embedded objects and what they could provide.

I have not looked into ActiveX controls to see if there is anything there which may help.

I have not looked into HTML5 video.

Animated GIFs appear to only be optimized away in Firefox. Kudos again, Mozilla!

So, the mystery and magic of how each browser vendor optimizes page rendering continues to stifle my progress. However, element position alone cannot be relied upon for a true indication of whether an element is actually viewable or not as the page may have other layers which obscure framed content, or the content may be displayed in a way which precludes any meaningfully visible rendering (such as changing the 3d rotation of content so that it is not visible.)

Any brainstorming on how the browsers actually optimize displayed content or additional directions or actual answers to this question are welcomed.

Similarly, here is an older SO question which partially encompasses this question as well.

Semiporcelain

Interesting observations! So to rephrase your question/problem, you want to have a requestAnimationFrame that is bound to a specific region of the page and is throttled when that region is not in the view? I don't think there's a good answer for your phrasing "how can I detect internal browser behaviour". – Tachometer

Is the goal to detect when the node is no longer visible to prevent some expensive or repeated task? For example, if a chat window is no longer visible, you could stop getting new messages. – Grogshop

That is correct, Justin. The goal is to optimize the response of the page and allow the javascript to react accordingly. In existing solutions, there's no true way to know if a loaded component is being viewed or is in view other than to check the repaint count or framerate of a plugin near that component. Because browsers haven't exposed where they're doing optimizations, the presumed answer seems to be "you can't" which would seem incorrect. – Semiporcelain

Additionally, I'd also like to know which pieces are optimized and which are not in different browsers to test performance aspects of different elements. A good example is the animated gifs which will constantly redraw areas in all browsers other than Firefox. – Semiporcelain

Answer 1

I used the Mozilla mozPaintCount in the following code, but I am not get the FPS value that I would expect.

Part 1: run this code once to populate the variables.

var currentFrames = window.mozPaintCount, currentTime = new Date();

Part 2: run this code anytime after Part 1.

(window.mozPaintCount - currentFrames)/((new Date() - currentTime)/1000)

The result shows FPS, for my site, I'm getting 9FPS. Chrome FPS counter shows me 60FPS.

Overall, browser's Page Rendering Performance is a relatively new topic of interest in the Web Dev World. Besides Chrome DevTools, every other major vendor does not seems to have a large interest in exposing tools for performance. Since the interest in Web Page Rendering performance is limited, so is a lot of data.

The big performance ideas that a developers are focusing are. Scroll Performance, CSS effect on Paint Time, Animated Gifs as you mentioned, and of course requestAnimationFrame. http://www.html5rocks.com/en/tutorials/speed/unnecessary-paints/

Really,before jumping into this field head first I recommend that you spend a little time actually understanding what is it that a browser does to render your page and updates to that page. http://www.html5rocks.com/en/tutorials/internals/howbrowserswork/

I would add more links for you, but my SO rep is too low.

Yap

Answer 2

Regarding what you wrote about requestAnimationFrame:

Also, because raf is only attached at the window level, it is unable to be tied to specific dom nodes

You can (in theory) have the code running in an iframe, therefore the window object is the whole element, or a 1px iframe could be a point on your content.

I have got this working in Firefox by looking at mozPaintCount (as you said and as @alexK85 suggests in their answer) to see if there has been a repaint after CSS change (browser does not do repaint immediately after CSS change when iframe out of viewport). But, I have not yet been able to do a similar thing with requestAnimationFrame in other browsers to establish if there has been a repaint of the iframe immediately after CSS change.

Does anyone have any suggestions about how raf could be used to establish if the browser does a repaint?

Lefton

mozPaintCount support has been the only way to capture repaints as all other browsers measure optimization based on the top.window, and subsequently don't throttle based on the position within the viewport. For other browsers, where supported you'd use Flash to gather this stat and report it. In old IE you'd use mouse events to triagulate positions in the page as they leak information by design in 8-10. 8 does not give you window size however, so you're left with just 9&10 there. – Semiporcelain

@JeffreyGilbert OK thanks for adding that. Many organisations claim to be able to do it cross-browser so they must rely on Flash for browsers other than FF. This paper is very useful pdrf.net/wp-content/uploads/2013/11/38deJager.pdf (see the table on page 6) but the wording as to Flash dependency is ambiguous. – Lefton

To disambiguate, Flash sometimes works and sometimes doesn't. It requires Flash to be installed to read throttling information, but Flash being installed or even being a particular version does not simply enable it to work. This support changes based on the environment Flash lives in and necessitates the use of alternative measurement techniques when checked in, for example, Firefox, where you have a pain count or geometry api available to compensate. – Semiporcelain

Other browsers, such as Internet Explorer, sometimes have quirks you can take advantage of. I've only been able to check geometry based on assumption of window chrome in IE 9&10, leaving IE behind as it would skew results if i were to simply attempt a capture of the ad as "on the screen" vs "in the viewport" – Semiporcelain