Accessing GPU via web browser

Question

I came across this proof of concept earlier today (on TechCrunch.com) and was blown away and intrigued as to how they had managed to accomplish the end result. They state that they don't use webGL or any plugins yet they are able to interact directly with the GPU and render 3D visuals at up to 60 fps using nothing but Javascript. Any ideas how this could be done, or how to access the GPU from Javascript in general without the use of plugins?

Site Address is: famo.us

ps: Try using your arrow keys to shift orientation its far out!!

Loquacity

Answer 1

They're using standard HTML5 Javascript APIs to achieve this.

I saw several references to requestAnimationFrame in their code. This method allows one that uses a compatible browser, to display more fluid and optimized animations, with far better frame rates than setInterval based animations will ever allow you to have. This is certainly achieved by using the GPU and the available hardware background.

GPU or every other hardware component cannot be accessed directly using any Javascript routines. Instead, the browser, based on the called JS directives and the execution context, will use the GPU if possible to optimize some specific treatments, calculus and renderings.

EDIT

For future references, I recently found out (7 years after the original answer) that a new initiative - the W3C GPU for the Web Community Group, has recently been created in 2020 to do just that. It describes itself as follow.

The mission of the GPU on the Web Community Group is to provide an interface between the Web Platform and modern 3D graphics and computation capabilities present in native system platforms. The goal is to design a new Web API that exposes these modern technologies in a performant, powerful and safe manner. It should work with existing platform APIs such as Direct3D 12 from Microsoft, Metal from Apple, and Vulkan from the Khronos Group. This API will also expose the generic computational facilities available in today's GPUs to the Web, and investigate shader languages to produce a cross-platform solution.

In the long-term, this initiative might allow developers to directly interact with the GPU from all web browsers. You can track the implementation status of the WebGPU API Spec on Github here.

Galateah

Thanks, so they are just using standard html5 functions? I'm slightly less impressed now, they make out like they are directly accessing GPU hardware via Javascript. Naturally, I began thinking about all the cool possibilities this would present if this was true. Aahh well. – Loquacity

They can say that they're using the GPU because this is somehow true. Nevertheless, they are not achieving this themselves, at the end, the only application that will choose whether to use the GPU or any sort of available hardware interaction is the browser itself. – Galateah

they actually say that they use a wrapper for each platform that they run on. so instead of using css3, they use requestAnimationFrame and normal html elements, but the 3D calculations are done in GPU. Think of it as phonegap for GPU :) if you will. They mention that on xbox for example the wrapper is in DirectX, while on iOS in OpenGL. – Gatehouse

wheather that makes sense, from a tech and business point of view, remains to be seen ... i personally think is a waist of good talent :) – Gatehouse

Yes, everything is done using requestAnimationFrame and CSS3 transforms. In the future, when Safari enables WebGL per default, they aim to render using WebGL instead of using inline CSS3 transforms. This should increase number of smoothly rendered surfaces from around 500 to much much more. They try to flatten out the DOM tree and transfer all rendering and layout to Javascript (using transforms) to avoid slow DOM-layout and repaint processes. – Postboy

Answer 2

Concerning famo.us: they did analysed the bottlenecks of the Webkit rendering pipe and then found a way to bypass them while building the pages. Basically: the DOM tree construction, the Render tree construction, the Layout of Render Tree are bypassed. Take a look on this article for a whole explanation.

Anticlimax

Answer 3

They're using CSS 3D transforms. Web browsers are increasingly using hardware acceleration to do these kinds of CSS3 things.

Dulcia

Answer 4

i think the webGL glsl.js library might be good for this, though i havnt seen benchmarks... https://github.com/gre/glsl.js/

also this approach seems viable;

basically to use the gpu in the way we like to, hardware optimised functions (have a little look into "blas") are used, you do not want to write these! strangely it seems that people still use the old fortran blas.... there is some stuff with compiling via llvm and then using emscripten to turn it into javascript. Use Emscripten with Fortran: LAPACK binding

the emscripten way seems the most versatile, im just about to check it out, but it looks like a mountain, also this topic seems to be somewhat of a call to arms, emscripten only works with fortran if you hack it (see links from the second link). i think what we are looking for is blas support in javascript, this is not a closed issue by any means, and for a few of us out here it is very frustrating! hopefully someone has a link to these blas libraries we cant find... please let me know if i dont quite have my head round this issue, js is new to me.

also, suggesting html5 is sufficiently "optimised" to use the hardware, how can we trust this!? we need gpu optimised blas (basic linear algebra subprograms) e.g. dot product.

i think maybe also that these old fortran blas arnt actually the right thing for a modern gpu? node-cuda seems very likely the best thing i have found...

Poler