Using Web Workers for drawing using native canvas functions

Question

It's possible to send a CanvasPixelArray obtained via getImageData to a worker script, and let the worker script manipulate the pixels in its background thread, and eventually post the manipulated pixel array back.

However, I'm using native canvas drawing functions, like drawImage. The drawImage calls are currently blocking the UI thread. This causes slow redraw of buttons, and a noticeable delay when clicking on a button, just to name a few drawbacks. (Edit: A small improvement can now be accomplished with ctx.imageSmoothingEnabled = false, at least on WebKit with the webkit prefix.)

I'd like to move the drawing from the main thread into a background thread using Web Workers. However, I don't seem to be able to send the canvas nor the context to the worker.

I did find this notice on MDN:

Note: As usual, background threads—including workers—cannot manipulate the DOM. If actions taken by the background thread need to result in changes to the DOM, they should post messages back to their creators to do that work.

But I would like to leave the DOM as-is; I simply want to draw things on a canvas element. Is this possible, or are Web Workers really only allowed to calculate and not to draw?

(Or is it perhaps possible to use functions like drawImage to manipulate a CanvasPixelArray instead of drawing on a canvas?)

Knoll

Answer 1

[community edit: This answer was written and accepted in 2011. Other technologies have emerged (or are emerging) which may allow Web Workers and Canvas to co-exist better; the reader should be aware of all the answers on this page besides this answer.]

You cannot pass a canvas object or canvas context to a worker thread because the canvas is part of the DOM.

Parenteral

That's a pity. So is it correct I'm basically forced to use the main thread for drawing? Is there indeed no way to draw to a CanvasPixelArray? – Knoll

Yep. Even if you had an equivalent API to drawImage() that operated on a CanvasPixelArray, you'd still need to post that data back to the UI thread for actual drawing. I've heard talk of an offscreen Canvas for use in these situations but it's something you'll have to wait on browser vendors to implement. – Parenteral

Wouldn't it be awesome if someone implemented the canvas API in pure javascript so we could use it in web workers – Internalize

You can JSON.stringify the CanvasPixelArray, manipulate it in the worker, and when you get your result back, parse the json, and use putImageData to put the result on the canvas – Provencal

There's a new API to do this (only supported in Firefox if you enable a pref). See developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas and hacks.mozilla.org/2016/01/webgl-off-the-main-thread. – Gestapo

@JohnWatson, Re "still need to post that data back to the UI thread for actual drawing"; But w just that, it wouldn't lag the UI isn't it? – Selfridge

@DrewLeSueur, Meaning? – Selfridge

@bigblind, stringify is pretty retarded considering you can just post the whole thing to a webworker. – Selfridge

Answer 2

[Edit ~5 years later: some of this is beginning to change, and there are new web platform features that actually allow rendering to a canvas from a Worker! See this blog for more info: https://hacks.mozilla.org/2016/01/webgl-off-the-main-thread/ - the rest of the answer is provided for 2011-era information ;)]

Web workers can only calculate, not modify the DOM or make any calls to draw to a canvas. However like you say you can post an array of pixels to a web worker to process it there and post it back. Since that's asynchronous I don't see why that would cause any slowdown in the UI thread, unless you deliberately block until the web worker responds (which you shouldn't).

So it seems odd that your drawImage calls are taking so long it's affecting the UI. Most canvases are hardware-accelerated these days so they should skip along nicely. My guess is that you're drawing via a web worker to a canvas pixel array every frame, which effectively means you're software rendering the canvas in javascript. Javascript is still far too slow to do this - even C++ software renderers are kind of slow, which is why hardware acceleration is important. So you can render something to a canvas pixel array in a web worker once, and then when you get your result cache it in to an Image once, and then draw that Image to the canvas as much as you like. That should still be really fast.

Edit: you might want to look in to WebGL, where you can write fragment shaders which are effectively little programs to process pixel effects. They run entirely on the graphics card so they're stupidly fast.

Dichromatism

In fact I haven't implemented Web Workers yet, that's why it's currently blocking. WebGL seems interesting; it looks like a whole new world of processing and drawing. Thanks for your suggesions. (Hardware acceleration is for some reason much slower on my computer on Chrome than software rendering, so enabling acceleration is not a possibility (yet)). – Knoll

Answer 3

You can post the ImageData to the Web Worker, which sends the manipulated ImageData back to the caller (Main UI) thread.

E.g.:

1. Create a Web Worker:

   this.renderer = new Worker("renderer.js");

2. Post the ImageData created from the canvas to the Web Worker:

   var ctx = this.canvas.getContext('2d');
   var imageData = ctx.createImageData(width, height);
   this.renderer.postMessage({ image: imageData });

3. Do ImageData manipulation in Web Worker and post it back to Main thread:

   onmessage = function(e) {
      var processedImage = self.doImageProcessing(e.data.image);
      postMessage({ image: processedImage });
   };

4. Set the manipulated ImageData to the canvas in the Main thread:

   this.renderer.onmessage = function (e) {
      var ctx = this.canvas.getContext('2d');
      ctx.putImageData(e.data.image, 0, 0);
   }

Impious

Now if only someone makes it possible to create a CanvasRenderingContext2D to operate on the imageData on the web worker – Moonlighting

@hvdd, Did you try it with [imageData] in the second argument of postMessage? – Selfridge

@Selfridge I've tried to transfer ImageData in chrome v79 and got: "TypeError: Failed to execute 'postMessage' on 'Worker': Value at index 0 does not have a transferable type." – Decillion

the solution is to pass underlying buffer of ImageData: worker.postMessage(imageData.data.buffer, [imageData.data.buffer]) – Decillion

Answer 4

There's a new API to do this (currently only supported in Firefox if you enable a pref).

See https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas and https://hacks.mozilla.org/2016/01/webgl-off-the-main-thread/.

Gestapo

Is this supposed to be a faster version of Osi's answer? Or is the performance similar? – Selfridge

Yes, it should be faster. – Gestapo