JavaScript: getting ImageData without canvas

Question

Is it possible to get an ImageData Object from an image which is not on the canvas but somewhere else in the DOM tree as a normal <img> ?

If yes, how?

Prudish

See also: Get image data url in JavaScript? – Insistence

Answer 1

You have to create an in memory canvas and then draw on this canvas the image :

var canvas = document.createElement('canvas');
var context = canvas.getContext('2d');
var img = document.getElementById('myimg');
canvas.width = img.width;
canvas.height = img.height;
context.drawImage(img, 0, 0 );
var myData = context.getImageData(0, 0, img.width, img.height);

But this won't work if the image comes from another domain. This is a security restriction you can't get around if you have no control of the server (be careful that if you open your html file with file:// you'll have a lot of additional restrictions, use http://)

Moschatel

Another point to be careful of: pixel values rendered with drawImage may be different from values in your image because of color space correction. Good thing that this only happens if image contains color space information. – Jollanta

@Jollanta Yes, I encountered that problem, it's worth noting as it can be very painful to debug. – Acus

The title explicitly says "without canvas" regardless of the canvas is been exists onn DOM tree or not. This is why i gave -1 to this answer. – Runoff

@Runoff My answer is helpful and solves the real problem. You should try being helpful too. – Acus

so, you can't upload an image in an IMG tag to a server? that's a use case that canvas is useless for because (1) image likely is from a random domain and (2) rendering the image necessitates a lossy re-encoding at some point – Requiem

You should probably use img.naturalWidth and img.naturalHeight to get the dimensions of the source data instead of the dimensions of the rendered DOM element. – Hyphen

@Runoff On the other hand, the question has the tag "canvas," and it mentions "an image which is not on the canvas," implying there is a canvas somewhere. – Mancuso

This "security restriction" you speak of, is that maybe circumventable by setting crossOrigin = "Anonymous"? (I'm doing that somewhere, and it seems to work.) – Epiphysis

@Epiphysis of course, that's why I precised "if you have no control of the server". CORS headers must be set on the resource server. – Acus

@DenysSéguret Ah. What got me confused is that I'm doing that on a server I have no control over (ipfs.io), but they do have that set. A few image sharing sites do, too, it seems. Ty. – Epiphysis

Answer 2

As already implied, canvas offers the only solution for creating ImageData objects.

If you are really set against using the canvas element to load image data (perhaps we are talking lte IE8), you could always consider parsing the base64 image data using the src location of an image object

http://blog.calyptus.eu/seb/2009/05/png-parser-in-javascript/

It's difficult, but if you must, could potentially parse images to an array this way.

https://github.com/devongovett/png.js/blob/master/png.js

This shows you how to load the data with an xhr request and parse the png data. Internally it uses the canvas for some other things but the subset you are interested in is canvas free. You would need to follow a similar implementation for each image format you are interested in.

I should mention that the image pixel reading limitations are identical in terms of security. You will never be able to read pixels that have come from a third party, with or without canvas. The XMLHTTPRequest is going to be bound to the governance of cross-domain policies. This prevents scripts from stealing third party content, which includes images that may contain sensitive user information.

If you need to read images on a third party domain (that don't require authentication to view), you should run an image proxy server on your domain which allows you to request images on the same domain. If you need to go to the trouble of that, it might be easier to simply provide the image data as a json array in the first place.

Junoesque

authentication and browser sniffing are two big reasons that server loading of the URL aren't feasible – Requiem

Arg. Sadly the first link has died - the post loads, but the actual demo code never made it through a site transition, and the web archive never got it. – Princeling

@Princeling Look at the github repository github.com/calyptus/labs/blob/master/JSBin/Demo/Viewer.html – Glyceric

Answer 3

If you are using a webworker you can use OffscreenCanvas as an alternative for document.createElement('canvas')

  var response = await fetch(imageUrl);
  var fileBlob = await response.blob();
  var bitmap = await createImageBitmap(fileBlob);
  var canvas = new OffscreenCanvas(bitmap.width, bitmap.height);
  var context = canvas.getContext('2d');
  context.drawImage(bitmap, 0, 0);
  var myData = context.getImageData(0, 0, bitmap.width, bitmap.height);

Note that support for OffscreenCanvas is limited: https://caniuse.com/#feat=offscreencanvas

Bedplate

Answer 4

You must manually decode the data from the file. You'll need the source URL i.e. from <img src="..."> to get the raw file and then use the right decoder to get the RGBA data.

https://github.com/photopea/UPNG.js - PNG encoder/decoder - needs Pako https://github.com/nodeca/pako - I'm using it for both encoding and decoding to bypass Canvas as some browsers add anti-fingerprinting which ruins heightmaps which require exact values.

fetch(url)
   .then(response => response.arrayBuffer())
   .then((buffer) => {
       const pngImg = UPNG.decode(buffer)
       const rgbaArr = new Uint8ClampedArray(UPNG.toRGBA8(pngImg)[0])
       const imageData = new ImageData( rgbaArr, pngImg.width, pngImg.height );
    })

https://github.com/jpeg-js/jpeg-js - JPEG encoder/decoder - haven't tested

For webp I built my own libwebp decoder with emscripten a few years ago but maybe there's a bespoke JS implementation these days. Every format has a JS decoder somewhere on Github and decoding will work(more or less) as the example code I posted.

Normy