Raycast in Three.js with only a projection matrix

Question

I'm rendering some custom layers using Three.js in a Mapbox GL JS page following this example. I'd like to add raycasting to determine which object a user has clicked on.

The issue is that I only get a projection matrix from Mapbox, which I use to render the scene:

class CustomLayer {
  type = 'custom';
  renderingMode = '3d';

  onAdd(map, gl) {
    this.map = map;
    this.camera = new THREE.Camera();
    this.renderer = new THREE.WebGLRenderer({
      canvas: map.getCanvas(),
      context: gl,
      antialias: true,
    });
    this.scene = new THREE.Scene();
    // ...
  }

  render(gl, matrix) {
    this.camera.projectionMatrix = new THREE.Matrix4()
      .fromArray(matrix)
      .multiply(this.cameraTransform);
    this.renderer.state.reset();
    this.renderer.render(this.scene, this.camera);
  }
}

This renders just great, and tracks changes in view when I pan/rotate/zoom the map.

Unfortunately, when I try to add raycasting I get an error:

  raycast(point) {
    var mouse = new THREE.Vector2();
    mouse.x = ( point.x / this.map.transform.width ) * 2 - 1;
    mouse.y = 1 - ( point.y / this.map.transform.height ) * 2;
    const raycaster = new THREE.Raycaster();
    raycaster.setFromCamera(mouse, this.camera);
    console.log(raycaster.intersectObjects(this.scene.children, true));
  }

This gives me an exception:

THREE.Raycaster: Unsupported camera type.

I can change from a generic THREE.Camera to a THREE.PerspectiveCamera without affecting the rendering of the scene:

this.camera = new THREE.PerspectiveCamera(28, window.innerWidth / window.innerHeight, 0.1, 1e6);

This fixes the exception but also doesn't result in any objects being logged. Digging a bit reveals that the camera's projectionMatrixInverse is all NaNs, which we can fix by calculating it:

  raycast(point) {
    var mouse = new THREE.Vector2();
    mouse.x = ( point.x / this.map.transform.width ) * 2 - 1;
    mouse.y = 1 - ( point.y / this.map.transform.height ) * 2;
    this.camera.projectionMatrixInverse.getInverse(this.camera.projectionMatrix);  // <--
    const raycaster = new THREE.Raycaster();
    raycaster.setFromCamera(mouse, this.camera);
    console.log(raycaster.intersectObjects(this.scene.children, true));
  }

Now I get two intersections wherever I click, with two faces of the cube. Their distances are 0:

[
  { distance: 0, faceIndex: 10, point: Vector3 { x: 0, y: 0, z: 0 }, uv: Vector2 {x: 0.5, y: 0.5}, ... },
  { distance: 0, faceIndex: 11, point: Vector3 { x: 0, y: 0, z: 0 }, uv: Vector2 {x: 0.5, y: 0.5}, ... },
]

So clearly something isn't working here. Looking at the code for setCamera, it involves both projectionMatrix and matrixWorld. Is there a way I can set matrixWorld, or construct the raycaster's ray directly using only the projection matrix? It seems that I only need the projection matrix to render the scene, so I'd hope that it would also be all I need to cast a ray.

Full example in this codepen.

Mameluke

"Is there a way I can set matrixWorld" Have you tried updateMatrixWorld()? All Cameras are also Object3Ds... – Related

@Related I tried adding this.camera.updateMatrixWorld(true); both instead of and in addition to the this.camera.projectionMatrixInverse line in the codepen but to no avail. Same behavior. – Mameluke

@Related specifically, this.camera.matrixWorld is the identity matrix both before & after calling updateMatrixWorld. – Mameluke

Also some interesting material on this mapbox-gl issue github.com/mapbox/mapbox-gl-js/issues/7395 – Mameluke

@Mameluke did you got your solution? Can you have a look a similar issue of mine #61656554 – Marten

@abhishekranjan the answer below is the solution incl. a fully functional JS Fiddle. If you like it, please upvote. I will have a look at your issue in the evening as well. – Viborg

@Viborg thanks, its a similar problem, but not all the same. When I click at the bottom of my 3d-object the array stays empty but on specific points (random) on 3d-object the click gives a non-empty array. Also when I click close to the 3d-object but not on the object itself then also this above mentioned uneven click detection takes place. I would seriously appreciate any kind of help. thank you. – Marten

Answer 1

Difficult to debug, but solved!

TL;DR: Full code in this working Fiddle.

1. I think that not having the world matrix of the camera from MapBox is not the main problem, rather the incompatibility of the coordinate spaces. Mapbox delivers a left-handed system with z pointing up. Three uses a right-handed system with y pointing up.

2. During the debugging I created a reduced copy of the raycaster setup functions to have everything under control and it paid off.

3. A cube is not the best object for debugging, It is way too symmetric. The best are asymmetric primitives or compound objects. I prefer a 3D coordinate cross to see the axes orientation straight away.

Coordinate systems

• There is no need for changing the DefaultUp in the BoxCustomLayer constructor, since the goals is to have everything aligned with the default coordinates in Three.
• You flip the y-axis in the onAdd() method, but then you also scale all objects when initializing the group. This way, the coordinates you operate in after inverting the projection in raycast() are not in meters anymore. So let's join that scaling part with this.cameraTransform. The left-handed to right-handed conversion should be done here as well. I decided to flip the z-axis and rotate 90deg along x-axis to get a right-handed system with y pointing up:

const centerLngLat = map.getCenter();
this.center = MercatorCoordinate.fromLngLat(centerLngLat, 0);
const {x, y, z} = this.center;

const s = this.center.meterInMercatorCoordinateUnits();

const scale = new THREE.Matrix4().makeScale(s, s, -s);
const rotation = new THREE.Matrix4().multiplyMatrices(
        new THREE.Matrix4().makeRotationX(-0.5 * Math.PI),
        new THREE.Matrix4().makeRotationY(Math.PI)); //optional Y rotation

this.cameraTransform = new THREE.Matrix4()
        .multiplyMatrices(scale, rotation)
        .setPosition(x, y, z);

3. Make sure to remove the scaling from the group in makeScene , in fact you don't need the group anymore.

4. No need to touch the render function.

Raycasting

Here it gets a bit tricky, basically it's what Raycaster would do, but I left out some unnecessary function calls, e.g. the camera world matrix is identity => no need to multiply with it.

1. Get the inverse of the projection matrix. It does not update when Object3D.projectionMatrix is assigned, so let's compute it manually.
2. Use it to get the camera and mouse position in 3D. This is equivalent to Vector3.unproject.
3. viewDirection is simply the normalized vector from the cameraPosition to the mousePosition.

const camInverseProjection = 
        new THREE.Matrix4().getInverse(this.camera.projectionMatrix);
const cameraPosition =
        new THREE.Vector3().applyMatrix4(camInverseProjection);
const mousePosition =
        new THREE.Vector3(mouse.x, mouse.y, 1)
        .applyMatrix4(camInverseProjection);
const viewDirection = mousePosition.clone()
        .sub(cameraPosition).normalize();

4. Setup the raycaster ray

this.raycaster.set(cameraPosition, viewDirection);

5. The rest can stay the same.

Full code

Fiddle for demonstration.

• I added mousemove to display real-time debug info with jQuery.
• The full info about the hit gets logged to the console upon click.

Viborg

Amazing, thanks @isolin! The incompatible coordinate systems are truly annoying. I believe the DefaultUp change came about from issues with materials and lights that I removed for this fiddle. (I had to flip textures inside out to get them to display.) I'll see if I can reproduce that. But regardless, thanks so much for your solution. "Difficult to debug" indeed! – Mameluke

@Viborg thanks for the answer! However, I haven't fully understood how it works, could you explain? The Raycaster.setFromCamera method for PerspectiveCamera works like this: - it picks origin as a position of the camera (position component of matrixWorld of the camera) - for direction, it unprojects passed mouse coordinates to 3d space using camera's projectionMatrix (and matrixWorld). Since matrixWorld is identity in our code the only difference with your answer is that you unproject origin point as well. I can't understand why this is needed. – Audacity

Just made a "2nd part" for this one here, to discuss about how to implement this for georeferenced stuf... – Biofeedback