ARKit: Finding the coordinates of a SCNNode on the screen

Question

I have a simple Swift ARKit setup where I have a SCNNode with a 3D object that is visible in an ARSCNView.

I want to determine the 2D coordinates of this object on the ARSCNView. By this I mean the x- and y-coordinates the object has when it is drawn onto the screen.

I have provided a sketch to illustrate what I mean:

Is there a way to get these coordinates, or at least an approximation? I need this in order to do some further processing with the camera frame. Basically I am interested in the area the object occupies on the screen.

Unholy

Did you have a look at the viewMatrix method of ARCamera? developer.apple.com/documentation/arkit/arcamera/… – Engineer

Isn't this what projectPoint does? – Hybridize

@Hybridize Yes, it is exactly what I was looking for. – Unholy

Answer 1

You can use SCNSceneRenderer projectPoint:point :

Projects a point from the 3D world coordinate system of the scene to the 2D pixel coordinate system of the renderer.

let node:SCNNode = // Your node
let nodeWorldPosition = node.position
let nodePositionOnScreen = renderer.projectPoint(nodeWorldPosition)
let x = nodePositionOnScreen.x
let y = nodePositionOnScreen.y

Note : A point projected from the near (resp. far) clip plane will have a z component of 0 (resp. 1).

---

The same method can be used with ARAnchor :

let anchor:ARAnchor = // ...
let anchorWorldPosition = SCNVector3(anchor.transform.columns.3)
let anchorPositionOnScreen = renderer.projectPoint(anchorWorldPosition)
let x = anchorPositionOnScreen.x
let y = anchorPositionOnScreen.y

Berte

Is there a way to make it work with ARImageAnchor ? – Hackney

ARImageAnchor is a subclass of ARAnchor so I think it should work the same, I did not test it though – Berte

Reference to member 'projectPoint' cannot be resolved without a contextual type – Cryptocrystalline

Note that ARSCNView conforms to SCNSceneRenderer protocol, so you can use projectPoint:point: method directly with your reference to ARSCNView. This was not readily obvious to me as someone new to SceneKit. – Ergocalciferol

this line: let nodeWorldPosition = node.position should be: let nodeWorldPosition = node.worldPosition – Jilt