ARKit Place a SCNNode facing the camera

Question

I'm using ARKit to display 3D objects. I managed to place the nodes in the real world in front of the user (aka the camera). But I don't manage to make them to face the camera when I drop them.

let tap_point=CGPoint(x: x, y: y)
let results=arscn_view.hitTest(tap_point, types: .estimatedHorizontalPlane)
guard results.count>0 else{
    return
}
guard let r=results.first else{
    return
}

let hit_tf=SCNMatrix4(r.worldTransform)
let new_pos=SCNVector3Make(hit_tf.m41, hit_tf.m42+Float(0.2), hit_tf.m43)

guard let scene=SCNScene(named: file_name) else{
    return
}
guard let node=scene.rootNode.childNode(withName: "Mesh", recursively: true) else{
    return
}
node.position=new_pos
arscn_view.scene.rootNode.addChildNode(node)

The nodes are well positioned on the plane, in front of the camera. But they are all looking in the same direction. I guess I should rotate the SCNNode but I didn't manage to do this.

Buckles

Related: https://mcmap.net/q/824252/-apply-arcamera-rotation-transform-to-node-arkit/3151675 – Kumamoto

Answer 1

First, get the rotation matrix of the camera:

let rotate = simd_float4x4(SCNMatrix4MakeRotation(sceneView.session.currentFrame!.camera.eulerAngles.y, 0, 1, 0))

Then, combine the matrices:

let rotateTransform = simd_mul(r.worldTransform, rotate)

Lastly, apply a transform to your node, casting as SCNMatrix4:

node.transform = SCNMatrix4(rotateTransform)

Hope that helps

EDIT

here how you can create SCNMatrix4 from simd_float4x4

   let rotateTransform = simd_mul(r.worldTransform, rotate)

  node.transform =  SCNMatrix4(m11: rotateTransform.columns.0.x, m12: rotateTransform.columns.0.y, m13: rotateTransform.columns.0.z, m14: rotateTransform.columns.0.w, m21: rotateTransform.columns.1.x, m22: rotateTransform.columns.1.y, m23: rotateTransform.columns.1.z, m24: rotateTransform.columns.1.w, m31: rotateTransform.columns.2.x, m32: rotateTransform.columns.2.y, m33: rotateTransform.columns.2.z, m34: rotateTransform.columns.2.w, m41: rotateTransform.columns.3.x, m42: rotateTransform.columns.3.y, m43: rotateTransform.columns.3.z, m44: rotateTransform.columns.3.w)

Turrell

What is "r.worldTransform" referring to? I don't see what it could be. – Zorine

"r" is just the TC's variable for results.first, meaning the user's first touch point on the screen (I'm assuming within their touchesBegan method). The documentation explains it best, worldTransform is "The transformation matrix that defines the intersection’s rotation, translation and scale relative to the world." – Turrell

Ahh okay, i get what world transform is/does, just not where r came from, thanks! – Zorine

thank you!!! I swear yours is the only answer on stack overflow that worked for me! adding a node as child of camera would make the node follow it, with your way instead it is only initially oriented towards the camera but stays in place. in my case I would extract a SCNVector3 inside touchesBegan to use as position of my textNode and then apply the rotateTransform as you showed. Thanks Again, this should be marked as the answer @op – Leboeuf

No problem! Glad it helped you out. – Turrell

what does TC stand for? – Grandchild

It just means Topic Creator – Turrell

I am trying to do this in a method that gets called every second and after about 20 seconds it crashes. Any idea why is this happening? – Riverhead

@Riverhead Your situation is quite vague, may I ask why you require calling this method every second? If you want a node to continually face the user, I would look into SCNBillboardConstraint. – Turrell

I was calling it every second since I wanted the node to constantly face the camera. I think I was able to find an odd solution. I was calling a function that caused the issue. After I copied the code from the function and put it into the class itself, it started working. Not sure what is going on, but it works – Riverhead

You definitely want SCNBillboardConstraint then as it does exactly what you are looking for without needing to do something every second. – Turrell

Answer 2

guard let frame = self.sceneView.session.currentFrame else {
    return
}
node.eulerAngles.y = frame.camera.eulerAngles.y

Currajong

I was looking for this for the past couple of days. thank you. – Fouts

Answer 3

here's my code for the SCNNode facing the camera..hope help for someone

    let location = touches.first!.location(in: sceneView)

    var hitTestOptions = [SCNHitTestOption: Any]()
    hitTestOptions[SCNHitTestOption.boundingBoxOnly] = true

    let hitResultsFeaturePoints: [ARHitTestResult]  = sceneView.hitTest(location, types: .featurePoint)

    let hitTestResults = sceneView.hitTest(location)
    guard let node = hitTestResults.first?.node else {
        if let hit = hitResultsFeaturePoints.first {
            let rotate = simd_float4x4(SCNMatrix4MakeRotation(sceneView.session.currentFrame!.camera.eulerAngles.y, 0, 1, 0))
            let finalTransform = simd_mul(hit.worldTransform, rotate)
            sceneView.session.add(anchor: ARAnchor(transform: finalTransform))
        }
        return
    }

Donley

Not work for me :\ , The anchor isn't facing the camera in all angles.. – Ocana

Answer 4

Do you want the nodes to always face the camera, even as the camera moves? That's what SceneKit constraints are for. Either SCNLookAtConstraint or SCNBillboardConstraint can keep a node always pointing at the camera.

Do you want the node to face the camera when placed, but then hold still (so you can move the camera around and see the back of it)? There are a few ways to do that. Some involve fun math, but a simpler way to handle it might just be to design your 3D assets so that "front" is always in the positive Z-axis direction. Set a placed object's transform based on the camera transform, and its initial orientation will match the camera's.

Intolerant

The second :) I want the node to face the camera when I place it then keep it here (and be able to move around). – Buckles

I don't understand what you mean when you say > design your 3D assets so that "front" is always in the positive Z-axis direction. I have a dae file I converted to scn in xCode. The node position is (0, 0, 0). But I also placed the camera at (0, 0, 0.9) in order to have the object in front of me. Might be important to precise the camera is in the node hierarchy (a subnode I guess). – Buckles

Nodes are always right positioned in front of the camera when I drop them, they are visible (no behind the camera). If I move in the room and drop several nodes they are all kind of parallels. When I drop the first node, it is "looking at me". But not the next ones. I don't know if what I say is clear... :/ – Buckles

If you are bringing your DAE in from Blender you have to rotate it -90 degrees. – Cyndy

@MarieDm did you resolve this issue? I am also facing the same issue and couldn't find a concrete solution for this. – Vagabondage

Hi @Venkatesh, no I didn't... :( I have not been working on this since a few weeks ago and have not find any solution unfortunately. – Buckles

@Cyndy why do you have to rotate if the DAE comes from Blender? – Mccafferty

It's been a while since I did this but if I remember correctly the DAE from Blender comes oriented differently, so in order for it to appear correctly in the app you need to adjust it by 90 degrees. – Cyndy

Answer 5

Here's how I did it:

func faceCamera() {
    guard constraints?.isEmpty ?? true else {
        return
    }
    SCNTransaction.begin()
    SCNTransaction.animationDuration = 5
    SCNTransaction.completionBlock = { [weak self] in
        self?.constraints = []
    }
    constraints = [billboardConstraint]
    SCNTransaction.commit()
}

private lazy var billboardConstraint: SCNBillboardConstraint = {
    let constraint = SCNBillboardConstraint()
    constraint.freeAxes = [.Y]
    return constraint
}()

As stated earlier a SCNBillboardConstraint will make the node always look at the camera. I am animating it so the node doesn't just immediately snap into place, this is optional. In the SCNTransaction.completionBlock I remove the constraint, also optional.

Also I set the SCNBillboardConstraint's freeAxes, which customizes on what axis the node follows the camera, again optional.

Actinic

Answer 6

I want the node to face the camera when I place it then keep it here (and be able to move around). – Marie Dm Blockquote

You can put object facing to camera, using this:

if let rotate = sceneView.session.currentFrame?.camera.transform {
    node.simdTransform = rotate
}

This code will save you from gimbal lock and other troubles.

The four-component rotation vector specifies the direction of the rotation axis in the first three components and the angle of rotation (in radians) in the fourth. The default rotation is the zero vector, specifying no rotation. Rotation is applied relative to the node’s simdPivot property.

The simdRotation, simdEulerAngles, and simdOrientation properties all affect the rotational aspect of the node’s simdTransform property. Any change to one of these properties is reflected in the others.

https://developer.apple.com/documentation/scenekit/scnnode/2881845-simdrotation https://developer.apple.com/documentation/scenekit/scnnode/2881843-simdtransform

Knickknack

This fixed the gimbal lock for me, but I only needed the camera's orientation, not its size: node.simdOrientation = sceneView.session.currentFrame?.camera.transform.orientation – Hoot