Dynamically change text of RealityKit entity

Question

I have created a very simple scene ("SpeechScene") using Reality Composer, with a single speech callout object ("Speech Bubble") anchored to a Face anchor.

I have loaded this scene into code via the following:

let speechAnchor = try! Experience.loadSpeechScene()
arView.scene.anchors.append(speechAnchor)

let bubble = (arView.scene as? Experience.SpeechScene)?.speechBubble

It renders as expected. However, I would like to dynamically change the text of this existing entity.

I found a similar question here, but it's unclear to me how to refer to the meshResource property of a vanilla RealityKit.Entity object.

Is this possible? Thank you!

Orthohydrogen

I updated my answer. – Exponent

Answer 1

First Approach

At first you need to find out what's an hierarchy in Reality Composer's scene containing Bubble Speech object. For that I used simple print() command:

print(textAnchor.swift!.children[0].components.self)   /* Bubble Plate */

print(textAnchor.swift!.children[1].components.self)   /* Text Object */

Now I can extract a text entity object:

let textEntity: Entity = textAnchor.swift!.children[1].children[0].children[0]

And bubble plate entity object:

let bubbleEntity: Entity = textAnchor.swift!.children[0]

Here's a final code version that you can adapt for your needs:

import RealityKit

class GameViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        let textAnchor = try! SomeText.loadTextScene()
        
        let textEntity: Entity = textAnchor.swift!.children[1].children[0].children[0]

        textAnchor.swift!.parent!.scale = [4,4,4]  // Scale for both objects
        
        var textModelComp: ModelComponent = (textEntity.components[ModelComponent])!
                
        var material = SimpleMaterial()
        material.baseColor = .color(.red)
        textModelComp.materials[0] = material

        textModelComp.mesh = .generateText("Obj-C",
                            extrusionDepth: 0.01,
                                      font: .systemFont(ofSize: 0.08),
                            containerFrame: CGRect(),
                                 alignment: .left,
                             lineBreakMode: .byCharWrapping)

        textEntity.position = [-0.1,-0.05, 0.01]

        textAnchor.swift!.children[1].children[0].children[0].components.set(textModelComp)
        arView.scene.anchors.append(textAnchor)
    }
}

Second Approach

And you can always use a simpler approach for this case – to create several scenes in Reality Composer, each one must contain different speech-object.

Consider, this code isn't for tracking, it's just a test for dynamically switching two objects using Tap Gesture. Then you need to adapt this code for tracking faces.

import RealityKit

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    var counter = 0
    var bonjourObject: FaceExperience.Bonjour? = nil
    var holaObject: FaceExperience.Hola? = nil
     
    override func viewWillAppear(_ animated: Bool) {
        super.viewWillAppear(animated)
        
        // Reality Composer Scene named "Bonjour"
        // Model name – "french"
        bonjourObject = try! FaceExperience.loadBonjour()
        bonjourObject?.french?.scale = SIMD3(x: 2, y: 2, z: 2)
        bonjourObject?.french?.position.y = 0.25
        
        // Reality Composer Scene named "Hola"
        // Model name – "spanish"
        holaObject = try! FaceExperience.loadHola()
        holaObject?.spanish?.scale = SIMD3(x: 2, y: 2, z: 2)
        holaObject?.spanish?.position.z = 0.3
    }
    @IBAction func tapped(_ sender: UITapGestureRecognizer) {            
        if (counter % 2) == 0 {
            arView.scene.anchors.removeAll()
            arView.scene.anchors.append(holaObject!)
        } else {
            arView.scene.anchors.removeAll()
            arView.scene.anchors.append(bonjourObject!)
        }
        counter += 1
    }
}

If you want a text portion to be on the same place – just copy-paste object from one scene to another.

Exponent

If I wanted to change text like this but then show it in a QLPreviewController rather than an ARView, how would I do that? – Rauscher

Thanks Andy. Posted the question here: #75127237 – Rauscher

Ok added some basic code. – Rauscher

Answer 2

@maxxfrazer is correct in his assertion that currently the only way to change text dynamically is to replace the ModelComponentof the Entity assuming of course it adheres to the HasModel Protocol.

I have written a simple extension which can help with this:

//-------------------------
//MARK: - Entity Extensions
//-------------------------

extension Entity{

  /// Changes The Text Of An Entity
  /// - Parameters:
  ///   - content: String
  func setText(_ content: String){ self.components[ModelComponent] = self.generatedModelComponent(text: content) }

  /// Generates A Model Component With The Specified Text
  /// - Parameter text: String
  func generatedModelComponent(text: String) -> ModelComponent{

    let modelComponent: ModelComponent = ModelComponent(

      mesh: .generateText(text, extrusionDepth: TextElements().extrusionDepth, font: TextElements().font,
                          containerFrame: .zero, alignment: .center, lineBreakMode: .byTruncatingTail),

      materials: [SimpleMaterial(color: TextElements().colour, isMetallic: true)]

    )

    return modelComponent
  }

}

//--------------------
//MARK:- Text Elements
//--------------------

/// The Base Setup Of The MeshResource
struct TextElements{

  let initialText = "Cube"
  let extrusionDepth: Float = 0.01
  let font: MeshResource.Font = MeshResource.Font.systemFont(ofSize: 0.05, weight: .bold)
  let colour: UIColor = .white

}

In order to use it lets say you create an Entity called textEntity:

 var textEntity = Entity()

You can then set the dynamically change the Text via replacing the ModelComponent and setting the MeshResource at any time by simply calling the following method:

textEntity.setText("Stack Overflow")

Of course in regard to centering or aligning the text you will need to do some simple calculations (which I have ommited here).

Hope it helps.

Selfloading

Answer 3

Find your model entity (maybe by putting a breakpoint and looking through the children initially), find the Entity that conforms to the HasModel protocol, then replace its model with a different one using generatetext:

https://developer.apple.com/documentation/realitykit/meshresource/3244422-generatetext

Preset