Where are 'behaviours' created in Reality Composer stored in .rcproject?

Question

The situation

I am making an AR app in xcode (11.3.1). I have added objects (e.g. a cube) into the scene using reality composer and added behaviours (i.e. tap and flip and look at camera) to those objects, also using reality composer. Saved that, switched to ViewController.swift

In ViewController, I load in the Experience.rcproject and access the default Box scene by writing var box = try! Experience.loadBox(). All works as expected.

I am then printing the various objects in the hierachy to understand how they are constructed. So I will for example write print(box) and see all the entities and components as they are described here

The problem

I can see things like Transform for position etc. and ModelComponent for the mesh, materials etc. and this all makes sense but I cannot see where the behaviours are stored within the object that the .rcproject becomes within swift.

For example, if I have added a look at camera behaviour in reality composer, my assumption would be that there would be something like a `billboard' component attached to that object, but I don't see any difference between objects that have behaviours applied, and those that don't..

Another example would be, having addded tap and flip to an object, I would expect to find some animation information somewhere within the object, but again I can't see it attached to the object. Nor can I see any animation info, or behaviour components any where within the scene object.

Does anywhere know where I might be able to access these? There seems to be something under box called actions but printing that simply returns Experience.Box.Actions with no more info.

Am I looking in the wrong place? Or are these not exposed?

Americana

Answer 1

Partial Solution

You can access and recreate behaviours in a certain, quite limited way. For that you need to include Notify action with definite name in action's sequence in Reality Composer.

Here's how a code for RealityKit looks like:

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()

        let boxAnchor = try! Experience.loadBox()
        arView.scene.anchors.append(boxAnchor)

        boxAnchor.actions.repetitive.onAction = self.remodelling
    }
    
    fileprivate func remodelling(_ entity: Entity?) {

        guard let entity = entity else { return }
        entity.children[0].position.x = -0.25
        
        let modelEntity = entity.children[0] as! ModelEntity
        modelEntity.model?.mesh = .generateSphere(radius: 0.1)
        modelEntity.model?.materials = [UnlitMaterial(color: .green)]
    }
}

Where actions is a computed property:

public private(set) lazy var actions: Actions { get set }

repetitive is an object of NotifyAction class:

public private(set) lazy var repetitive: Experience.NotifyAction { get set }

onAction is a completion handler:

public var onAction: ((Entity?) -> Void)?

Additional Info

Here's non-editable content for ActionsAndBehaviours.Experience.Box.Actions:

import Combine
import Foundation
import RealityKit
import SwiftOnoneSupport
import UIKit
import simd

@available(iOS 13.0, macOS 10.15, *) public enum Experience {

    public class NotifyAction {    
        public let identifier: String    
        public var onAction: ((Entity?) -> Void)?
    }   
    public enum LoadRealityFileError : Error {    
        case fileNotFound(String)
    }

    public static func loadBox() throws -> ActionsAndBehaviours.Experience.Box   
    public static func loadBoxAsync(completion: @escaping (Result<ActionsAndBehaviours.Experience.Box, Error>) -> Void)

    public class Box : Entity, HasAnchoring {    
        public var steelBox: Entity? { get }    
        lazy public private(set) var actions: ActionsAndBehaviours.Experience.Box.Actions

        public class Actions {    
            lazy public private(set) var repetitive: ActionsAndBehaviours.Experience.NotifyAction   
            lazy public private(set) var allActions: [ActionsAndBehaviours.Experience.NotifyAction]
        }
    }
}

Rinehart

Hi AndyFedo, thanks for your response but printing the components does not show any of the ‘behaviours’ that have been added in Reality Composer (E.g. animations or look at camera) perhaps they are not components? But if not, what are they and how can they be accessed within the scene object? – Americana

Thanks for the info AndyFedo. I look forward to a future version. And thanks for the snippet, that's helpful. – Americana

stackoverflow.com/users/6599590/andy-jazz I am sorry, but I have missed something very important in this code? Where does this public enum experience come into the picture? I don't find it in my project and I have included all the libraries you name here? – Hypersensitive

@user3069232, this public enum is a non-editable content of Experience.rcproject file (Reality Composer's content). – Rinehart

Command-click Experience word in your code and choose Jump to Definition from context menu. You'll see an autogenerated RC code. – Rinehart

@Andy Jazz I want scream... I type in the "keyword/enum" Experience and it is not found! It does not exist in my project? which include all the libraries ARKit, SceneKit, RealityKit, UIKit, SwiftUI, Combine....I google it, I find nothing, it makes no sense – Hypersensitive

Create a new RealityKit game template in Xcode. Experience.rcproject file is already in left pane. ( do you see a line containing try! Experience.loadBox() inside viewDidLoad() method ?) Command-click Experience word. – Rinehart