How to make grouped objects in one Reality Composer scene draggable?

Question

So I have a Reality Composer file "Experience" loaded onto an XCode project with three scenes.

I have added behaviors and actions that work in XCode. The actions work perfectly, hiding and showing the 2 other scenes when I click on the different objects.

But on one scene, named "cubes", I added two cubes. I grouped these two cubes as "twoCubes" whereby, according to Apple's documentation, they will now behave as a single object:

You can combine multiple assets in a scene into a group. Grouped objects behave as a single combined object in Reality Composer’s scene view.

There are no associated actions on this scene, I only wanted to make the two cubes draggable and scalable when I pinch. I haven't been able to make that happen. Even when I name each one of the cubes differently, nothing happens.

I suspect it has something to do with what happens when objects are grouped [reference from Apple documentation here ] but I haven't been able to figure out a solution. I've looked at a lot of answers from resident genius Andy Jazz who is the expert on this subject, but nothing... I am doing this because I have a much larger model I am working on my own application, that has about 20 children per scene... so I am hoping I can figure out a solution here that I can use for something much larger, later...

I have tried multiple Reality Composer models where I was able to drag when viewing the reality file on my phone, but that capability was removed once I imported the scenes into an Xcode model.

I have tried everything, including a single cube... and dragging doesn't work either.

import UIKit
import RealityKit

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        let boxAnchor = try! Experience.loadBox()
        arView.scene.anchors.append(boxAnchor)
        
        let cubesAnchor = try! Experience.loadCubes()
        arView.scene.anchors.append(cubesAnchor)
                   
        guard let addingGestures = cubesAnchor.twoCubes as? ModelEntity else { return }
        addingGestures.generateCollisionShapes(recursive: true)
        arView.installGestures([.all], for: addingGestures as (Entity & HasCollision))            
        
        DispatchQueue.main.asyncAfter(deadline: .now() + 2.0){
            self.arView.scene.anchors.removeAll()
            let coneAnchor = try! Experience.loadCone()
            self.arView.scene.anchors.append(coneAnchor)                
            print(coneAnchor)                
        }
        
        func handleTapOnEntity(_ entity: Entity?) {
            guard let entity = entity else { return }  
        }
    }
}

See related question here

Acierate

Answer 1

What Models Grouping actually means

In Reality Composer and RealityKit, grouping several objects together means combining several children under single parent (i.e. parenting op). Grouped objects can be simultaneously moved, rotated and scaled using their Parent Entity. However, this cannot be applied to the physics – or in other words, collisions, various dynamics forces, raycasting and three types of RealityKit gestures (EntityGestureRecognizer).

As you wrote, this is unambiguously stated in the docs.

If you’ve enabled physics in your scene, grouped objects still behave as separate, individual objects.

Thus, if you grouped models in Reality Composer, you can't make the two cubes draggable / scalable when you apply corresponding pan or pinch gestures to a group. Nevertheless, there is one workaround.

Workaround

In RealityKit, create an empty group as a ModelEntity. Make its CollisionShape larger than both models. In this case, you can use pan, rotate and pinch gestures to control several objects at once. You can generate collision shape as box, sphere, capsule, and convex. You can also create your collision shape from separate parts.

import UIKit
import RealityKit

class ViewController: UIViewController {
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        self.arView.debugOptions = .showPhysics
        
        let sph01 = ModelEntity(mesh: .generateSphere(radius: 0.05))
        sph01.position.x = -0.1
        let sph02 = ModelEntity(mesh: .generateSphere(radius: 0.05))
        sph02.position.x = 0.1
    
        let group = ModelEntity() as ModelEntity & HasCollision
        group.addChild(sph01)
        group.addChild(sph02)
        
        group.generateCollisionShapes(recursive: false)
        self.arView.installGestures(.all, for: group)
        
        let shape = ShapeResource.generateSphere(radius: 0.16)
        let collision = CollisionComponent(shapes: [shape],
                                             mode: .trigger,
                                           filter: .sensor)    
        group.components.set(collision)
        
        let anchor = AnchorEntity()
        anchor.addChild(group)
        anchor.scale = [5,5,5]
        arView.scene.anchors.append(anchor)    
    }
}

Eastward

Thank you so much, Andy Jazz, THE LEGEND, for answering my question. I tested it and it works perfectly! However, I still have one remaining issue which was addressed in a question you answered yesterday ( #76137068 ). Like the person who asked the question, I struggle with the findEntity(named: ) method for Reality Composer because all objects within the scene have the same name "simpBld_root" even when I name them on RC. findEntity(named: ) didn't work. – Acierate

I need to go back to the documentation and study parenting better as you mentioned. Is there an alternative for the findEntity(named: ) method on RC? I tried calling the objects by name but it didn't work. Only when I used sceneName.objectName and the Entity & hasCollision ...for example let greenCubesGest = cubesAnchor.greenCube as? Entity & HasCollision arView.installGestures(for: greenCubesGest!) but this won't work here as the goal is to call each object as a ModelEntity – Acierate

Hi @Manuela, Create a new question – include the model and your code, please. Tomorrow I'll answer it. – Eastward

Thank you so much, Andy, I added a new question here #76150591 – Acierate