How to get a ordinary Mixamo character animation working in SceneKit?
Asked Answered
E

1

6

Go to mixamo.com, pick a character, tap animations, pick one, simply download as .dae.

enter image description here

Have the file on your Mac desktop; tap file info). It will perfectly animate the character move.

Xcode, drag in the folder. Tap the .dae file, tap the Play icon at the bottom. It will perfectly animate the character move.

Now, add the character to your existing SceneKit scene. For example:

let p = Bundle.main.url(forResource: "File Name", withExtension: "dae")!
modelSource = SCNSceneSource(url: p, options: nil)!
let geom = modelSource.entryWithIdentifier("geometry316",
                 withClass: SCNGeometry.self)! as SCNGeometry
theModel = SCNNode(geometry: geom)
.. your node .. .addChildNode(theModel)

(To get the geometry name, just look in the .dae text example )

You will PERFECTLY see the character, in T pose

However it seems impossible to run the animation on the character.

Code would look something like ...

theAnime = amySource.entryWithIdentifier("unnamed_animation__0", withClass: CAAnimation.self)!
theModel.addAnimation(theAnime, forKey:"aKey")

No matter what I try it just doesn't animate.

At the moment you addAnimation, the character jumps to a different static position, and does nothing. (If you arrange to "end" the animation removeAllAnimations(), it simply returns to the T-pose.)

Clearly the dae file is perfect since the shows the animation perfectly simply in the Mac finder, and perfectly on the actual screen of the .dae file in Xcode!

In short, from the mixamo image above, has anyone been able to get the animation to run, actually, in a SceneKit scene?

(PS not ARKit .. scene kit.)

Edsel answered 12/1, 2023 at 0:49 Comment(0)
S
8

First, you need your character in the T-Position only. Download that file as Collada (DAE) with the Skin. Do NOT include any animations to this File. No Further modifications are required to this file then.

Then, for any animation effect you will implement like walking, running, dancing, or whatever - do it like so:

Test/Apply your desired animation in Mixamo on the character, adjust the settings as you want then download it. Here it is very important to Download as Collada (DAE) and choose WITHOUT Skin!!! Leave Framerate and keyframe reduction default.

This will give you a single DAE File for each animation you want to implement. This DAE contains no mesh data and no rig. It only contains the deformations of the Model to which it belongs (this is why you choose to download it without Skin).

Then you need to do two additional operations on all DAE Files which contains animations.

First, you need to pretty-print the XML structure of each DAE containing an animation. You can do this i.Ex. using the XML Tools in Notepad++ or you open a terminal on your Mac and use this command:

xmllint —-format my_anim_orig.dae > my_anim.dae

Then install this Tool here on your Mac. (https://drive.google.com/file/d/167Le084XFM5euTm4TRItbhXXPVaPJhZN/view?usp=sharing) Converter Tool

Convert all of your DAE Animations with this converter: (But do NOT convert your T-Pose Model using this tool!!!) Collada converter

Now we are ready to setup the Animation:

you should organise the DAE's within the art.scnassets folder

enter image description here

Let's configure this:

I usually organise this within a struct called characters. But any other implementation will do

add this:

struct Characters {
    
    // MARK: Characters
    var bodyWarrior                         : SCNNode!
    
    private let objectMaterialWarrior      : SCNMaterial = {
        let material = SCNMaterial()
        material.name                       = "warrior"
        material.diffuse.contents           = UIImage.init(named: "art.scnassets/warrior/textures/warrior_diffuse.png")
        material.normal.contents            = UIImage.init(named: "art.scnassets/warrior/textures/warrior_normal.png")
        material.metalness.contents         = UIImage.init(named: "art.scnassets/warrior/textures/warrior_metalness.png")
        material.roughness.contents         = UIImage.init(named: "art.scnassets/warrior/textures/warrior_roughness.png")
        material.ambientOcclusion.contents  = UIImage.init(named: "art.scnassets/warrior/textures/warrior_AO.png")
        material.lightingModel              = .physicallyBased
        material.isDoubleSided              = false
        return material
    }()
    
    // MARK: MAIN Init Function
    init() {
        
        // Init Warrior
        bodyWarrior = SCNNode(named: "art.scnassets/warrior/warrior.dae")
        bodyWarrior.childNodes[1].geometry?.firstMaterial = objectMaterialWarrior // character body material
        
        print("Characters Init Completed.")
        
    }
    
}

Then you can init the struct i.Ex. in the viewDidLoad var characters = Characters()

Pay Attention to use the correct childNodes! child nodes

in this case the childNodes[1] is the visible mesh and childNodes[0] then will be the animation Node.

you might also implement this SceneKit extension to your code, it is very useful to import Models. (attention, it will organise the model nodes as Childs from a new node!)

extension SCNNode {
    convenience init(named name: String) {
        self.init()
        guard let scene = SCNScene(named: name) else {return}
        for childNode in scene.rootNode.childNodes {addChildNode(childNode)}
    }
}

also add that extension below. You'll need it for the animation player later.

extension SCNAnimationPlayer {
    class func loadAnimation(fromSceneNamed sceneName: String) -> SCNAnimationPlayer {
        let scene = SCNScene( named: sceneName )!
        // find top level animation
        var animationPlayer: SCNAnimationPlayer! = nil
        scene.rootNode.enumerateChildNodes { (child, stop) in
            if !child.animationKeys.isEmpty {
                animationPlayer = child.animationPlayer(forKey: child.animationKeys[0])
                stop.pointee = true
            }
        }
        return animationPlayer
    }
}

Handle Character setup and Animation like so: (here is a simplified version of my Class)

class Warrior {
    
    // Main Nodes
    var node                 = SCNNode()
    private var animNode     : SCNNode!
    
    // Control Variables
    var isIdle               : Bool = true
    
    // For Initial Warrior Position and Scale
    private var position            = SCNMatrix4Mult(SCNMatrix4MakeRotation(0,0,0,0), SCNMatrix4MakeTranslation(0,0,0))
    private var scale               = SCNMatrix4MakeScale(0.03, 0.03, 0.03) // default size ca 6m height
    
    // MARK: ANIMATIONS
    private let aniKEY_NeutralIdle       : String = "NeutralIdle-1"       ; private let aniMAT_NeutralIdle       : String = "art.scnassets/warrior/NeutralIdle.dae"
    private let aniKEY_DwarfIdle         : String = "DwarfIdle-1"         ; private let aniMAT_DwarfIdle         : String = "art.scnassets/warrior/DwarfIdle.dae"
    private let aniKEY_LookAroundIdle    : String = "LookAroundIdle-1"    ; private let aniMAT_LookAroundIdle    : String = "art.scnassets/warrior/LookAround.dae"
    private let aniKEY_Stomp             : String = "Stomp-1"             ; private let aniMAT_Stomp             : String = "art.scnassets/warrior/Stomp.dae"
    private let aniKEY_ThrowObject       : String = "ThrowObject-1"       ; private let aniMAT_ThrowObject       : String = "art.scnassets/warrior/ThrowObject.dae"
    private let aniKEY_FlyingBackDeath   : String = "FlyingBackDeath-1"   ; private let aniMAT_FlyingBackDeath   : String = "art.scnassets/warrior/FlyingBackDeath.dae"
    
    // MARK: MAIN CLASS INIT
    init(index: Int, scaleFactor: Float = 0.03) {
        
        scale = SCNMatrix4MakeScale(scaleFactor, scaleFactor, scaleFactor)
        
        // Config Node
        node.index = index
        node.name = "warrior"
        node.addChildNode(GameViewController.characters.bodyWarrior.clone()) // childNodes[0] of node. this holds all subnodes for the character including animation skeletton
        node.childNodes[0].transform = SCNMatrix4Mult(position, scale)
        
        // Set permanent animation Node
        animNode = node.childNodes[0].childNodes[0]
        
        // Add to Scene
        gameScene.rootNode.addChildNode(node) // add the warrior to scene
        
        print("Warrior initialized with index: \(String(describing: node.index))")
        
    }
    
    
    // Cleanup & Deinit
    func remove() {
        print("Warrior deinitializing")
        self.animNode.removeAllAnimations()
        self.node.removeAllActions()
        self.node.removeFromParentNode()
    }
    deinit { remove() }
    
    // Set Warrior Position
    func setPosition(position: SCNVector3) { self.node.position = position }
    
    // Normal Idle
    enum IdleType: Int {
        case NeutralIdle
        case DwarfIdle // observe Fingers
        case LookAroundIdle
    }
    
    // Normal Idles
    func idle(type: IdleType) {
        
        isIdle = true // also sets all walking and running variabled to false
        
        var animationName : String = ""
        var key           : String = ""
        
        switch type {
        case .NeutralIdle:       animationName = aniMAT_NeutralIdle        ; key = aniKEY_NeutralIdle      // ; print("NeutralIdle   ")
        case .DwarfIdle:         animationName = aniMAT_DwarfIdle          ; key = aniKEY_DwarfIdle        // ; print("DwarfIdle     ")
        case .LookAroundIdle:    animationName = aniMAT_LookAroundIdle     ; key = aniKEY_LookAroundIdle   // ; print("LookAroundIdle")
        }
        
        makeAnimation(animationName, key, self.animNode, backwards: false, once: false, speed: 1.0, blendIn: 0.5, blendOut: 0.5)
        
    }
    
    func idleRandom() {
        switch Int.random(in: 1...3) {
        case 1: self.idle(type: .NeutralIdle)
        case 2: self.idle(type: .DwarfIdle)
        case 3: self.idle(type: .LookAroundIdle)
        default: break
        }
    }
    
    // MARK: Private Functions
    // Common Animation Function
    private func makeAnimation(_ fileName           : String,
                               _ key                : String,
                               _ node               : SCNNode,
                               backwards            : Bool = false,
                               once                 : Bool = true,
                               speed                : CGFloat = 1.0,
                               blendIn              : TimeInterval = 0.2,
                               blendOut             : TimeInterval = 0.2,
                               removedWhenComplete  : Bool = true,
                               fillForward          : Bool = false
                              )
    
    {
        
        let anim   = SCNAnimationPlayer.loadAnimation(fromSceneNamed: fileName)
        
        if once { anim.animation.repeatCount = 0 }
        anim.animation.autoreverses = false
        anim.animation.blendInDuration  = blendIn
        anim.animation.blendOutDuration = blendOut
        anim.speed = speed; if backwards {anim.speed = -anim.speed}
        anim.stop()
        print("duration: \(anim.animation.duration)")
        
        anim.animation.isRemovedOnCompletion = removedWhenComplete
        anim.animation.fillsForward          = fillForward
        anim.animation.fillsBackward         = false
        
        // Attach Animation
        node.addAnimationPlayer(anim, forKey: key)
        node.animationPlayer(forKey: key)?.play()
        
    }
    
}

you can then initialise the Class Object after you initialised the characters struct.

the rest you'll figure out, come back on me, if you have questions or need a complete example App :)

Summon answered 12/1, 2023 at 7:53 Comment(10)
Dear Fattie, you are welcome, as always ;)Summon
(Woot! I didn't realize mixamo has an "in place" button if you are doing root motion, fantastic.) Minor things I have found! For others googling here: (1) Actually on current Mac (on my Macs anyway), you just xmllint anim.dae > fixed.dae; if you use the —-format arg it seems to do nothing and throw a pointless error... (2) When you "ConvertToXcodeCollada" using context menu, it creates an additional ".dae-e" file which is simply the original; can be thrown away.Edsel
unfortunatly not every mixamo animation has the in-place option, but most of the basic movement stuff, like running, walking, etc. does. @Fattie: if you give me your e-mail, i'd like to show you somethingSummon
Hi @ZAY, how is it possible to set a node index like you do in your warrior init (node.index = index)?Illlooking
please ask a separate question and I will answer it :)Summon
tnx, now I got it... see ya laterSummon
Does anyone have a working link to the ConvertCollada script?Chin
I updated the download linkSummon
THANK YOU!!! I can confirm it works on 2024Chin
great to hear that you like this solution/answerSummon

© 2022 - 2024 — McMap. All rights reserved.