Read and display animations from *.FBX file

Question

I want to read 3d model from fbx file and display it within openGL es 2.0 engine on iPhone (not using Unity) and also I want to display animations for read 3d object.

How can i get animations from fbx file?

Currently I able to get list of poses names with, as I understood, coming with transformation matrix, full list of poses in layer, stacks, layers, and a lot of curves.

How this all information can be combined for displaying proper animations?

I also try to parse some info within TakeInfo, but result a little bit strange as for me, for example:

    FbxTakeInfo *ltakeInfo  =  pScene->GetTakeInfo(lAnimStack->GetName());
    FbxTime start = ltakeInfo->mLocalTimeSpan.GetStart();
    FbxTime end = ltakeInfo->mLocalTimeSpan.GetStop();

    self.startTime = start.GetSecondDouble();
    self.endTime = end.GetSecondDouble();

here i got start = 0 and end = 0.014 for each of parsed layers, so i guees this is incorrect (fbx file that I want to display contains 1 mesh with simple 5 sec duration animations).

---

Update

After few hours of investigating I got next things:

For reference this is structure of test obj that i want to display:

Here u can see a lot of bones (more concrete - 19) I able to get (as noted above) a 19 animated obj in list (like names of bone/obj) and 19 groups of curves within 151 frames each (with frame rate 30 exactly 5 sec of anim. - 30*5=150 + 1 last identity matrix).

If I try use each of curve group to my mesh one by one (I able to parse 1 mesh only) I see animation of different part of mesh applied to all mesh (for example vertical rotation or horizontal translation), so I think that this curves in each group should be applied exactly to specific bone and as result i will got animation for my mesh. The problem is that i don't know how to animate only selected bone vertex part only.

Now the problem is how to apply this all animations divided in to separate group specific to bone to whole obj (because I have only one mesh)?

How can I get 1 global curve's list for each frame from list with all curve's groups?

---

Update2

Thanks to @codetiger advice I follow instruction within provided link in comment and with that technique I'm able to retrieve list of bone-specific mat's with start and end time and required transforms, but this is almost same like I got before with curves - the only difference that with curves I should create mat from 9 curves (translate/ scale/ rotate for x y z) instead of using full matrix, but problem are still present - how can i combine them in 1 global matrix?

The code that I use (found few links for it):

FbxNode* modelNode = _fbxScene->GetRootNode();
FbxAMatrix geometryTransform = GetGeometryTransformation(modelNode);
for (unsigned int deformerIndex = 0; deformerIndex < numOfDeformers; ++deformerIndex) {
    FbxSkin* currSkin = reinterpret_cast<FbxSkin*>(mesh->GetDeformer(deformerIndex, FbxDeformer::eSkin));
    if (!currSkin) {
        continue;
    }
    unsigned int numOfClusters = currSkin->GetClusterCount();
    for (unsigned int clusterIndex = 0; clusterIndex < numOfClusters; ++clusterIndex) {
        FbxCluster* currCluster = currSkin->GetCluster(clusterIndex);
        std::string currJointName = currCluster->GetLink()->GetName();

        FbxAMatrix transformMatrix;
        FbxAMatrix transformLinkMatrix;
        FbxAMatrix globalBindposeInverseMatrix;

        currCluster->GetTransformMatrix(transformMatrix);   
        currCluster->GetTransformLinkMatrix(transformLinkMatrix);   
        globalBindposeInverseMatrix = transformLinkMatrix.Inverse() * transformMatrix * geometryTransform;

        FbxAnimStack* currAnimStack = _fbxScene->GetSrcObject<FbxAnimStack>(0);
        FbxString animStackName = currAnimStack->GetName();
        char *mAnimationName = animStackName.Buffer();
        FbxTakeInfo* takeInfo = _fbxScene->GetTakeInfo(animStackName);
        FbxTime start = takeInfo->mLocalTimeSpan.GetStart();
        FbxTime end = takeInfo->mLocalTimeSpan.GetStop();
        FbxLongLong mAnimationLength = end.GetFrameCount(FbxTime::eFrames24) - start.GetFrameCount(FbxTime::eFrames24) + 1;

        for (FbxLongLong i = start.GetFrameCount(FbxTime::eFrames24); i <= end.GetFrameCount(FbxTime::eFrames24); ++i) {
            FbxTime currTime;
            currTime.SetFrame(i, FbxTime::eFrames24);

            FbxAMatrix currentTransformOffset = modelNode->EvaluateGlobalTransform(currTime) * geometryTransform;
            FbxAMatrix mat = currentTransformOffset.Inverse() * currCluster->GetLink()->EvaluateGlobalTransform(currTime);

        }
    }
}

Here I receive 121 matrix instead of 151 - but duration of some matrix transforms take more than duration of 1 frame draw, so the q-ty here I think also correct

float duration = end.GetSecondDouble() - start.GetSecondDouble(); //return 5 as and expected

I guess that that Autodesk SDK there is a way to get 1 Global transform per each drawCall

Any suggestions? Is this possible?

---

Add bounty for anyone who can describe how to display animations on iPhone within openGLES 2.0 with Autodesk SDK... (sorry typo instead of Facebook)

---

Here what i'm able to get

Original object in blender

If i draw bone separately (same VBO with different transform and only indices for each bone accordantly)

Frulla

Please check this link gamedev.stackexchange.com/questions/59419/… – Looksee

i saw this link - try to get mats with described method but all that i receive - 151 mat (with fps 30 it's mean 5 sec - correct) - are Identity matrix... but should be as i understood mats for global transform of each mesh that i have with specific start and end time – Frulla

I also try to play file within blender and FBX Review app from Autodesk - all work correctly... How about usage of stacks and layers - can u advice something how to work with that items? – Frulla

sorry, my knowledge is limited to reading Bones and other information from FBX and dont have much exp on animation data. Lets wait for someone to respond. – Looksee

Have you read the animation information from the file? The bounty is to give you information on how to render the animation on iOS? – Looksee

Im able to read stacks, layers, curves, anim names qtyty of readed curves equal to qty of anim frame-> from this im able to create anim mattices for different bone(i guess, because name of layer curves same like bones in blebder) But when i try to apply it to obj i able to see movement of while obj as movement of one bone, so the problem is how to apply transforms from curves to few bones, when i have only one single vbo, orhow to create global transform ( if its possible). Another way that i try - directly extract matrices fir each bone from mesh – Frulla

On second approach i also able extract some indices, but they are partial only, if needed i can share chunk of code with extraction logic and drawing also - just let me know - not share it now because its to many lines if code – Frulla

Am not clear where you are struck, but as you have mentioned that you are able to read matrices for bones from the file, I am give an answer that give information on how to render animation in opengles – Looksee

i also can share my code to make all that i done much clear - i guess i miss some major part, but not sure... – Frulla

Answer 1

Here is how to render animated mesh in OpenGL ES. This will give you fare information on what details you need to read from the file.

Method 1: (GPU Skinning)

This method works only on limited number of bones based on the capabilities of the hardware. Usually it depends on the amount of matrices you can send to the shader.

Bind the mesh information to the GPU using BindAttributes once and send the matrixes to the shader in uniforms.

Step 1: Read all Bone Matrix and create an array of Matrices and send this data to the shader in uniforms.

Step 2: In the Vertex Shader, calculate the gl_position like the below shader

attribute vec3 vertPosition;
attribute vec3 vertNormal;
attribute vec2 texCoord1;
attribute vec3 vertTangent;
attribute vec3 vertBitangent;
attribute vec4 optionalData1;
attribute vec4 optionalData2;

uniform mat4 mvp, jointTransforms[jointsSize];

void main()
{
    mat4 finalMatrix;

    int jointId = int(optionalData1.x);
    if(jointId > 0)
        finalMatrix = jointTransforms[jointId - 1] * optionalData2.x;

    jointId = int(optionalData1.y);
    if(jointId > 0)
        finalMatrix = finalMatrix + (jointTransforms[jointId - 1] * optionalData2.y);

    jointId = int( optionalData1.z);
    if(jointId > 0)
        finalMatrix = finalMatrix + (jointTransforms[jointId - 1] * optionalData2.z);

    jointId = int( optionalData1.w);
    if(jointId > 0)
        finalMatrix = finalMatrix + (jointTransforms[jointId - 1] * optionalData2.w);

    gl_Position = mvp * finalMatrix * vec4(vertPosition, 1.0);
}

In this shader, I've send the weight paint information in the attributes optionalData1 & optionalData2. The data is packed like this: (BoneId1, BoneId2, BoneId3, BoneId4) and (Weight4Bone1, Weight4Bone2, Weight4Bone3, Weight4Bone4). So in this case you are limited with a maximum of 4 bones affecting each attributes. The fragment shader part is usual.

Method 2: (CPU Skinning)

If you cannot live with the limitation in the GPU skinning, then the only way is to do the calculations in the CPU side.

Step 1: Calculate the vertex position in the current frame by multiplying the matrixes that belong to the bones that affects the vertex.

Step 2: Collect the new positions for the current frame and send the information to GPU in Vertex Attributes.

Step 3: Recalculate new Vertex positions in every frame and update the attributes buffer.

This method bring the load of the calculations to the CPU.

Looksee

i guess i follow 2nd approach but not via direct getting matrices but via getting anim curves from layer and creating from this curves (curves are like tx ty tz sx sy sz rx ry rz) matrices for transform... When i try to use metrices from clusters that i got within functions like GetTransformMatrix and other - on screen i see some strange things... – Frulla

i accept u'r answer - becasue it's good point to start, but honestly i'm hoping that anyone faced same problem as i and can advice in more details how to do animations with bones extracted from fbx file.... – Frulla