Create a full 3d skeleton with Kinect SDK

Question

This is for my first graphics subject at uni, and some parts of the implementation just aren't working for me. I can get the joints to draw properly, but I'm trying to write a function that puts in 'bones' between the joints. At this point the bones are just cubes that are transformed to be rectangular prisms, at a later point I'll attempt to introduce proper models from blender or something.

My trouble is with the rotations. After about 5 hours or so my partner and I have it sort of working, but as soon as you move your arms or legs out the cubes warp and look strange. Any help would be appreciated. Below is the function which attempts to draw the bones.

private void DrawBone(Skeleton skeleton, JointType jointType0, JointType jointType1)
{
    Joint joint0 = skeleton.Joints[jointType0];
    Joint joint1 = skeleton.Joints[jointType1];

    // If we can't find either of these joints, exit
    if (joint0.TrackingState == JointTrackingState.NotTracked ||
        joint1.TrackingState == JointTrackingState.NotTracked)
    {
        return;
    }

    // Don't draw if both points are inferred
    if (joint0.TrackingState == JointTrackingState.Inferred &&
        joint1.TrackingState == JointTrackingState.Inferred)
    {
        return;
    }

    // We assume all drawn bones are inferred unless BOTH joints are tracked
    if (joint0.TrackingState == JointTrackingState.Tracked && joint1.TrackingState == JointTrackingState.Tracked)

        //jointvector(joint) takes a joint and returns a Vector2 with its position data, with the z invtred to work properly with directx11
        Vector3 bonevector = Vector3.Subtract(jointVector(joint0), jointVector(joint1));
        //cubevector is supposed to describe the initial vector of a cube centred at (0,0,0) with a side length of 2. A fair amount of guesswork has gone into this bit.
        Vector3 cubevector = new Vector3(0, 2, 0);
        //midpoint of two joints
        Vector3 transTest = Vector3.Divide(Vector3.Add(jointVector(joint0),jointVector(joint1)), 2);
        //divide by two because our original cube has side length 2
        float scaleFactor = Math.Abs(bonevector.Length() / cubevector.Length());

        //we haven't been taught quaternions in class or anything, but after a fair while searching for similar problems they seemed like the go-to option for rotations.
        world = Matrix.Transformation(new Vector3(0, 0, 0), new Quaternion(0), new Vector3(0.05f, scaleFactor, 0.05f), new Vector3(0, 0, 0), new Quaternion(Vector3.Cross(cubevector, bonevector), (float)Math.Acos(Vector3.Dot(bonevector, cubevector))), transTest);     

        //view and proj are defined elsewhere and are working fine, it's just the world that is being lame         
        worldViewProj = world * view * proj;
        worldViewProj.Transpose();
        sDXdata.context.UpdateSubresource(ref worldViewProj, sDXdata.constantBuffer);
        //36 vertices of the cube (two triangles per face) defined elsewhere
        sDXdata.context.Draw(36, 0);
        return;

Gerry

Hi Vince, did you ever get this going? Would be interested in understanding how you displayed the model in 3D. – Propeller

Answer 1

For bone orientations, check:

skeleton.BoneOrientations[joint.JointType]

it will give you a BoneOrientation class

http://msdn.microsoft.com/en-us/library/microsoft.kinect.boneorientation.aspx

From there you can have rotation as either Quaternion/Matrix, either in world space, or in parent bone space (for skinning)

Also you use:

new Quaternion(0)

This is a 0 vector, not a valid quaternion for rotation, use:

Quaternion.Identity;

which will be (0,0,0,1)

Healey

Answer 2

Although this was asked awhile ago, for what it is worth I think there is also an error when you calculate your angle. You have:

(float)Math.Acos(Vector3.Dot(bonevector, cubevector)

I think you need:

(float)Math.Acos(Vector3.Dot(bonevector, cubevector) / (bonevector.Length() *
cubevector.Length()))

Here is a nice little example of the maths. You could also make them unit vectors, so they have a length of one and not bother with the division:

(float)Math.Acos(Vector3.Dot(bonevector.Normalize(), cubevector.Normalize()))

Check out the MSDN page for more information.

Involuntary