Point-Cloud of Body Using Kinect SDK

Question

I am making a program with the SDK, where when users are detected, The program draws a skeleton for them to follow. I recently saw a game advertised on my Xbox, Nike+ Kinect and saw how it displays a copy of the character doing something else like:

http://www.swaggerseek.com/wp-content/uploads/2012/06/fcb69__xboxkinect1.jpg

Or

http://www.swaggerseek.com/wp-content/uploads/2012/06/fcb69__xboxkinect.jpg

Can I create a point-cloud representation of the only the person detected (not any of the background)? Thanks in advance!

---

EDIT

Using this site, I can create point clouds, but still can't crop around the body of the person.

Gallantry

Answer 1

It doesn't look like they are displaying a complete point cloud but rather a blue shaded intensity map. This could be done with the depth image from the Kinect for Windows sdk. What you are looking for is the player index. This is a provided bit in each pixel of the depth image. In order to get the player index bit you have to also enable the skeletal stream in your initialization code.

So this is how I would do it. I am modifying one of the Kinect for Windows SDK quickstarts found here load it up and make the following changes:

//Change image type to BGRA32
image1.Source = 
                BitmapSource.Create(depthFrame.Width, depthFrame.Height, 
                96, 96, PixelFormats.Bgra32, null, pixels, stride); 

        //hardcoded locations to Blue, Green, Red, Alpha (BGRA) index positions       
        const int BlueIndex = 0;
        const int GreenIndex = 1;
        const int RedIndex = 2;
        const int AlphaIndex = 3;

//get player and depth at pixel
int player = rawDepthData[depthIndex] & DepthImageFrame.PlayerIndexBitmask;
int depth = rawDepthData[depthIndex] >> DepthImageFrame.PlayerIndexBitmaskWidth;

//check each pixel for player, if player is blue intensity.

            if (player > 0)
            {
                pixels[colorIndex + BlueIndex] = 255;
                pixels[colorIndex + GreenIndex] = intensity;
                pixels[colorIndex + RedIndex] = intensity;
                pixels[colorIndex + AlphaIndex] = 100;

            }
            else
            {
                //if not player make black and transparent
                pixels[colorIndex + BlueIndex] = 000;
                pixels[colorIndex + GreenIndex] = 000;
                pixels[colorIndex + RedIndex] = 000;
                pixels[colorIndex + AlphaIndex] = 0;
            }

I like using this example for testing the colors since it still provides you with the depth viewer on the right side. I have attached an image of this effect running below:

The image to the left is the intensity map with slightly colored pixel level intensity data.

Hope that helps David Bates

Meyerhof

After thinking about the pixel level effect. You could for every other pixel increase the intensity so that you could get that sparkle effect shown on the sample image. Just sayin :) – Meyerhof

Answer 2

You can do a very simple triangulation of the points. Check this tutorial:

http://www.riemers.net/eng/Tutorials/XNA/Csharp/Series1/Terrain_basics.php

Check the result:

Satterlee

Very nice. could you post the actual project or how you are mapping from depth to triangles? The tutorial creates a static grid but does not speak on mapping. – Meyerhof

It's simple. Each pixel in the depthstream became in a vertex, X and Y is the pixel position and Z is the depth. So using the method on that tutorial create the indices of the triangles. – Satterlee

Here is the function to create the triangles. It pretty dirty code, je je. paste.ideaslabs.com/show/wUQpbnvU5T – Satterlee

Very nice, Trying it out now. One thing I notice from your code is that it is building vertices each time a depthstream is sent to it. How does this affect performance, I know vectors are cheap but this is a ton of them and all rendered software side. – Meyerhof

Well, in my old pc CoreDuo, intel graphics run fine. But sure, you can do some optimizations. – Satterlee

Could you please provide the necessary code for this? It seems like the URL for the function you have provided is not working – Bushido

Answer 3

This is not possible automatically with official Kinect SDK. But it is implemented in alternative SDK called OpenNI, there you can just get the set of points which of which user consists. If you don't want to use it I can suggest rather easy method of separating user from background. Since you know the z-position of user you can just take points which z is from 0 to userZ + some value representing thickness of body.

Another idea is walk over point cloud starting from some joint (or joints) and taking points only if distance is changing smoothly, because if you take background point, border body and next body point the distance drop will be easily noticeable. The problem here is that you will start counting floor as a part of body, because transition there is smooth, so you should validate it using lowest (ankle) joint.

Or you can use segmentation in PCL (http://docs.pointclouds.org/trunk/group__segmentation.html) but I don't know if the feet-floor problem is solved there. Looks like they are good with it (http://pointclouds.org/documentation/tutorials/planar_segmentation.php).

Promethium

@OutlawLemur for PCL there is link to tutorial. If you decide to implement it yourself there can not be an example, it is task to be solved. – Promethium

Answer 4

Kinect for Windows SDK v1.5 has a sample that could be modified for this.

Sample names: depth-d3d or depthwithcolor-d3d.

They both do point clouds.

Donelu

Cant help but notice those are both C++ examples, I am using C# – Gallantry