Kinect Depth Image

Question

In my application I am getting the depth frame similar to the depth frame retrieved from Depth Basics Sample. What I don't understand is, why are there discrete levels in the image? I don't know what do I call these sudden changes in depth values. Clearly my half of my right hand is all black and my left hand seems divided into 3 such levels. What is this and how do I remove this?

When I run the KinectExplorer Sample app I get the depth as follows. This is the depth image I want to generate from the raw depth data.

I am using Microsoft Kinect SDK's (v1.6) NuiApi along with OpenCV. I have the following code:

BYTE *pBuffer = (BYTE*)depthLockedRect.pBits; //pointer to data having 8-bit jump
USHORT *depthBuffer = (USHORT*) pBuffer; //pointer to data having 16-bit jump
int cn = 4;
this->depthFinal = cv::Mat::zeros(depthHeight,depthWidth,CV_8UC4); //8bit 4 channel
for(int i=0;i<this->depthFinal.rows;i++){
   for(int j=0;j<this->depthFinal.cols;j++){
      USHORT realdepth = ((*depthBuffer)&0x0fff); //Taking 12LSBs for depth
      BYTE intensity = (BYTE)((255*realdepth)/0x0fff); //Scaling to 255 scale grayscale
      this->depthFinal.data[i*this->depthFinal.cols*cn + j*cn + 0] = intensity;
      this->depthFinal.data[i*this->depthFinal.cols*cn + j*cn + 1] = intensity;
      this->depthFinal.data[i*this->depthFinal.cols*cn + j*cn + 2] = intensity;
      depthBuffer++;
   }
}

Stewpan

Please double check your images, they are not showing up. – Hydroxide

Changed the image URLs. Check again. – Stewpan

What is cn and does it really have to be 4 and not 3? It would also be gread to see where you actually output the image. – Radioluminescence

cn is the number of channels and for my purpose it has to be of 4 channels BGRA – Stewpan

@Stewpan you should check out my answer - it provides a solution to your problem. – Verisimilar

Answer 1

The stripes that you see, are due to the wrapping of depth values, as caused by the %256 operation. Instead of applying the modulo operation (%256), which is causing the bands to show up, remap the depth values along the entire range, e.g.:

BYTE intensity = depth == 0 || depth > 4095 ? 0 : 255 - (BYTE)(((float)depth / 4095.0f) * 255.0f);

in case your max depth is 2048, replace the 4095 with 2047.

More pointers:

1. the Kinect presumably returns a 11bit value (0-2047) but you only use 8bit (0-255).
2. new Kinect versions seem to return a 12bit value (0-4096)
3. in the Kinect explorer source code, there's a file called DepthColorizer.cs where most of the magic seems to happen. I believe that this code makes the depth values so smooth in the kinect explorer - but I might be wrong.

Verisimilar

If you wrapped the depth values by using a lesser value as the modulo, would the bands be further apart? – Jasmin

Answer 2

I faced the same problem while I was working on a project which involved visualization of depth map. However I used OpenNI SDK with OpenCV instead of Kinect SDK libraries. The problem was same and hence the solution will work for you as it did for me.

As mentioned in previous answers to your question, Kinect Depth map is 11-bit (0-2047). While in examples, 8-bit data types are used.

What I did in my code to get around this was to acquire the depth map into a 16-bit Mat, and then convert it to 8-bit uchar Mat by using scaling options in convertTo function for Mat

First I initialize a Mat for acquiring depth data

Mat depthMat16UC1(XN_VGA_Y_RES, XN_VGA_X_RES, CV_16UC1);

Here XN_VGA_Y_RES, XN_VGA_X_RES defines the resolution of the acquired depth map.

The code where I do this is as follows:

depthMat16UC1.data = ((uchar*)depthMD.Data());
depthMat16UC1.convertTo(depthMat8UC1, CV_8U, 0.05f); 
imshow("Depth Image", depthMat8UC1);

depthMD is metadata containing the data retrieved from Kinect sensor.

I hope this helps you in some way.

Politicking

Answer 3

The visualization of the depth image data has discreet levels that are coarse (0 to 255 in your code example), but the actual depth image data are numbers between 0 and 2047. Still discreet, of course, but not in such coarse units as the colors chosen to depict them.

Mccaffrey

Answer 4

The kinect v2 can see 8 meter depth, (but accuracy beyond 4.5 decreases). It start around 0.4 meter. So one needs to express a number 8000 to a color. A way to do this is use RGB colors just a numbers. then you could potentially store a number like 255x255x255 i a pixel. Or if you had different color format then it would be different. Storing 8000 in that 255x255x255 max number will result in a certain amount of R+G+B, and that gives this banding effect.

But you could ofcourse devide 8000 or substract a number, or remove beyond a certain value.

Cristiecristin