How does Kinect calculates depth?

Question

I'm getting a little bit confused here.

How does Kinect calculates depth: What I understand is that

• The IR projector throws out a pattern which is reflected back and read by the IR camera.
• Now the IR camera knows the pattern for a particular depth. The difference between the incoming and the known pattern is exploited to calculate the depth known using triangulation (using proportionality of similar triangles).

Question 1: Does it consider the distance between IR projector and IR camera? I guess no because they are too close to be considered.

Question 2: Now we are getting the depth directly from the pattern. When are we using disparity map to calculate depth?

Briefcase

Answer 1

The disparity map is basically the difference between the known and the observed pattern that you mention in the beginning. You use this during the depth computation.

The distance between the projector and the camera gets taken into account too.

Check out the following figure:

Pr is the position of a speckle in a reference depth Zr, and Po is the same speckle captured by the Kinect at a depth Zo (the depth we want to calculate). D is the 3D disparity between the 2 points, while d is the disparity on the 2D image plane. f is the focal length, and b is the is the distance between the camera C and the laser projector L.

As you mentioned, using similar triangles, the depth is calculated as:

The paper where the figure is from is Accuracy Analysis of Kinect Depth Data by K. Khoshelham. I'd suggest reading it for a more thorough explanation of the depth calculation process.

Microwave

I thought from the pattern, we directly compute depth map. But from the paper, I understand that the difference in the two patterns gives the disparity map. – Briefcase

You may want to edit the diagram. Po lies on the intersection of Pr-L and the object plane. – Briefcase

@Briefcase Yes, you are right, I wanted to make it larger and missed that, thanks! – Microwave

Sorry for pointing again. But now the projection of Po is not passing through C :) – Briefcase

@Briefcase Ahhh, fixed again! Let me know if you see anything else, I haven't had my morning coffee yet! :) – Microwave