Remove Kinect depth shadow

Question

I've recently started hacking on my Kinect and I want to remove the depth shadow. The shadow is caused by the IR emitter being positioned slightly to the side of the camera, so any close object will get a big shadow and distant object less or no shadow.

The shadow length is related to the distance between the closest and the farthest spot on each side of the shadow.

My goal is to be able to map the color image correctly onto the depth. This doesn't work without processing the shadow as this picture shows:

Glasswort

Maybe some kind of trapezoid resize will work? – Agate

It'd help to be more explicit about what data you have to work with, for the Kinectless like me. It sounds like you have a grid of pixels from the camera and a corresponding grid of distances from the IR emitter. Right? – Hexapody

I have the data you see in the two pictures. The black area is shadow with color 0. The rest has a color value depending on distance. – Glasswort

maybe you're too close to the sensor? MS does advice you to be a few feet away from it, I guess this is the reason. – Milliliter

@Tedd I notice you tagged this with C#, which libraries are you using for this? I've been waiting for the official MS SDK to come out so that I can use C# in a University project. – Imaimage

@Darren: blog.tedd.no/2011/02/01/… ... its come a long way since then, but that is the latest public release. ... I'm waiting for MS SDK too. :) – Glasswort

@Tedd Thanks. Hopefully the offical SDk shall be released soon. MS did say it would be released in Spring, so hopefully anytime now. – Imaimage

@TeddHansen did you solve the problem in the meantime? – Shelf

I never found a solution to this, but also haven't looked at the MS SDK yet. – Glasswort

Was the Temporal Median of no help? – Thenna

Answer 1

Kinect Method - MapDepthFrametoColorFrame

Get the [x,y] positions in the depth frame, and use that method to fill in

Recall

Answer 2

Does the depth shadow always come out black?

If so you could use a simple method like a temporal median to calculate the background of the image (more info here: http://www.roborealm.com/help/Temporal_Median.php) and then whenever a pixel is black, set it to the background value at that pixel location.

Thenna

Answer 3

I did some preliminary work on this problem a few weeks ago. My code works directly on a WriteableBitmap rather than the depth data, but if you're only doing image processing, it should work. The algorithm isn't perfect and would benefit with some more tweaking. If you update the code at all, let me know; I'd be very interested to see what you're doing!

The source code is posted on my blog:

http://richardpianka.com/2011/02/trackingni-depth-correction/

Weintrob

Answer 4

I don't know how it is with c# but openni c++ has a function called xnSetViewPoint() the only problem is, you lose the top 20 or so rows of imagedata due to the transformation.

the reason is due to using two different sensors, which are placed close by each other but not exactly at the same position.

Bowlds

Answer 5

The color camera and kinect depth camera dont have the same dimensions, and origin of the infra red dots are not from the same cam, its a IR projector a few cm aside from it (as that displacement is used to calculate depth).

However the solution seams easy here, your shadow data is on the left side. so you need to extend the last known color data before it went black.

And to fit it better move translate the color cam data to the right.

Stevenson

Thanks. This problem was solved with the Kinect SDK. Marking MapDepthFrametoColorFrame as correct answer. :) – Glasswort

Answer 6

I'm sorry to say but that shadow is caused by your body blocking the inferred dots from hitting that spot of the room so it creates a black spot... Nothing you can do but change the base background to a different color other than black so it won't be a noticeable shadow

Nucleolated

That I understand, but if you look at the image of me then you see that the shadow is a perfect match, but wrongly aligned and sized. Size and alignment depends on proximity. if it was calibrated I would be able to repaint myself perfectly in the foreground while removing the background. (It is easy to see that all the required data is there.) – Glasswort

oh, i see what you meen... i can't help – Nucleolated