Stereo Disparity map generation

Question

I am working on KITTI data set i am taking 2 images and finding the disparity to get 3D point cloud .The problem which i am facing is that i am not able to get a good disparity map.Most of the disparity value is less than 0.1 .The disparity values are between 0 to 1 (do i need to scale them). The parameters of my stereo are listed below

cv::StereoBM sbm;
    sbm.state->SADWindowSize = 9;
    sbm.state->numberOfDisparities = 112;
    sbm.state->preFilterSize = 5;
    sbm.state->preFilterCap = 1;
    sbm.state->minDisparity = 0;
    sbm.state->textureThreshold = 5;
    sbm.state->uniquenessRatio = 5;
    sbm.state->speckleWindowSize = 0;
    sbm.state->speckleRange = 20;
    sbm.state->disp12MaxDiff = 64;
sbm(leftimage, rightimage,disp);
    normalize(disp, disp8, 0.1, 255, CV_MINMAX, CV_8U);

Vesperal

My gut feeling is that your window size is too small. Your window size is 9 x 9, yet if you look at the displacement on the right where you see the road sign, I'm sure it's much larger than a 9 pixel window. Have you tried increasing your window size? What about 15? 25? – Pharr

The second problem is that almost all disparity values are under 1.Which shouldn't be the case – Vesperal

Did you change the window size? – Pharr

yup ....got a better image when i used 15 ....got a worse image when used 25 ....so i sicked with 15.....thanks ...I tuned all the parameter using the toolbox downloaed from this link martinperis.com/stereobmtuner/stereoBMTuner-1.0.tgz – Vesperal

No problem. Yeah, when you're doing disparity with sliding windows, you have to find the right window. Too small means you won't capture any disparity.... and too large means that you may be introducing false information into the comparison windows. Even though using sliding windows are fast, they don't provide the best results. There has been some work on adaptively determining the size of the window. Check Kanade and Okutomi's work: cs.cmu.edu/~ph/869/papers/kanade-okutomi.pdf - anyway, I'm going way off topic. Glad you got it to work! – Pharr

For ground points using rectangular window (more wide than tall) as they have changing disparity along the y axis (so disparity is not constant in the sliding window area). Related to this topic, you might want to take a look to "V-disparity image" related research. – Spongioblast

sorry @Spongioblast i didn't got your point.Can you elaborate a little bit.Thanks – Vesperal

Think to a square window centered on a point of the ground. The top pixels of this window corresponds to point of the ground further away than those corresponding to the bottom pixels of the same window. Therefore, the correct disparity of top pixels is lower than bottom pixels within the same window. For big windows, this causes a poor match. However, e.g. if you use a window with double width and half height (same area) the difference in disparity between top and bottom pixels will be lower, with a more homogeneous disparity within the window, possibly leading to a better match. – Spongioblast

Answer 1

The disparity map you have "looks" good for Block Matching.

Block Matching is the most basic method to obtain disparity maps. It is a local method that computes the disparity estimate via a brute force search (modulo filtering from opencv). Hence its output is limited in accuracy and is typically noisy.

As others have mentioned you can adjust the window size to improve the results slightly, but this will not make the disparity significantly better.

Look at the stereo evaluation on the KITTI benchmark and select a more accurate algorithm if you have to. OpenCV has an implementation of SGM, which produces smoother disparities. The desired quality of disparity maps depends on your application. In some cases block matching is sufficient. For others, it may not be.

Remember, the definition of a disparity is: the difference between the x-coordinate of a pixel in the left image and its corresponding pixel in the right image. That is, the unit of disparity is "pixels".

Larger disparities, mean closer objects. When you scale the image for display, larger disparities appear brighter. For example, the sign on the road is closer to the camera and it appears brighter than pixels far away on the road.

Your disparity values are not supposed to be between 0 and 1. You are scaling the image for display as a uint8 which is ok for display, but not suitable to use the disparity for an actual measurement.

In OpenCV, the default behavior is to produce a disparity map as a signed short obtained by multiplying subpixel shifts with 16. To obtain the true disparity values, you divide opencv's output by 16 and convert to float.

You can do something like this:

cv::Mat<float> true_dmap = disp * (1.0 / 16.0f);

or

disp.convertTo(true_dmap, CV_32F, 1.0/16.0, 0.0);

Or, you can call reprojectImageTo3D to get a point cloud given the estimated disparity map and stereo calibration.

Note, if you attempt to display true_map via imshow, you will not see something meaningful.

Good luck,

Mydriasis

Hey, Bender that was quite an impressive answer, I have been using GPU version of the stereoBM in OpenCV which oddly returns different results than CPU and most of the tweaking parameter setter seems to do nothing, The GPU version of the code returns in 8UC1, unlike CPU counterpart which returns in 16SC1. My Depth calculations seem to be messed like I'm getting 20000cm for a 50 cm distance. What should I do get the right distance. – Colburn

"if you attempt to display true_map via imshow, you will not see something meaningful" perhaps you can add what "tool" would be able to display something meaningful. – Antefix