How do you use OpenCV's DisparityWLSFilter in Python?

Question

I can compute a depth map with cv2.StereoSGBM that looks pretty good. Now I want to apply WLS filtering as described here.

This answer has some info, which I follow below, but I can't quite get to work.

How do I use ximgproc_DisparityWLSFilter in Python? I know the theory and how to do it in C++, but can't find any documentation for how the functions were wrapped in Python. (Using OpenCV 4.2.0). This is the source on GitHub, but it doesn't contain the Python bindings either.

Doing:

wls = cv2.ximgproc_DisparityWLSFilter.filter(disparity_SGBM, imgL)

Gives:

Traceback (most recent call last):
  File ".\stereo_SGBM_filtering.py", line 158, in <module>
    wls = cv2.ximgproc_DisparityWLSFilter.filter(disparity_SGBM, imgL)
TypeError: descriptor 'filter' requires a 'cv2.ximgproc_DisparityFilter' object but received a 'numpy.ndarray'

So I know I'm at least able to access the functions.

Fixing it like this:

wls = cv2.ximgproc_DisparityWLSFilter(stereoSGBM)
filtered_disparity_map = wls.filter(disparity_SGBM, imgL)

Doesn't give any errors, but also doesn't give me an image.

Full details:

Original images (Tsukuba from the Middlebury data set).

My depth map looks like this:

import cv2 

imgL = cv2.imread("tsukuba_l.png", cv2.IMREAD_GRAYSCALE)  # left image

... 

win_size = 2
min_disp = -4
max_disp = 9
num_disp = max_disp - min_disp  # Needs to be divisible by 16
stereoSGBM = cv2.StereoSGBM_create(
    minDisparity=min_disp,
    numDisparities=num_disp,
    blockSize=5,
    uniquenessRatio=5,
    speckleWindowSize=5,
    speckleRange=5,
    disp12MaxDiff=2,
    P1=8 * 3 * win_size ** 2,
    P2=32 * 3 * win_size ** 2,
)

disparity_SGBM = stereoSGBM.compute(imgL_undistorted, imgR_undistorted)
wls = cv2.ximgproc_DisparityWLSFilter(stereoSGBM)
filtered_disparity_map = wls.filter(disparity_SGBM, imgL)

Disheveled

Answer 1

You need to use the createDisparityWLSFilter factory method to get an instance of DisparityWLSFilter and use it.

For example:

wsize=31
max_disp = 128
sigma = 1.5
lmbda = 8000.0
left_matcher = cv2.StereoBM_create(max_disp, wsize);
right_matcher = cv2.ximgproc.createRightMatcher(left_matcher);
left_disp = left_matcher.compute(left_image, right_image);
right_disp = right_matcher.compute(right_image,left_image);

# Now create DisparityWLSFilter
wls_filter = cv2.ximgproc.createDisparityWLSFilter(left_matcher);
wls_filter.setLambda(lmbda);
wls_filter.setSigmaColor(sigma);
filtered_disp = wls_filter.filter(left_disp, left_image, disparity_map_right=right_disp);

Voltmeter

What version of OpenCV is this for? Did you learn this from official python documentation? Where can I find that? I'm on version 4.1.1 and cv2 has no attribute ximgproc – Apperceive

Can you also give an example of using WLS for guided filtering, e.g. using rgb image to guide the filtering of a depth map? (something like Fig 10 in kaiminghe.com/publications/pami12guidedfilter.pdf) – Housefather

@ABlankenship: It took me a bit to figure out, but you have to install the package that comes with the extra OpenCV modules by doing pip install opencv-contrib-python. – Demoniac

It seems that you do not need to install the community contrib package anymore for OpenCV-4.9.0, as the ximgproc module has been merged upstream. – Jurisprudence