OpenCV: How to visualize a depth image

Question

I am using a dataset in which it has images where each pixel is a 16 bit unsigned int storing the depth value of that pixel in mm. I am trying to visualize this as a greyscale depth image by doing the following:

cv::Mat depthImage; 
depthImage = cv::imread("coffee_mug_1_1_1_depthcrop.png", CV_LOAD_IMAGE_ANYDEPTH | CV_LOAD_IMAGE_ANYCOLOR ); // Read the file 
depthImage.convertTo(depthImage, CV_32F); // convert the image data to float type   
namedWindow("window");
float max = 0;
for(int i = 0; i < depthImage.rows; i++){
    for(int j = 0; j < depthImage.cols; j++){
        if(depthImage.at<float>(i,j) > max){
            max = depthImage.at<float>(i,j);
        }
    }   
}
cout << max << endl;


float divisor = max / 255.0;
cout << divisor << endl;
for(int i = 0; i < depthImage.rows; i++){
    for(int j = 0; j < depthImage.cols; j++){
        cout << depthImage.at<float>(i,j) << ", ";
        max = depthImage.at<float>(i,j) /= divisor;
        cout << depthImage.at<float>(i,j) << endl;
    }   
}


imshow("window", depthImage);
waitKey(0);

However, it is only showing two colours this is because all of the values are close together i.e. in the range of 150-175 + the small values which show up black (see below).

Is there a way to normalize this data such that it will show various grey levels to highlight these small depth differences?

Egestion

Answer 1

According to the documentation, the function imshow can be used with a variety of image types. It support 16-bit unsigned images, so you can display your image using

cv::Mat map = cv::imread("image", CV_LOAD_IMAGE_ANYCOLOR | CV_LOAD_IMAGE_ANYDEPTH);
cv::imshow("window", map);

In this case, the image value range is mapped from the range [0, 255*256] to the range [0, 255].

If your image only contains values on the low part of this range, you will observe an obscure image. If you want to use the full display range (from black to white), you should adjust the image to cover the expected dynamic range, one way to do it is

double min;
double max;
cv::minMaxIdx(map, &min, &max);
cv::Mat adjMap;
cv::convertScaleAbs(map, adjMap, 255 / max);
cv::imshow("Out", adjMap);

Ironworks

I dont see why scaling it by 255/max (sam as me dividing every element by max/255) will make it use the full range. I mean it does, and I will accept the answer but I just don't get it. What else is that function doing? – Egestion

The convertScaleAbs function performs 3 operations: scale, compute absolute value, and convert to unsigned 8-bit type. That's why the factor 255/max ensures the full range ([0-255] for unsigned 8-bit) is used. Moreover, as @sammy has mentioned, the dynamic range of the adjusted image is better used by taking into account the minimum value of your data. – Ironworks

Answer 2

Adding to samg' answer, you can expand even more the range of your displayed image.

double min;
double max;
cv::minMaxIdx(map, &min, &max);
cv::Mat adjMap;
// expand your range to 0..255. Similar to histEq();
map.convertTo(adjMap,CV_8UC1, 255 / (max-min), -min); 

// this is great. It converts your grayscale image into a tone-mapped one, 
// much more pleasing for the eye
// function is found in contrib module, so include contrib.hpp 
// and link accordingly
cv::Mat falseColorsMap;
applyColorMap(adjMap, falseColorsMap, cv::COLORMAP_AUTUMN);

cv::imshow("Out", falseColorsMap);

The result should be something like the one below

Harmonic

is it possible for openCV to output a colormap with the scale, as you have shown on the right? – Sadi

No. That is a Matlab plot. – Harmonic

Cheers Sammy, appreciate the prompt response – Sadi

The 4th parameter should be -255*min/(max-min) instead of -min. It took me half an hour to debug this out. Please correct it. – Spike

when I did it for a CV_32FC3 I got a fatal exception which broke on extern "C" LONG WINAPI __scrt_unhandled_exception_filter(LPEXCEPTION_POINTERS const pointers) { auto const exception_record = reinterpret_cast<EHExceptionRecord*>(pointers->ExceptionRecord); if (PER_IS_MSVC_PURE_OR_NATIVE_EH(exception_record)) { terminate(); } return EXCEPTION_CONTINUE_SEARCH; } would it work for cv::Mat of type CV_32FC3 as well? – Quarterback

Answer 3

Ifimshow input has floating point data type then the function assumes that pixel values are in [0; 1] range. As result all values higher than 1 are displayed white.

So you need not divide your divisor by 255.

Habit

After these lines: depthImage = cv::imread("coffee_mug_1_1_1_depthcrop.png", CV_LOAD_IMAGE_ANYDEPTH | CV_LOAD_IMAGE_ANYCOLOR ); depthImage.convertTo(depthImage, CV_32F); If I print the values they are in the range [0,1161] so my divisor is 1161/255 to get all values in range [0,255], perhaps if I then convert is to CV_8UC1 ? – Egestion

Ah yes, this worked. Plus if I use equalize histogram it gives a much better representation – Egestion

First, imread function reads you image preserving original values, so 1161 is Ok for 16 bits per pixel image. Second, convertTo method does not scale values by default, it only change type and saturates. So this explains why printed values are so big. – Habit

Answer 4

Adding to Sammy answer, if the original range color is [-min,max] and you want to perform histogram equalization and display the Depth color, the code should be like below:

double min;
double max;
cv::minMaxIdx(map, &min, &max);
cv::Mat adjMap;
// Histogram Equalization
float scale = 255 / (max-min);
map.convertTo(adjMap,CV_8UC1, scale, -min*scale); 

// this is great. It converts your grayscale image into a tone-mapped one, 
// much more pleasing for the eye
// function is found in contrib module, so include contrib.hpp 
// and link accordingly
cv::Mat falseColorsMap;
applyColorMap(adjMap, falseColorsMap, cv::COLORMAP_AUTUMN);

cv::imshow("Out", falseColorsMap);

Patriotism

Dear, is it Histogram Equalization or just Contrast Enhancement? – Monogenetic