Detecting clusters of white pixels in an image using openCV

Question

Input Image:

Expected Output:

I intend to fit three (or some number of) polygons (for this case, rectangles) to signify the "big" white blobs in this image. The rectangles drawn in the output image are as per my perception of the white regions. I do not expect the algorithm to come up with these same bouding regions. What I wish for is to fit some number of tight polygons around the clusters of white pixels.

My initial solution consisted of finding contours for this image, and fitting a closed convex polygon around each contour, by finding the convex hull of the points in each contour.

However, since the white regions are highly fragmented with black regions within and ridged around the edges, the number of contours returned by cv2.findContours is very high (around 500 or so). Due to this, fitting a convex hull does not improve the shape of the white regions. The white regions mostly retain their original abstract shapes. My goal would be to merge the many small contours of a white region into one whole containing contour over which I can then fit a convex hull.

How do I solve this problem? Should I use a clustering algorithm on the contour points initially to find the contours that are close by each other?

Harriet

en.wikipedia.org/wiki/DBSCAN may help. under python scikit. demo here scikit-learn.org/stable/auto_examples/cluster/plot_dbscan.html – Suint

Answer 1

You first need to perform morphological-closing(which is dilation followed by erosion) on this image. This closes all the tiny "holes" your image has while preserving the shape and size of the individual components. Opposed to it, when erosion is followed by dilation, it removes the noisy dots in the image. I am working on a similar image and I had to perform dilation+erosion as much as 10 times to even out my components. After you do it, use connected components or find contours. This will certainly bring the contour count down from 400 to 20-30.

Secondly, you mentioned you need 3 clusters. Though the two little clusters(covered by the red line) could have merged into one. What I made out of it was that you want each of your cluster to be as tightly fitting into its bounding rectangle as possible. So, I would suggest you to set a threshold efficiency (say 80%) and use hierarchical clustering to merge each connected component into a cluster. When your white pixels exert less than 80% of space of their bounding rectangle(of a cluster), you would stop the clustering and get the clusters.

Hydraulic

Answer 2

You could first dilate the image before finding contours. Dilation causes bright regions to grow. You can think of it as adding white pixels around every existing white pixel in your image. This way the neighboring bright shapes get merged. See http://docs.opencv.org/doc/tutorials/imgproc/erosion_dilatation/erosion_dilatation.html

You can also blur and threshold again, but blurring can be a lot more costly than dilation depending on the amount of blur.

Burglar

Answer 3

You could use kmeans clustering using the x y coordinate as the feature for each white point and three clusters. Then take the convex hull of the the resulting three clusters. You may have to try different starting points and select the best result. See http://docs.opencv.org/modules/core/doc/clustering.html#kmeans

Illumination

Answer 4

You can draw approximated contours around your shapes until you get all the needed regions connected. With this I am effectively eroding the image. If you draw a hull around those connected regions, you get your red rectangles.

Just repeat until your biggest three hulls have some required property (for example if they cover 99% of all white dots)

#include <vector>
using std::vector;
#include <algorithm>
using std::sort;
#include <string>
using std::string;
using std::to_string;
#include <iostream>
using std::clog;
using std::endl;
#include <opencv2/opencv.hpp>
using namespace cv;

int main()
{
  typedef vector<Point> Polygon;
  typedef vector<Polygon> Polygons;
  Mat mFrame;
  Mat mOrig;
  mFrame = imread("R2TsZ.png");
  mFrame.copyTo(mOrig);
  Mat mOrigHull;
  Mat mOut;
  int fileCounter = 0;
  while(true){
    clog<< "image read"<< endl;
    cvtColor(mFrame, mOut, CV_BGR2GRAY);
    clog<< "image grayscaled"<< endl;
    Polygons contours;
    Polygons aContours;
    Polygons hulls;
    OutputArray hierarchy = {};

    findContours(mOut, contours, hierarchy, RETR_EXTERNAL, CHAIN_APPROX_SIMPLE);
    clog<< contours.size()<< " contours found"<< endl;
    sort(contours.begin(), contours.end(), [](auto p1, auto p2){
      return contourArea(p1) > contourArea(p2);
    });
    clog<< "contours sorted"<< endl;
    aContours.resize(contours.size());
    hulls.resize(contours.size());
    for(size_t i = 0; i < aContours.size() - 1; ++ i){
      approxPolyDP(contours[i], aContours[i], 20, true);
      drawContours(mFrame, aContours, i, Scalar(255, 255, 255), 10);
      convexHull(aContours[i], hulls[i], true);
    }
    mOrig.copyTo(mOrigHull);
    for(size_t i = 0; i < 3; ++ i){
      drawContours(mOrigHull, hulls, i, Scalar(0, 0, 255), 10);
    }
    imshow("out", mOrigHull);
    int key = waitKey() & 0xff;
    if(key == 27){
      return EXIT_SUCCESS;
    }
    if(key == 'p'){
      string file = "test_" + to_string(++ fileCounter) + ".png";
      imwrite(file, mOrigHull);
      clog<< file<< " saved."<< endl;
    }
  }
}

See more in this tutorial from opencv.

Botts

Any implementation in python? – Ember

@Dev I used the approxPolyDP function from the opencv tutorial which I gave a link to. The example in that link is in python. – Botts