best way to segment a tree in plantation aerial image using opencv

Question

so i want to segment a tree from an aerial image

sample image (original image) :

and i expect the result like this (or better) :

the first thing i do is using threshold function in opencv and i didn't get expected result (it cant segment the tree crown), and then i'm using black and white filter in photoshop using some adjusted parameter (the result is shown beloww) and do the threshold and morphological filter and got result like shown above.

my question, is there a some ways to do the segmentation to the image without using photoshop first, and produce segmented image like the second image (or better) ? or maybe is there a way to do produce image like the third image ?

ps: you can read the photoshop b&w filter question here : https://dsp.stackexchange.com/questions/688/whats-the-algorithm-behind-photoshops-black-and-white-adjustment-layer

Hurty

Answer 1

You can do it in OpenCV. The code below will basically do the same operations you did in Photoshop. You may need to tune some of the parameters to get exactly what you want.

#include "opencv2\opencv.hpp"
using namespace cv;

int main(int, char**)
{
    Mat3b img = imread("path_to_image");

    // Use HSV color to threshold the image
    Mat3b hsv;
    cvtColor(img, hsv, COLOR_BGR2HSV);

    // Apply a treshold
    // HSV values in OpenCV are not in [0,100], but:
    // H in [0,180]
    // S,V in [0,255]

    Mat1b res;
    inRange(hsv, Scalar(100, 80, 100), Scalar(120, 255, 255), res);

    // Negate the image
    res = ~res;

    // Apply morphology 
    Mat element = getStructuringElement( MORPH_ELLIPSE, Size(5,5));
    morphologyEx(res, res, MORPH_ERODE, element, Point(-1,-1), 2);
    morphologyEx(res, res, MORPH_OPEN, element);

    // Blending
    Mat3b green(res.size(), Vec3b(0,0,0));
    for(int r=0; r<res.rows; ++r) {
        for(int c=0; c<res.cols; ++c) {
            if(res(r,c)) { green(r,c)[1] = uchar(255); }
        }
    }

    Mat3b blend;
    addWeighted(img, 0.7, green, 0.3, 0.0, blend);

    imshow("result", res);
    imshow("blend", blend);
    waitKey();

    return 0;
}

The resulting image is:

The blended image is:

Cai

It would be interesting to see the outline of the segmented objects superimposed on the original. – Steib

Answer 2

This has been an interesting topic of research in the past - mainly in the remote sensing literature.

While the morphological methods proposed using OpenCV will work in certain cases, you might want to consider more sophisticated approaches (depending on how variable your data is and how robust a detector you want to build).

For example, this paper, and those who cite it - give you a flavour of what has been attempted.

Pragmatically speaking - I think a neat solution would be one founded more on statistical texture analysis. There are many ways to classify (and then count) regions of an image as belong to a texture (co-occurance matrices, filter banks, textons, wavelets, etc, etc.).

Sadly, this is an area where OpenCV is rather deficient - it only provides a subset of the useful algorithms out there... However, here are a few quick ideas (none of which I have tried directly, just what I'm aware of are based on underlying OpenCV):

• Use OpenCV Gabor filter support and cluster (for example).
• You could also possibly train an OpenCV SVM with Local Binary Patterns.
• A new library - but probably not so relevant for static images - LIBDT

Anyways, I hope you get something that just works for your purposes!

Appose

i have consider about the statistical texture analysis but i'm afraid if the spaces between trees is very small (dense) it will be very hard to do if i'm using observing window to iterate trough the image (or there are any other method than using moving window ?). or should i do a segementation first ? and thats also my problem i have tried some segmentation algorithm and it is not very good, till i use photoshop's b&w filter. – Hurty