OpenCV Surf and Outliers detection

Question

I know there are already several questions with the same subject asked here, but I couldn't find any help.

So I want to compare 2 images in order to see how similar they are and I'm using the well known find_obj.cpp demo to extract surf descriptors and then for the matching I use the flannFindPairs.

But as you know this method doesn't discard the outliers and I'd like to know the number of true positive matches so I can figure how similar those two images are.

I have already seen this question: Detecting outliers in SURF or SIFT algorithm with OpenCV and the guy there suggests to use the findFundamentalMat but once you get the fundamental matrix how can I get the number of outliers/true positive from that matrix? Thank you.

Biquarterly

Answer 1

Here is a snippet from the descriptor_extractor_matcher.cpp sample available from OpenCV:

if( !isWarpPerspective && ransacReprojThreshold >= 0 )
    {
        cout << "< Computing homography (RANSAC)..." << endl;
        vector<Point2f> points1; KeyPoint::convert(keypoints1, points1, queryIdxs);
        vector<Point2f> points2; KeyPoint::convert(keypoints2, points2, trainIdxs);
        H12 = findHomography( Mat(points1), Mat(points2), CV_RANSAC, ransacReprojThreshold );
        cout << ">" << endl;
    }

    Mat drawImg;
    if( !H12.empty() ) // filter outliers
    {
        vector<char> matchesMask( filteredMatches.size(), 0 );
        vector<Point2f> points1; KeyPoint::convert(keypoints1, points1, queryIdxs);
        vector<Point2f> points2; KeyPoint::convert(keypoints2, points2, trainIdxs);
        Mat points1t; perspectiveTransform(Mat(points1), points1t, H12);

        double maxInlierDist = ransacReprojThreshold < 0 ? 3 : ransacReprojThreshold;
        for( size_t i1 = 0; i1 < points1.size(); i1++ )
        {
            if( norm(points2[i1] - points1t.at<Point2f>((int)i1,0)) <= maxInlierDist ) // inlier
                matchesMask[i1] = 1;
        }
        // draw inliers
        drawMatches( img1, keypoints1, img2, keypoints2, filteredMatches, drawImg, CV_RGB(0, 255, 0), CV_RGB(0, 0, 255), matchesMask
#if DRAW_RICH_KEYPOINTS_MODE
                     , DrawMatchesFlags::DRAW_RICH_KEYPOINTS
#endif
                   );

#if DRAW_OUTLIERS_MODE
        // draw outliers
        for( size_t i1 = 0; i1 < matchesMask.size(); i1++ )
            matchesMask[i1] = !matchesMask[i1];
        drawMatches( img1, keypoints1, img2, keypoints2, filteredMatches, drawImg, CV_RGB(0, 0, 255), CV_RGB(255, 0, 0), matchesMask,
                     DrawMatchesFlags::DRAW_OVER_OUTIMG | DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );
#endif
    }
    else
        drawMatches( img1, keypoints1, img2, keypoints2, filteredMatches, drawImg );

The key lines for the filtering are performed here:

if( norm(points2[i1] - points1t.at<Point2f>((int)i1,0)) <= maxInlierDist ) // inlier
                matchesMask[i1] = 1;

Which is measuring the L2-norm distance between the points (either 3 pixels if nothing was specified, or user-defined number of pixels reprojection error).

Hope that helps!

Reuben

thank you for your answer... I can't use directly this code in the find_obj.cpp since that is coded in C and this snippet is C++. However I have tried it inside the SURFFlannMatcher.cpp demo and it doesn't really give good results. For example, if I get 2 pictures of the same object, same scene, that is to say almost identical pictures but different resolutions, the findHomography function will find only outliers and no inliers... this is very strange. – Biquarterly

Answer 2

you can use the size of the vector named "ptpairs" in order to decide how similiar the pictures are. this vector contains the matching keypoints, so his size/2 is the number of matches. i think you can use the size of ptpairs divided by the total number of keypoints in order to set an appropriate threshold. this will probably give you an estimation to the similiarty between them.

Blaisdell