How do I use SIFT in OpenCV 3.0 with c++?

Question

I have OpenCV 3.0, and I have compiled & installed it with the opencv_contrib module so that's not a problem. Unfortunately the examples from previous versions do not work with the current one, and so although this question has already been asked more than once I would like a more current example that I can actually work with. Even the official examples don't work in this version (feature detection works but not other feature examples) and they use SURF anyway.

So, how do I use OpenCV SIFT on C++? I want to grab the keypoints in two images and match them, similar to this example, but even just getting the points and descriptors would be enough help. Help!

Spleeny

Answer 1

1. get the opencv_contrib repo
2. take your time with the readme there, add it to your main opencv cmake settings
3. rerun cmake /make / install in the main opencv repo

then:

   #include "opencv2/xfeatures2d.hpp"

  // 
  // now, you can no more create an instance on the 'stack', like in the tutorial
  // (yea, noticed for a fix/pr).
  // you will have to use cv::Ptr all the way down:
  //
  cv::Ptr<Feature2D> f2d = xfeatures2d::SIFT::create();
  //cv::Ptr<Feature2D> f2d = xfeatures2d::SURF::create();
  //cv::Ptr<Feature2D> f2d = ORB::create();
  // you get the picture, i hope..

  //-- Step 1: Detect the keypoints:
  std::vector<KeyPoint> keypoints_1, keypoints_2;    
  f2d->detect( img_1, keypoints_1 );
  f2d->detect( img_2, keypoints_2 );

  //-- Step 2: Calculate descriptors (feature vectors)    
  Mat descriptors_1, descriptors_2;    
  f2d->compute( img_1, keypoints_1, descriptors_1 );
  f2d->compute( img_2, keypoints_2, descriptors_2 );

  //-- Step 3: Matching descriptor vectors using BFMatcher :
  BFMatcher matcher;
  std::vector< DMatch > matches;
  matcher.match( descriptors_1, descriptors_2, matches );

also, don't forget to link opencv_xfeatures2d !

Cooperstein

In the documentation detect and compute only appears as python functions. Is the documentation incomplete? Is the SIFT class better documented anywhere? And is the SIFT::operator() unusable or am I just doing something horribly wrong? Edit: the create(...) function isn't anywhere that I've looked in the online documentation... I guess I'm either looking in all the wrong places or I'm just going to have to do without the dox. – Spleeny

operator() is still there, but using that from a pointer, would make it something like f2d->operator(...). better use: f2d->detectAndCompute(...) then. (and yea, the docs need a fix urgently...) – Cooperstein

The edit helps. After a bit of research I'd like to add that you can also use detectAndCompute(img, mask, keypoints, descriptors) and that you can use noArray() on that if you don't want a mask. – Spleeny

Also for future reference, you can use drawMatches() to actually see the matches. – Spleeny

@Cooperstein How do you link opencv_xfeature2d? – Celandine

Answer 2

There are useful answers, but I'll add my version (for OpenCV 3.X) just in case the above ones aren't clear (tested and tried):

1. Clone opencv from https://github.com/opencv/opencv to home dir
2. Clone opencv_contrib from https://github.com/opencv/opencv_contrib to home dir
3. Inside opencv, create a folder named build
4. Use this CMake command, to activate non-free modules: cmake -DOPENCV_EXTRA_MODULES_PATH=/home/YOURUSERNAME/opencv_contrib/modules -DOPENCV_ENABLE_NONFREE:BOOL=ON .. (Please notice that we showed where the contrib modules resides and also activated the nonfree modules)
5. Do make and make install afterwards

The above steps should work out for OpenCV 3.X

After that, you may run the below code using g++ with the appropriate flags:

g++ -std=c++11 main.cpp `pkg-config --libs --cflags opencv` -lutil -lboost_iostreams -lboost_system -lboost_filesystem -lopencv_xfeatures2d -o surftestexecutable

The important thing not to forget is to link the xfeatures2D library with -lopencv_xfeatures2d as shown on the command. And the main.cpp file is:

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include "opencv2/xfeatures2d.hpp"
#include "opencv2/xfeatures2d/nonfree.hpp"

using namespace cv;
using namespace std;

int main(int argc, const char* argv[])
{

    const cv::Mat input = cv::imread("surf_test_input_image.png", 0); //Load as grayscale

    Ptr< cv::xfeatures2d::SURF> surf =  xfeatures2d::SURF::create();
    std::vector<cv::KeyPoint> keypoints;
    surf->detect(input, keypoints);

    // Add results to image and save.
    cv::Mat output;
    cv::drawKeypoints(input, keypoints, output);
    cv::imwrite("surf_result.jpg", output);


    return 0;
}

This should create and save an image with surf keypoints.

Nonobservance