feature detectors and descriptors comparison [closed]

Question

There are several kinds of detectors and descriptors, like SIFT, SURF, FAST. I wonder are they all eligible for real-time applications? Which is the best or better?

And furthermore, is Harris-Laplacian dectector still useful when we already have the above three? Is it better than them?

Squib

SIFT and SURF are relatively slow descriptors when compared to binary descriptors such as BRISK, BRIEF, ORB and FREAK. I'm not sure about their speed in detecting (rather then description), maybe someone else can answer that. Also, SIFT and SURF are much better in their precision and recall curves (meaning, their performance), compared to binary descriptors. If you interested, I can give you a link to a post that gives an introduction to binary descriptors. – Dissever

@GilLevi, thank you so much, you're a CV major, :-). Actually, I'm having a hard time to make a choice between (SIFT,SURF,FAST) or (harris-laplacian). I don't quite familiar with them, but according to some articles, harris-laplacian seems to be good in terms of feature detection, but it might be slower than the other 3, right? Could you please help me with that? – Squib

You're interested in a detector or a descriptor? If you're concerned about speed, why don't you just check for yourself? I can refer you to some code that I think you can easily modify to check for running times. – Dissever

@GilLevi, well, I actually try to implement a detector from scratch, I have a basic understanding of harris-laplacian and SIFT, I just want to know whether harris-laplacian is better than SIFT in terms of feather detecting. – Squib

Sorry, I'm really not sure which one - SIFT or Harris detects better keypoints. – Dissever

Answer 1

I can advise you to use Hessian-Affine and MSER for detection, if you need invariance to different factors (e.g., viewpoint change) or FAST, if you need real time. FAST is doing similar job to the Harris, but much faster.

You can look into "Local Invariant Feature Detectors: A Survey", and "A Comparison of Affine Region Detectors" where many detectors are tested and described very well.

Update: "WxBS: Wide Baseline Stereo Generalizations" does extended benchmark of the novel and classical detectors and descriptors.

Second, the description part is usually slower than detection, so to be real-time you have to use GPU or binary descriptor like BRIEF or FREAK.

Update2: "HPatches (Homography Patches) dataset and benchmark" and corresponding workshop at ECCV 2016. http://www.iis.ee.ic.ac.uk/ComputerVision/DescrWorkshop/index.html .

Update3: "Comparative Evaluation of Hand-Crafted and Learned Local Features" Descriptors (and a bit detectors) evaluation on large-scale 3D reconstruction task CVPR 2017 .

Update4: "Interest point detectors stability evaluation on ApolloScape dataset" Detector evaluation on authonomous driving dataset, ECCVW2018 .

Update5: "From handcrafted to deep local invariant features" Huuuge survey-overview paper about handcrafted and learned features, 2018.

Update6: "Image Matching across Wide Baselines: From Paper to Practice" Large scale benchmark of the abovementioned and more recent methods for the camera pose estimation. IJCV, 2020.

Antoninaantonino

Thanks, but why not SIFT/SURF? – Squib

Because SIFT and SURF as detectors are only similarity covariant (i.e. "immune" to rotation, translation and scale changes, but not affine covariant - their performance degrades fast with change of viewpoint). Standard practice in image matching - to use MSER and or Hessian-Affine as detector + SIFT as descriptor. Also SIFT and SURF are slow (as well as Hessian-Affine and less - MSER) compared to the very fast FAST keypoints. – Antoninaantonino

@Antoninaantonino I came across your answer while researching feature detectors comparisons. I'm looking to implement facial landmark detection (location of corner of mouth, eyes, nose etc) and was evaluating the various detectors / descriptors. It seems your reason for not using SIFT / SURF do not apply considering I will be using a training / test set where the image is a frontal shot of the subject (i.e change of viewpoint not important). Do you have any suggestions on what might be some good detectors / descriptors and why? Most literature on the subject is outdated by a couple years...Thanks! – Tierratiersten

I`d say, that "couple of years" is not "outdated". I recommend to begin with ics.uci.edu/~xzhu/face and facebook.com/publications/546316888800776. – Antoninaantonino

Answer 2

My speed test for point detectors in OpenCV 3.0 using AVT Manta G-125C (1292x964)

Unit is FPS

1. Free running (no processing): 21.3
2. FAST (nonmaxima suppression: true): 16-21
3. FAST (nonmaxima suppression: false): 12.8
4. ORB: 12.8
5. GoodFeaturesToTrack: 9
6. BRISK: 8-9.2

Hundredweight