OpenCV: Extract SURF Features from user-defined keypoints

Question

I want to compute SURF Features from keypoints that I specify. I am using the Python wrapper of OpenCV. The following is the code I am trying to use, but I cannot find a working example anywhere.

surf = cv2.SURF()
keypoints, descriptors = surf.detect(np.asarray(image[:,:]),None,useProvidedKeypoints = True)

How can I specify the keypoints to be used by this function?

Similar, unanswered, question: cvExtractSURF don't work when useProvidedKeypoints = true

Documentation

Downspout

Did you mange to get it working in the end ? – Koodoo

I did and I even posted the answer here, but I just noticed that it was deleted for some reason. Strange. Anyway, you can use Mahotas to do this, or perhaps check out some of the other answers that have been posted in the meantime. – Downspout

Answer 1

Try using cv2.DescriptorMatcher_create for that.

For instance, in the following code I am using pylab, but you can get the message ;)

It computes the keypoints using GFTT, and then uses the SURF descriptor and the Brute force matching. The output of each code part is show as header.

---

%pylab inline
import cv2
import numpy as np

img = cv2.imread('./img/nail.jpg')
gray= cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
imshow(gray,  cmap=cm.gray)

Output is something like this https://i.sstatic.net/8eOTe.png

(For this example I will cheat and use the same image to get the keypoints and descriptors).

img1 = gray
img2 = gray
detector = cv2.FeatureDetector_create("GFTT")
descriptor = cv2.DescriptorExtractor_create("SURF")
matcher = pt1=(int(k1[m.queryIdx].pt[0]),int(k1[m.queryIdx].pt[1]))("FlannBased")

# detect keypoints
kp1 = detector.detect(img1)
kp2 = detector.detect(img2)

print '#keypoints in image1: %d, image2: %d' % (len(kp1), len(kp2))

keypoints in image1: 1000, image2: 1000

# descriptors
k1, d1 = descriptor.compute(img1, kp1)
k2, d2 = descriptor.compute(img2, kp2)

print '#Descriptors size in image1: %s, image2: %s' % ((d1.shape), (d2.shape))

Descriptors size in image1: (1000, 64), image2: (1000, 64)

# match the keypoints
matches = matcher.match(d1,d2)

# visualize the matches
print '#matches:', len(matches)
dist = [m.distance for m in matches]

print 'distance: min: %.3f' % min(dist)
print 'distance: mean: %.3f' % (sum(dist) / len(dist))
print 'distance: max: %.3f' % max(dist)

matches: 1000

distance: min: 0.000

distance: mean: 0.000

distance: max: 0.000

# threshold: half the mean
thres_dist = (sum(dist) / len(dist)) * 0.5 + 0.5

# keep only the reasonable matches
sel_matches = [m for m in matches if m.distance < thres_dist]

print '#selected matches:', len(sel_matches)

selected matches: 1000

#Plot
h1, w1 = img1.shape[:2]
h2, w2 = img2.shape[:2]
view = zeros((max(h1, h2), w1 + w2, 3), uint8)
view[:h1, :w1, 0] = img1
view[:h2, w1:, 0] = img2
view[:, :, 1] = view[:, :, 0]
view[:, :, 2] = view[:, :, 0]

for m in sel_matches:
    # draw the keypoints
    # print m.queryIdx, m.trainIdx, m.distance
    color = tuple([random.randint(0, 255) for _ in xrange(3)])
    pt1=(int(k1[m.queryIdx].pt[0]),int(k1[m.queryIdx].pt[1]))
    pt2=(int(k2[m.queryIdx].pt[0]+w1),int(k2[m.queryIdx].pt[1]))
    cv2.line(view,pt1,pt2,color)

Output is something like this https://i.sstatic.net/8CqrJ.png

Koodoo

Answer 2

If I understand the source code of the Python bindings correctly, the "keypoints" argument that is present in the C++ interface is never used in the Python bindings. So I hazard that it's not possible to do what you are trying to do with the current bindings. A possible solution would be to write your own binding. I know it's not the answer you were hoping...

Intercurrent

I was starting to suspect the same actually... I have started looking into using a Python library for SURF, such as Python Mahotas – Downspout

It shouldn't be too hard to write your own binding to your own custom function though. – Stephanotis

Author of mahotas here: mahotas can do what you want. – Dispense

Answer 3

Example of how this can be done with the before-mentioned Mahotas:

import mahotas
from mahotas.features import surf
import numpy as np


def process_image(imagename):
    '''Process an image and returns descriptors and keypoints location'''
    # Load the images
    f = mahotas.imread(imagename, as_grey=True)
    f = f.astype(np.uint8)

    spoints = surf.dense(f, spacing=12, include_interest_point=True)
    # spoints includes both the detection information (such as the position
    # and the scale) as well as the descriptor (i.e., what the area around
    # the point looks like). We only want to use the descriptor for
    # clustering. The descriptor starts at position 5:
    desc = spoints[:, 5:]
    kp = spoints[:, :2]

    return kp, desc

Hellen