Detect more than 1 object on picture

Question

I wrote a little script which let to find an object in a global picture by the SIFT descriptors method. But I have a question about multiple detections in the same picture.

I have this global picture :

I have this template :

My script looks like :

import numpy as np
import cv2

#########################
# SIFT descriptors part #
#########################

img1 = cv2.imread('/Users/valentinjungbluth/Desktop/SIFT:SURF Algo/lampe.jpg',0)
img2 = cv2.imread('/Users/valentinjungbluth/Desktop/SIFT:SURF Algo/ville.jpg',0)

# Initiate SIFT detector
sift = cv2.xfeatures2d.SIFT_create()

print (img1.dtype)
print (img2.dtype)


kp1, des1 = sift.detectAndCompute(img1,None)
kp2, des2 = sift.detectAndCompute(img2,None)

bf = cv2.BFMatcher()
matches = bf.knnMatch(des1,des2,k=2)

good = []
for m,n in matches :
    if m.distance < 0.2*n.distance :
        good.append([m])

img3 = cv2.drawMatchesKnn(img1,kp1,img2,kp2,good,None,flags=2)

cv2.imwrite('matches.jpg',img3)

And the result is :

My question is :

How I can detect this others lamps ? Because all lamps are very similar and I want to match with all lamps which are present in the picture.

Thank you so much !

EDIT With Micka's answer :

Nothing appears at 0.2 scale distance, but if I put 0.75 :

Diminished

hard to explain, but maybe you can use an approach similar to the one I used in this link and the link in this link: #34311414 but I guess that the other lamps just don't look similar enough in your case. You could try to remove the rightmost lamp form your image by overdrawing it and start your detection on the modified image. – Barhorst

@Barhorst Thank you for your answer. To my mind, the other lamps are very similar. So I need to find how I can show to my script the other lamps in order to take account. – Diminished

can you try your algorithm with your posted template in this image and post the result? picload.org/image/ralgwllg/sift_smaller.png – Barhorst

Sure, I edited my post ;) – Diminished

Answer 1

This is a good question. There are couple of ways I can think of doing this:

1.Sliding Windowing technique - You can search for the "template" in the global image by making a window, the size of the template, and sliding it in the entire image. You can do this for a pyramid so the scale and translational changes are taken care of.

2. SIFT - Try matching the global image with the template and find all matches. Then you should filter the matches with relative pose. May be you'd require another filtering but I think this method is more general as it caters for more constraints than the previous.

Hope it helps!

Smallish

Yeah and I used the SIFT method to do that. But, even if I get the template from the global image (I cropped all around a lamp), the other lamps are very similar but not detected ^^ – Diminished

I see that you are using SIFT method, but in knnmatch, match the global with the template and not the other way around. That will give you all matches from global to template and then you can filter them. The MatOfKeypoint container gives you 7 different scores which you can compare to filter for relative pose. – Smallish

Thank you Rick ! But MatOfKeypoint is a C++ function right ? I have to see if it exists a similar function in Python ? – Diminished

Yes it unfortunately doesn't exist for python. But I guess in python you do get something similar to it may be this – Smallish

Answer 2

Try to allow more good matches by being more permissive in the condition.

good = []
for m,n in matches :
    if m.distance < 0.2*n.distance :
        good.append([m])

A more robust approach would be to describe the lamp using the sift features extracted from the template image(s), and then try to find those features using a sliding window over the image. For each window, compute the sift features, and compute a "distance" to your template's features. If the distance is smaller than a given threshold, then the window contains a lamp!

Oriente

Thank you to your answer. But it's very strange because when I allow more good matches with distance modification, I don't find more lamp :/ I didn't mention it, but it's the first time I'm making this kind of things . – Diminished

i'll point you to a good python+opencv reference for the sliding window: pyimagesearch.com/2015/03/23/… – Oriente

Thank you, I will look this tutorial in order to make the same thing. Hopefully to get multiple detection :) – Diminished