scale and rotation Template matching

Question

I'm using the method of match template with CV_TM_CCORR_NORMED to compare two images ... I want to make to make this rotation and scale invariant .. any ideas?

I tried to use the same method on the fourier transform of the image and the template , but still the result after rotation is different

Dis

Answer 1

Template matching with matchTemplate is not good when your object is rotated or scaled in scene.

You should try openCV function from Features2D Framework. For example SIFT or SURF descriptors, and FLANN matcher. Also, you will need findHomography method.

Here is a good example of finding rotated object in scene.

Update:

In short, algorithm is this:

1. Finding keypoints of your object image 1.1. Extracting descriptors from those keypoints

2. Finding keypoints of your scene image 2.1 Extracting descriptors from keypoints

3. Match descriptors by matcher

4. Analyze your matches

There are different classes of FeatureDetectors, DescriptorExtractors, and DescriptorMatches, you may read about them and choose those, that fit good for your tasks.

• openCV FeatureDetector (steps 1 and 2 in algorithm above)
• openCV DescriptorExtractor ( steps 1.1 and 2.1 in algorithm above )
• openCV DescriptorMatcher ( step 3 in algorithm above )

Interrupted

**Thank you, I already used the homography and instead of surf I found the corners as the points of interest , will this work with FLANN? surf and sift may take long time and reduce the performance ** – Dis

Points of interest is general term. They are edges, and mostly corners. How did you find points of interest? You can use any of openCV FeatureDetectors, or write your own. There are FAST, STAR for example. They are simple and fast. But they don't keep information about angle of points of interest. SURF and SIFT are slower, but their information about points of interests are bigger (angles). SIFT and SURF allow you to find scale-invariant matches. But you can use any openCV detectors for your matcher. You may also try different matchers (there are few in openCV) – Interrupted

Note that the sift/surf/etc. methods are no good for objects with little features. E.g. homogeneous objects. They need suffient corners and edges in the template. – Rimini

Answer 2

There are easier ways of matching a template scale and rotationally invariant than going via feature detection and homographies (if you know its really only rotated and scales, but everything else is constant). For true object detection the above suggested keypoint based approaches work better.

If you know it's the same template and there is no perspective change involved, you take an image pyramid for scale-space detection, and match your templates on the different levels of that pyramid (via something simple, for example SSD or NCC). It will be cheap to find rough matches on higher (= lower resolution) levels of the pyramid. In fact, it will be so cheap, that you can also rotate your template roughly on the low resolution levels, and when you trace the template back down to the higher resolution levels, you use a more finely grained rotation stepping. That's a pretty standard template matching technique and works well in practice.

Worrisome

I know its been 7 years since you posted this but ive been trying to find a way to do this and I need some help. Can you suggest a path I can take to implement this is python or an existing solution that uses pyramid search? My use case is exactly as described, no perspective changes, just trying to find rotation. Thank you! Hope to hear from you soon. – Hillhouse

Answer 3

Rotation invariant

For each key points:

1. Take area around key point.
2. Calculate orientation angle of this area with gradient or another method.
3. Rotate pattern and request area on this angle to 0.
4. Calculate descriptors for this rotated areas and match them.

Scale invariant

See BRISK method

Memphis