Using Template Matching on different sizes efficiently

Question

Is there a more efficient way to use Template Matching with images of different sizes? Here is my current Script:

import cv2
import numpy as np

img_bgr = cv2.imread('./full.jpg')
img_gray = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2GRAY)

template = cv2.imread('./template.jpg', 0)

w, h = template.shape[::-1]

res = cv2.matchTemplate(img_gray, template, 
cv2.TM_CCOEFF_NORMED)
threshold = 0.8
loc = np.where(res >= threshold)

for pt in zip(*loc[::-1]):
  cv2.rectangle(img_bgr, pt, (pt[0]+w, pt[1]+h), (0,255,255), 2)

cv2.imshow('detected', img_bgr)
cv2.waitKey(0)
cv2.destroyAllWindows()

This is my Template: Template

And I have these images, the first one works and the second one does not because of the size:

Works!

Fails

At first thought, I'm thinking it's failing because of the size of the template vs image

So I tried using this tutorial: Multi Scale Matching

But this seems really slow and bulky especially since I intend to use this in videos when I get it working. Is there a better way to handle this

Also, eventually I would also only like to check the top right of the image, I know that's a completely different question, but if you have any ideas since we're talking about scaling :)

Goggle

If you know the approximate position, you should really trim the video to that region. Also, you can probably get away with not doing the matching, e.g. by looking at spikes in total pixel intensity in that region (maybe normalised by total pixel intensity in the frame, depending on if the background/lighting changes a lot). – Berezniki

That's pretty cool. I actually got it working, at least on my computer. I just cropped a previous image for the icon and just look for that. I know it's not going to work for smaller images still, but it's a start for now – Goggle

Answer 1

The easiest way is to use feature matching instead of template matching. Feature matching is exactly meant for this kind of applications. It can also detect if the image is rotated .. etc

Have a look at this https://docs.opencv.org/master/dc/dc3/tutorial_py_matcher.html

Education

Answer 2

For different size use multi-scale template matching

Krystin

Answer 3

What you are looking for isn't that simple. The need is for multi-scale template matching, but as you mentioned, it will be slow, especially when the image resolution is pretty high.

The best and easiest solution for such cases is to train a convolutional neural network, a small one. Make use of transfer learning and train an SSD mobilenet on your data and you'll have a network that can do this detection pretty well for you and trust me it will be really fast. Object detection would give you the fastest, better, and more accurate solutions here.

Here's a link for an article explaining how to train for object detection in videos.

Sightread

Answer 4

How about using AI based algorithms instead of OpenCV? Mask R-CNN, YOLO, TensorFlow Object Detection API etc. The above algorithms may already be old enough...

Jovitta