OpenCV error while feature matching with FLANN

Question

I have a piece of code for matching features between a movie clip and a reference image. It generally works well, but sometimes it throws an error in the middle of a clip. Since it's always in the same clips, and at the same time, I guess there is something wrong with the frame it tries to analyze.

My code:

 cap = cv2.VideoCapture(clip_file)
 img1 = cv2.imread(ref_image,0)

 while(cap.isOpened()):

    # read the frame and convert to gray-scale
    ret, frame = cap.read()
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Initiate ORB detector
    orb = cv2.ORB_create()

    # find the keypoints and descriptors with ORB
    kp1, des1 = orb.detectAndCompute(img1,None)
    kp2, des2 = orb.detectAndCompute(gray,None)

    # FLANN parameters
    FLANN_INDEX_LSH = 6
    index_params= dict(algorithm = FLANN_INDEX_LSH,
                table_number = 6, # 12
                key_size = 12,     # 20
                multi_probe_level = 1) #2
    search_params = dict(checks=50)   # or pass empty dictionary

    flann = cv2.FlannBasedMatcher(index_params,search_params)

    matches = flann.knnMatch(des1,des2,k=2)
    cv2.imshow('img3',frame)

The error it throws sometimes during the clip playing:

Traceback (most recent call last):
 File "movie_test.py", line 81, in <module>
flann_movie('data/movie.avi','data/ref.jpg')
 File "movie_test.py", line 35, in flann_movie
matches = flann.knnMatch(des1,des2,k=1)
 cv2.error: OpenCV(3.4.2) C:\projects\opencv- 
  python\opencv\modules\flann\src\miniflann.cpp:317: error: (-5:Bad 
   argument) Only continuous arrays are supported in function 
   'cv::flann::buildIndex_'

Any suggestions on what causes the error would be appreciated. Thanks.

Daedalus

Answer 1

I think error is caused by frames of your video where no trace of original feature template was detected. Check what mid-results of matching are for each frame and then if that is the cause change the parameters of FLANN or simply skip those frames before the error occurs.

Cons

Thanks, I just skipped the frames using: `try: matches = flann.knnMatch(des1,des2,k=2) except: continue' and it plays fine now till the end of the clip. – Daedalus

I have the same problem while matching orb features between two images. I prefer not to use exception handling. what does it mean when you say no trace of original feature template was detected? Also, how can we check mid-results of matching? – Hydracid

Answer 2

You have the following:

matches = flann.knnMatch(des1,des2,k=2)

With k=2 it means each element needs to have 2-nearest neighbors. As a result, each list of descriptors needs to have more than 2 elements each:

if(des1 is not None and len(des1)>2 and des2 is not None and len(des2)>2):
   matches = flann.knnMatch(des1,des2,k=2)

(k-nearest neighbors algorithm)

Preraphaelite

thank you for the explanation to this cryptic error message. Your answer must be the accepted answer. – Fasano