OpenCV Optical Flow assertion

Question

I'm attempting to track landmarks along the contour of facial features obtained via dsift with python 2.7 and openCV 2.4.11. I want to track these features between frames.

However I am receiving the following error. I have checked the input images are 1-channel equal dimensions (and unsigned 8bit type), and likewise with the prev points:

OpenCV Error: Assertion failed ((npoints = prevPtsMat.checkVector(2, CV_32F, tru
e)) >= 0) in cv::calcOpticalFlowPyrLK, file ..\..\..\modules\video\src\lkpyramid.cpp
cv2.error: ..\..\..\modules\video\src\lkpyramid.cpp:845: error: (-215) (npoints
= prevPtsMat.checkVector(2, CV_32F, true)) >= 0 in function cv::calcOpticalFlowP
yrLK

Line causing issue:

new_pts, ttl, err = cv2.calcOpticalFlowPyrLK(self.old_img, i_img, i_old_pts, None)

Does anyone have any idea where I can begin debugging this?

Gathering

Remains unsolved if anyone wants to help – Gathering

Answer 1

I had the same problem when I did optical flow based tracking. I tried many many different ways to solve this. But in vain.

Finally, there was an example program in which they tracked using shi-tomsi corner points detection and those points were used in the LK algorithm and it worked perfectly. So i probed into the data types and dimensions of the output of the Shi-Tomsi detector and I made sure my points to be tracked were of the same type. It woked!

Here's what you need to know.

• make sure the images are grayscale.
• your coordinate parameter that is i_old_pts should be single precision float meaning float32. This type is available in numpy use that. the float in python is float64
• the coordinate parameter i_old_pts(from your program) should be a numpy array with the dimension (n,1,2) where n represents the number of points.

This should work.

Detumescence

np.float32(coorX),np.float32(coorY) helped me – Hus

I had to initialize my numpy arrays to be of type np.float32. – Sink

Thank you so much! I spend hours trying to figure out what's up.. The cv2 python binding should explicitly mention the expected parameter type in the docs. They do it for the C function signature (e.g. in case of the corner array for the cornerSubPix function its CvPoint2D32f* corners). Let corners be my initial corner array, then I used corners.astype(np.float32) – Clyde

Thank you so much! These datatype things and the array shape thing are really hard to figure out without you explicitly writing them out. WTH cv2 doesn't directly tell us that? Geez – Clichy

This guy is a god – Ephemerid

Answer 2

I was following opencv sample for optical flow had the same problem. In my case the problem was the type of video file from which I was reading. I was reading from mkv file which didn't work.

Rasure

Answer 3

I am not sure why, but after a lot of code evaluation and debugging, I found that the first frame of my video (I captured video from the webcam) was an empty BLACK image. So, I used the second frame:

ret, prevframe = cap.read();
ret, prevframe = cap.read(); # the first frame was black and I was not sure why!!!

Judejudea

Answer 4

In my case, everything was working fine until a very specific moment where I had implemented a different method where the value of i_old_pts returned as None when there were no important points to track. My workaround to this was adding this line:

if len(i_old_pts) == 1: i_old_pts = np.zeros((1,2), dtype=np.float32)

So that there would at least be an array of zeros for optical flow to read.

Since the i_old_pts variable contains all the corners to track, it contains a numpy array of coordinates. In my case, there was a situation when the value of i_old_pts returned as None, so I just took advantage of the fact that the length of i_old_pts would only be one at this point (with only the None value in it) and subbed in an arbitrary point.

If you're facing the same error, you could always store a coordinate point or two from the previous frame to sub in in case of the NoneType error. This specifically happened to me when I was testing a fast video clip where it abruptly switched scenes, so I just plugged in a random point for that one scene of transition and allowed optical flow to get back to work as the scene continued.

Or if it doesn't work and the NoneType is still bugging you, try this:

while True:
    i_old_pts_is_none = False
    try:
        i_old_pts.all()
    except AttributeError:
        i_old_pts_is_none = True
    if not i_old_pts_is_none: 
        new_pts, ttl, err = cv2.calcOpticalFlowPyrLK(self.old_img, i_img, i_old_pts, None)
    i_old_pts_is_none = False

Since the whole code is in a loop anyways if you're using optical flow on a video file or stream, you could just skip that iteration which returned a None.

Hope this helps!

Duct