OpenCV warpPerspective slow performance

Question

In my application:

1. I track an object.

2. Get where its corners are coming in this frame.

3. I find the homography between its corners from last frame and current frame.

4. Use that homography to do a perspectiveTransform on the corners found in the current frame, to get transformed_corners.

5. Use the transformed_corners to find the homography between them and the overlay_image.

6. Apply above homography M to overlay_image, to get what would be called the warped_image using warpPerspective. This is the slow part.

7. And then using masking operations, I print the warped_image onto the current frame where the object was found.

Now I know after reading this blog article here why warpPerspective is slow.

And I'm getting ~300ms per frame in just the 6th step above, all because of warpPerspective. It's significantly affecting the FPS output of my application. Basically, it went down to 2FPS from 12 FPS without warping on every frame.

Is there any faster alternative to this? It's all done on Android, using NDK r9. What are some fast alternatives, and optimizations to reduce the warp time from 300ms to sub 50ms times?

Regale

what kind of interpolation method did you choose in warpPerspective? Do you allocate new memory in each warpPerspective call? – Making

@Making Not passing any flag values. So I suppose the default INTER_LINEAR is being used. See here – Regale

you could try INTER_NEAREST but from my experience it isn't much faster. Can you give some information about input/output images sizes etc? Maybe some code is necessary to tell whether there is any optimization possible. – Making

The snippet I gave you is all that's relevant. Rest is declaration of empty Mats and passing around from here and there. Image sizes vary from 1920x1080 for currentFrame and around 390x293 for logoImage. It's obvious that full HD res for currentFrame is too big. We tried smaller size, 960x540, and smaller, but time remains around ~250-350ms. – Regale

you warp from logo to currentFrame? Relevant for warping performance is only the resolution of the DESTINATION image. So if you know in advance that only a small part of the destination image is relevant, you can ignore the rest (e.g. forward-transform the bounding box of the source image to find the transformed bounding box in destination image). Btw, what snippet are you referring to? From the algorithm description it isn't visible whether you re-use an allocated warped_image and the chosen size of warped_image etc... – Making

This dude dpaste.de/PWNL – Regale

Let us continue this discussion in chat. – Regale

sorry, no time for intense chat. If currentFrame.size() is always the same and you reuse the destination matrix I don't see much optimization potential except warping to a smaller subimage only covering the transformed bounding box (which could give a lot of spee-up if the warped image is much smaller than the total frame). – Making

@Regale did you ever find a way to optimize warpperspective? – Janot