Motion-detector for moving camera on OpenCV

Question

When the camera is stationary, then it is easy to detect the movement simply by subtracting the previous frame of the current frame, using optical flow:

• OpenCV v2.4.9.0: Motion Analysis and Object Tracking

• OpenCV v3.0: Motion Analysis and Object Tracking

But how to detect the moving objects during our own motion? When camera is moving, any difference (subtracting) between previous and current frames will give to us movements of background.

Moreover, the foreground (poles, trees) will move faster than the background (sky). If we look at everything that deviates from the average value of the motion, we find these poles and trees, but they are actually stationary.

How to detect any motions of other objects, when our camera is moving?

Pralltriller

In psychology research there've been optical flow pattern which describe different "camera" movements. In theory everything that doesnt follow that pattern (you have to find the right pattern first) is a moving object itself. Google "Perception of self-motion from visual flow" – Confusion

This is more of a computer vision question than a programming one. – Affiliate

Answer 1

In order to be able to differentiate between camera motion and scene motion, you need to simultaneously estimate the pose change of the camera between two frames and the scene geometry in those frames.

There are methods that accomplish this, you should look into structure from motion (SfM), and fundamental matrix estimation. These are complicated methods, and each comes with its own issues (e.g. in case of small translations, the estimation of scene depth may be inaccurate). However, you need this kind of method since your moving objects only distinguish themselves from the scene when you look at their motion in world coordinates, instead of image coordinates.

Garibay