Moving background subtraction (OpenCV)

Question

Do you know any sources with background subtraction from moving camera? I want to write something similar to this: http://vimeo.com/72577177

There is a lot of materials about movement detection with still background, but I can't find any code sample with moving background.

edit: I thought about optical flow and removing background by detecting the biggest number of similar vectors. Could it be that simple?

Abeyta

That is an object tracker probably based on moving edges and other techniques but it does not seem to do background subtraction. – Krissykrista

I translated their article about it and they said something about background substraction – Abeyta

Answer 1

Here are two research articles on this topic:

• Mittal, A. et D. Huttenlocher. 2000, «Scene modeling for wide area surveillance and image synthesis» in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (PDF)

• Hayman, E. et J. Eklundh. 2003, «Statistical background subtraction for a mobile obser- ver», in Proceedings of the IEEE International Conference on Computer Vision. (PDF)

Methodical

Answer 2

This is not background subtraction! This is object tracking!

Watch this video. I think you're looking for something like this. It has some paper references at the end of the video. Search those papers.

You can find the source of the algorithm in that video in this and this link.

Good luck!

Carruthers

Answer 3

The link that you referred to is more like object tracking. But if you want to apply moving background subtraction using opencv it would be easier. It works as more like a motion tracker.

Here's the code:

import cv2
import numpy as np

cap = cv2.VideoCapture("input.mp4")
out = cv2.createBackgroundSubtractorMOG2()

fourcc = cv2.VideoWriter_fourcc(*'MJPG')
output = cv2.VideoWriter('output.avi', fourcc, 20.0, (640,480), isColor=False)

while(cap.isOpened()):
    ret, frame = cap.read()

    if ret==True:
        frame = cv2.flip(frame,180)

        outmask = out.apply(frame)
        output.write(outmask)

        cv2.imshow('original', frame)
        cv2.imshow('Motion Tracker', outmask)

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    else:
        break


output.release()
cap.release()
cv2.destroyAllWindows()

This code allows you to apply background subtraction in a video file and save it.

Orectic

Answer 4

It's not very hard to achieve that video. Because the moving objects are small and the background is static, though the camera is moving slowly.

First, determine the motion vector between consecutive frames. For example, the pixel at (100,100) in frame t-1 moves to (103, 102) in frame t. As most parts of the background are static, a common motion vector can be determined to describe the motion of the most pixels.

Second, using RANSAC or some other algorithms to find the pixels which do not move according to the motion vector. These pixels represent the moving objects.

Westerfield