How can I detect and track people using OpenCV?

Question

I have a camera that will be stationary, pointed at an indoors area. People will walk past the camera, within about 5 meters of it. Using OpenCV, I want to detect individuals walking past - my ideal return is an array of detected individuals, with bounding rectangles.

I've looked at several of the built-in samples:

• None of the Python samples really apply
• The C blob tracking sample looks promising, but doesn't accept live video, which makes testing difficult. It's also the most complicated of the samples, making extracting the relevant knowledge and converting it to the Python API problematic.
• The C 'motempl' sample also looks promising, in that it calculates a silhouette from subsequent video frames. Presumably I could then use that to find strongly connected components and extract individual blobs and their bounding boxes - but I'm still left trying to figure out a way to identify blobs found in subsequent frames as the same blob.

Is anyone able to provide guidance or samples for doing this - preferably in Python?

Vannie

+1 a bold endeavour. Interested to see what comes up. – Pitapat

@Trent Not sure if you're being serious or not. OpenCV doesn't have any such methods. – Vannie

@Nick Johnson, sorry it was a bad attempt at humor. Being able to detect and track people in a real-world environment is non-trivial. There are a lot of obstacles in overcome including: varying lighting conditions, obstructions, shadow removal, etc. – Allodium

this dude has one publication. check out the paper "Who are you? real time person identification" at robots.ox.ac.uk/~nema/publications. The basic theory seems to be radial basis functions. (just showing off my google skills.) – Washwoman

@Allodium it was a good attempt at humor. – Washwoman

Have you seen this question?: #2159401 – Izanami

Answer 1

The latest SVN version of OpenCV contains an (undocumented) implementation of HOG-based pedestrian detection. It even comes with a pre-trained detector and a python wrapper. The basic usage is as follows:

from cv import *

storage = CreateMemStorage(0)
img = LoadImage(file)  # or read from camera

found = list(HOGDetectMultiScale(img, storage, win_stride=(8,8),
                padding=(32,32), scale=1.05, group_threshold=2))

So instead of tracking, you might just run the detector in each frame and use its output directly.

See src/cvaux/cvhog.cpp for the implementation and samples/python/peopledetect.py for a more complete python example (both in the OpenCV sources).

Docent

Answer 2

Nick,

What you are looking for is not people detection, but motion detection. If you tell us a lot more about what you are trying to solve/do, we can answer better. Anyway, there are many ways to do motion detection depending on what you are going to do with the results. Simplest one would be differencing followed by thresholding while a complex one could be proper background modeling -> foreground subtraction -> morphological ops -> connected component analysis, followed by blob analysis if required. Download the opencv code and look in samples directory. You might see what you are looking for. Also, there is an Oreilly book on OCV.

Hope this helps, Nand

Maidie

Answer 3

This is clearly a non-trivial task. You'll have to look into scientific publications for inspiration (Google Scholar is your friend here). Here's a paper about human detection and tracking: Human tracking by fast mean shift mode seeking

Provided

As I commented, I don't need to determine that they're human - I just need to isolate moving blobs and track them. – Vannie

Answer 4

This is similar to a project we did as part of a Computer Vision course, and I can tell you right now that it is a hard problem to get right.

You could use foreground/background segmentation, find all blobs and then decide that they are a person. The problem is that it will not work very well since people tend to go together, go past each other and so on, so a blob might very well consist of two persons and then you will see that blob splitting and merging as they walk along.

You will need some method of discriminating between multiple persons in one blob. This is not a problem I expect anyone being able to answer in a single SO-post.

My advice is to dive into the available research and see if you can find anything there. The problem is not unsolvavble considering that there exists products which do this: Autoliv has a product to detect pedestrians using an IR-camera on a car, and I have seen other products which deal with counting customers entering and exiting stores.

Albion

I'm not actually too bothered about identifying multiple people in one 'blob' - I'm more interested in locating blobs of activity and finding their bounding boxes and centroids. I was hoping someone would be able to simply suggest a sequence of algorithms available in OpenCV for the purpose. :) – Vannie