How does the Viola-Jones face detection method work?

Question

Please explain to me, in few words, how the Viola-Jones face detection method works.

Cryosurgery

here is answer en.wikipedia.org/wiki/Viola-Jones_object_detection_framework ask what is not clear – Almira

Answer 1

The Viola-Jones detector is a strong, binary classifier build of several weak detectors

Each weak detector is an extremely simple binary classifier

During the learning stage, a cascade of weak detectors is trained so as to gain the desired hit rate / miss rate (or precision / recall) using Adaboost To detect objects, the original image is partitioned in several rectangular patches, each of which is submitted to the cascade

If a rectangular image patch passes through all of the cascade stages, then it is classified as “positive” The process is repeated at different scales

Actually, at a low level, the basic component of an object detector is just something required to say if a certain sub-region of the original image contains an istance of the object of interest or not. That is what a binary classifier does.

The basic, weak classifier is based on a very simple visual feature (those kind of features are often referred to as “Haar-like features”)

Haar-like features consist of a class of local features that are calculated by subtracting the sum of a subregion of the feature from the sum of the remaining region of the feature.

These feature are characterised by the fact that they are easy to calculate and with the use of an integral image, very efficient to calculate.

Lienhart introduced an extended set of twisted Haar-like feature (see image)

These are the standard Haar-like feature that have been twisted by 45 degrees. Lienhart did not originally make use of the twisted checker board Haar-like feature (x2y2) since the diagonal elements that they represent can be simply represented using twisted features, however it is clear that a twisted version of this feature can also be implemented and used.

These twisted Haar-like features can also be fast and efficiently calculated using an integral image that has been twisted 45 degrees. The only implementation issue is that the twisted features must be rounded to integer values so that they are aligned with pixel boundaries. This process is similar to the rounding used when scaling a Haar-like feature for larger or smaller windows, however one difference is that for a 45 degrees twisted feature, the integer number of pixels used for the height and width of the feature mean that the diagonal coordinates of the pixel will be always on the same diagonal set of pixels

This means that the number of different sized 45 degrees twisted features available is significantly reduced as compared to the standard vertically and horizontally aligned features.

So we have something like:

About the formula, the Fast computation of Haar-like features using integral images looks like:

Finally, here is a c++ implementation which uses ViolaJones.h by Ivan Kusalic

to see the complete c++ project go here

Intosh

Can you explain me, a bit better, what is a visual feature(the meaning of the squares in the picture) and what means the final formula that is in your answer? – Cryosurgery

Ok, thanks a lot. I have one last question (in reference to the JCooper answer): how can work the face detection with haar features if we have a white guy with blue or green eyes (so that the difference of light between the nose and the eyes isn't considerable)? – Cryosurgery

@Cryosurgery It still works pretty well because eyes are set back in the head. The shadows from top-down lighting can trigger the feature. – Catie

@cMinor +1 for such a good explanation, can you please just let me know what are the integral images ? – Matrices

The integral image computes a value at each pixel (x,y) that is the sum of the pixel values above and to the left of (x,y), inclusive (This can quickly be computed in one pass through the image). Robert Fergus explains it here cs.nyu.edu/courses/fall12/CSCI-GA.2560-001/… – Intosh

Answer 2

The Viola-Jones detector is a strong binary classifier build of several weak detectors. Each weak detector is an extremely simple binary classifier

The detection consists of below parts:

Haar Filter: extract features from image to calssify(features act to encode ad-hoc domain knowledge)

Integral Image: allows for very fast feature evaluation

Cascade Classifier: A cascade classifier consists of multiple stages of filters, to classify a image( sliding window of a image) is a face.

Below is an overview of how to detect a face in image.

A detection window shifts around the whole image extract feature(by haar filter computed by Integral Image then send the extracted feature to Cascade Classifier to classify if it is a face). The sliding window shifts pixel-by-pixel. Each time the window shifts, the image region within the window will go through the cascade classifier.

Haar Filter: You can understand the the filter can extract features like eyes, bridge of the nose and so on.

Integral Image: allows for very fast feature evaluation

Cascade Classifier:

A cascade classifier consists of multiple stages of filters, as shown in the figure below. Each time the sliding window shifts, the new region within the sliding window will go through the cascade classifier stage-by-stage. If the input region fails to pass the threshold of a stage, the cascade classifier will immediately reject the region as a face. If a region pass all stages successfully, it will be classified as a candidate of face, which may be refined by further processing.

For more details:

Firstly, I suggest you to read the source paper Rapid Object Detection using a Boosted Cascade of Simple Features to have a overview understanding of the method.

If you can't understand it clearly, you can see Viola-Jones Face Detection or Implementing the Viola-Jones Face Detection Algorithm or Study of Viola-Jones Real Time Face Detector for more details.

Here is a python code Python implementation of the face detection algorithm by Paul Viola and Michael J. Jones.

matlab code here .

Joettajoette