Classifiers confidence in opencv face detector

Question

I'm using opencv's har cascade face detector (cv.HaarDetectObjects) in python.

for example:

    faces = cv.HaarDetectObjects(grayscale, cascade, storage, 1.2, 2,
    cv.CV_HAAR_DO_CANNY_PRUNING, (50,50))

       for f in faces:
           print(f)

This will print a list of detections in this form:

 ((174, 54, 114, 114), 53)
 ((22, 51, 121, 121), 36)
 ((321, 56, 114, 114), 21)
 ((173, 263, 125, 125), 51)
 ((323, 272, 114, 114), 20)
 ((26, 271, 121, 121), 36)

Where each line represent a detection. The first 4 numbers are the x,y location of the top-left point, and the height, width of the bounding box. The last number is (quoting from the openCV documentation) the number of neighbors.

I guess I have two questions:

1) What does the last number mean? I couldn't find any reference to that when googling.

2) (more important)Is there a way to get a confidence score for each detection? How much is the face classifier certain that the detection corresponds to a real face?

Thanks

Shaft

might be useful: haoxiang.org/2013/11/… – Diophantus

Answer 1

1) The detection code produces more than one detection for an object - e.g. in different scales, slightly shifted, etc. The detections are then grouped and the number of neighbours in such a group is the number returned. See also Viola Jones paper, paragraph 5.6 (http://research.microsoft.com/en-us/um/people/viola/Pubs/Detect/violaJones_IJCV.pdf) and OpenCV source.

2) You can possibly use the number of neighbours as some measure of confidence.

Sweeping

Answer 2

Thanks a lot for your question and answer, I have been looking for a opencv face detection with confidence scores for a day. Your question and answer give me some guidance to solve the problem.

Like Palmstrom said, the last number means the number of object positions in that cluster. and you can use that as confidence score.

As far as I know, there is only such kind of API in the old python API. New API does not have this (number of object in the cluster) value.

I put my code here in case it will help some other people. This is a old python API whose tutorial is hard to find.

import sys
import cv

def detect_face(image):
    image_size = cv.GetSize(image)
    # # create grayscale version
    grayscale = cv.CreateImage(image_size, 8, 1)
    cv.CvtColor(image, grayscale, cv.CV_BGR2GRAY)
    # # equalize histogram
    cv.EqualizeHist( grayscale,grayscale )

    #parameters to the detection function    
    cascade = cv.Load('haarcascade_frontalface_alt.xml')
    haar_scale = 1.1
    min_neighbors = 3
    haar_flags = cv.CV_HAAR_DO_CANNY_PRUNING
    min_size = (30,30)

    faces = cv.HaarDetectObjects(grayscale, cascade, cv.CreateMemStorage(0),
                                haar_scale, min_neighbors, haar_flags, min_size)

    print faces

    if len(faces) > 0:
        print '=> ' +  str(len(faces)) + ' face detected!'
        for ((x,y,width,height), n) in faces:
            pt1 = (x,y)
            pt2 = (x + width, y + height)
            cv.Rectangle(image, pt1, pt2, cv.RGB(255, 0, 0), 3, 8, 0) 


if __name__ == '__main__':

    filename = sys.argv[1]
    image = cv.LoadImage(filename,cv.CV_LOAD_IMAGE_COLOR);
    detect_face(image)

    cv.ShowImage("cam", image)
    cv.WaitKey(0)

Badlands