Facial expression recognition from webcam [closed]

Question

I am currently working on a project where I have to extract the facial expression of a user (only one user at a time from a webcam) like sad or happy.

There are a lot of programs/APIs to do face detection but I did not find any one to do automatic expression recognition.

The best possibility I found so far:

I used OpenCV for face detection earlier, which was working great, so If anyone has some tips on how to do it with OpenCV, that would be great!

Some user on a OpenCV board suggested looking for AAM (active apereance models) and ASM (active shape models), but all I found were papers.

-So i'm Using Active Shape Models with Stasm, which will give me access to 77 different points within the face, so I would still have to manually map them to expressions.

Any programming language is welcome.

Monday

Do you have any progress with your project? – Goatherd

@NenadBulatovic #18641304 – Monday

Answer 1

After 2.4 release, opencv came with a face recognition API. You may define sad and happy faces as two different faces and use the API to classify them. If you are looking for any tutorials you can check out opencv's tutorial page, too.

Beer

if i will use this api i must get for every user the happy and the sad face !! – Monday

Yes, those are pre defined. Maybe you can create your own training pattern for face detection. Train for happy faces and sad faces in 2 different xml files, and try using them. – Beer

my target is to any user will use this application and is this method will be fast to use on webcam videos !! – Monday

You said you tried face detection using opencv (using the .xml file that comes together with opencv package), wasn't it fast enough? I tried it also within a skin detector, algoritm complexity was much more but it was working fast enough. – Beer

ok could you give me a tutorial about how to train xml file for opencv – Monday

opencvuser.blogspot.com/2011/08/… , note.sonots.com/SciSoftware/haartraining.html – Beer

also try delaunay triangulation, as it is used to recognize expressions, too. – Beer

@C.CanberkBacı Do you have a website or is there any way I can contact you directly? – Goatherd

Answer 2

Maybe this article can help you.

It is explanation of concepts used to achieve result shown here:

Also, you might want to have a look here.

All of above mentioned is a work of people who more or less achieved what you watn.

I want to do same as you - facial expression recognition with OpenCV, so I hope sooner or later we can exchange our ideas.

Warehouseman

Answer 3

Half the problem is solved for you using OpenCV. If you built OpenCV with examples, you can run an example named c-example-smiledetect which can detect a smiling face with different levels.

The code should be in you OpenCV folder under samples/c. i recall you can adjust the sensitivity in the code so that might help. You can also find the code here.

On another note, an implementation of AAM using OpenCV can be found in this book (chapter 7). This doesn't include the searching and fitting part though, just shows how to create an AAM from ASM and a texture.

Asturias

c-example-smiledetect is using pre trained Haar detector. I think that OP must train his own detectors (classifiers) in order to detect other facial expressions. If I recall correctly it is necessary to use several thousands images to do so. I rad and article where people said that on fast machines whole training process took more than non - stop 7 days for a classifier to be created. Then if it shows bad results, re-training must be applied. – Goatherd