Anti-spoofing methods for face recognition in Python

Question

I am working on a face recognition project where I can first enroll myself and then start a recognize script which will start my webcam and it will recognize myself. I am following this article and its working perfectly fine.

What I have noticed is that if I show my photo to the camera instead myself in front of the camera, it still detects and recognize my face. There is no anti-spoofing involved in it. I want to include anti-spoofing method in the code so that it can detect weather the face detected/recognized is real of fake. For this I thought of following below approaches:

1. Eye blink detection: Initially I thought I would implement an eye blink detection algorithm, but it also has its disadvantage. What if a real face person didn't blinked his eyes for sometime, in that case our code will tag that face as fake. Also the eyes was also not getting detected at a distance of 1-1.5 meter from the camera.

2. Using temperature sensor: I also interfaced omron thermal sensor so that I can get the temperature of the face. In normal human face, temperature is always above a threshold. In case of face in photo, it will always be below that threshold. I implemented this and it was working fine. But later realized that if someone showed photo in phone, in that case due to phone's high screen temperature, its always more than the threshold and thus it is tagged in as real photo.

Above mentioned methods didn't worked for me. I am looking for a simple solution which can work in all the scenarios. I am doing this project on Raspberry Pi, so looking for a solution which is compatible with Raspberry Pi.

Coparcener

Since the temperature sensor is also available and not used in face recognition, couldn't the spoofer just put a lighter close to it and have a picture of your face infront of the actual camera? – Derange

@Derange You are right and thats why the methods I have mentioned didn't worked for me. – Coparcener

Maybe you could implement a hand signal detection, so you can make a certain shape with your hand and that would also be needed, to get access. – Derange

@Derange Yes I think I can do that. But this application is like n access control or attendance system where users will come in front of the camera and their attendance will be marked. But thanks for your suggestion, I will think about it. – Coparcener

No problem. I suggested it, because it would be easy to remember and hard for someone to guess without knowing that a hand gesture is also needed, or what exactly you do with your hand. As long as that is secret, it should be good. Good luck with your project :) – Derange

@Derange There is an active flash method shown in this article at the bottom of the page which says by light reflection on face we can detect real of fake. I am not sure how this can be achieved but it looks promising. What are your thoughts.? – Coparcener

Have you checked out the article at medium.com/visionwizard/… which specifically deals with Face Spoofing Detection in Python. – Crambo

Answer 1

Sorry for any mistake because I am not from raspberry pi's background but as a decent guy helping people I think that you should try resolution check (if it is possible) because phone's screen would always have less resolution than the real face. And then you can use it with the eye blink method to catch hold of a phone as photos do not blink eyes. Average human blinks 12 times in a minute so 1 time every 5 seconds . This will help you to catch hold of the printed photos. Hope this would help.

Zebec

By resolution check, you mean the size of the face detected in phone vs the size of the real face.? – Coparcener

Resolution means in two ways here = 1) You must have observed that phone screen seems a bit blue and black (like old tv black dots type) and brighter than the real face. Also resolution mean what you told. – Zebec

Okay so you mean checking the intensity of the light of the captured photos. If its above than a normal threshold, there must be phone screen in that.? – Coparcener

yes also it appears a bit bluey than the normal face – Zebec

I have checked many articles online but I am not able to understand how can I do this in opencv. Do you have any link which I can refer.? – Coparcener

sorry @SAndrew as i said earlier , I am not from that background but this idea just struck me so I suggested it. – Zebec

pyimagesearch.com/2016/10/31/… === I think this would help – Zebec

Have already gone through this article, not really helpfull – Coparcener

OK Sorry because just did a raw search – Zebec

Just wanting to know that is my idea helpful ? – Zebec

Its helpful but I am not able to get useful resources to implement it – Coparcener

Answer 2

You should use an object detector on top of the face detector. It can definitely detect a phone.
You could retrain it to detect a photo being held up as well.

Have the object detector run first, save the bounding box coordinates of the phone, then see if the face bounding box coordinates reside inside of the phone.

Abstention

Great idea. Do you have any link to a model which can detect phone. I am searching for it – Coparcener

I did some research on it and realized that this method of detecting phone is also not possible. Reason being, when a spoofer puts phone in front of the camera to recognize face, the phone can be placed very close to the camera so that face is clearly visible so in this case phone will not be fully visible in the camera and thus its hard to detect a phone. – Coparcener

Answer 3

Use a camera/multi-camera setup with a depth sensor.

A photo or smartphone screen is flat. A real face is three-dimensional.

There are many ways you can detect depth.

One is to use a Kinect. The Kinect depth sensor uses a "infrared laser projector combined with a monochrome CMOS sensor" (Vemulapalli, p. 15/p. 26). It works by projecting infrared light in a dot pattern.

You can connect a Raspberry Pi to a Kinect.

The idea to use Kinect is from "Implementing face recognition on Raspberry Pi using Kinect camera", by Kanaka Sunanda Vemulapalli.

Vemulapalli, Kanaka Sunanda, "Implementing face recognition on Raspberry Pi using Kinect camera" (2018). Graduate Research Theses & Dissertations. 3498. https://huskiecommons.lib.niu.edu/allgraduate-thesesdissertations/3498

Damn