About finding pupil in a video

Question

I am now working on an eye tracking project. In this project I am tracking eyes in a webcam video (resolution if 640X480).

I can locate and track the eye in every frame, but I need to locate the pupil. I read a lot of papers and most of them refer to Alan Yuille's deformable template method to extract and track the eye features. Can anyone help me with the code of this method in any languages (matlab/OpenCV)?

I have tried with different thresholds, but due to the low resolution in the eye regions, it does not work very well. I will really appreciate any kind of help regarding finding pupil or even iris in the video.

Tlaxcala

I think it will be difficult with that resolution. Can you upload one screenshot at imageshack.us and provide its link here? – Illfavored

I do not know how to attach an image. Can you plz help me... – Tlaxcala

You cannot do that yet, because you don't have enough reputation on this site. – Wiretap

Thanks Tibor, I also tried some times ago to upload, but failed :( – Tlaxcala

go to http://imageshack.us/. You can upload image there. Then give the link of your image here. – Illfavored

If image is of good resolution, try this : https://mcmap.net/q/873985/-track-eye-pupil-in-a-video – Illfavored

Hi Abid, thanks a lot for the link. I tried with the link you gave, but it works well for the high resolution one, but for me, it's not working. I uploaded a sample image in the link imageshack.us/photo/my-images/585/eyea.jpg – Tlaxcala

Perhaps you could try to use the OpenCV function CvHoughCircles? This is enables you to detect the different circles in the eye. See the API reference for an example. – Hardily

Answer 1

What you need to do is to convert your webcam to a Near-Infrared Cam. There are plenty of tutorials online for that. Try this.

A Image taken from an NIR cam will look something like this -

You can use OpenCV then to threshold.

Then use the Erode function.

After this fill the image with some color takeing a corner as the seed point.

Eliminate the holes and invert the image.

Use the distance transform to the nearest non-zero value.

Find the max-value's coordinate and draw a circle.

Dunsany

Hi Anirudh, thanks for your answer. Problem is I can't use any infrared camera, or any other modified version of web cam. Only have to use a simple webcam. Anyway, thanks a lot. – Tlaxcala

A regular webcam can be converted to a NIR can very easily. Refer the link mentioned above in answer. – Dunsany

Answer 2

If you're still working on this, check out my OptimEyes project: https://github.com/LukeAllen/optimeyes

It uses Python with OpenCV, and works fairly well with images from a 640x480 webcam. You can check out the "Theory Paper" and demo video on that page also. (It was a class project at Stanford earlier this year; it's not very polished but we made some attempts to comment the code.)

Arronarrondissement

Answer 3

Depending on the application for tracking the pupil I would find a bounding box for the eyes and then find the darkest pixel within that box.

Some psuedocode:

box left_location = findlefteye()
box right_location = findrighteye()
image_matrix left = image[left_location]
image_matrix right = image[right_location]
image_matrix average = left + right
pixel min = min(average)
pixel left_pupil = left_location.corner + min
pixel right_pupil = right_location.corner + min

Gardener

I tried with the darkest pixel. But the problem with this one is, some times the light is reflected on the iris, then pupil is no longer the darkest point. I don't know how to solve this problem. – Tlaxcala

so it's sometimes the lightest and sometimes the darkest? You could find the most extreme point either using the absolute value of the gradient or by simply finding the max(abs(eye_image - mean(eye_image))) – Gardener

Hi vaebnkehn, I tried this approach yesterday, but it did't work. :( – Tlaxcala

Perhaps if you posted a few example images of eyes that would help. My only other thought would be to try to find the ring of colored pixels that are the iris and the center of that should be the pupil. I would look at the saturation channel in HSV space to find the iris. – Gardener

Hi Vaebnkehn, I've uploaded one sample image imageshack.us/photo/my-images/585/eyea.jpg – Tlaxcala

Awesome. Find the two biggest white areas and split the difference. – Gardener

I don't understand, can you please describe a little more. – Tlaxcala

Find all the white-ish pixels in the eye area. Label the regions and keep the two largest (these are the left and right whites of the eye). Find the centroid of each regon and then take the average of the two centroids. That point should be the pupil. – Gardener

The problem with that approach is that due to the image quality, eyes are not that white-ish. It will be difficult to avoid false positives i think. – Atterbury

Answer 4

In the first answer suggested by Anirudth...
Just apply the HoughCirles function after thresholding function (2nd step).
Then you can directly draw the circles around the pupil and using radius(r) and center of eye(x,y) you can easily find out the Center of Eye..

Colonialism