Eye-tracking for code editing

Question

Is there a decent eye tracking package to replace the mouse for code editing?

I want to free up the mouse, but keep using my keyboard for editing code.

Having done some research on it, I concluded that proper eye tracking hardware is expensive. Using a webcam or high resolution video camera seems to be the most viable option.

Unfortunately, image-based tracking (as opposed to infra-red tracking) restricts the accuracy, and so not all features might be practical.

Desired eye-tracking IDE features:

• Page scrolling
• Tab selection
• Setting cursor position
• Selecting gaze-focused text with keyboard

A similar question recommends Opengazer for webcams, but I am particularly interested in speeding up basic text-editing. Any recommendations are appreciated, especially if you have experience with eye tracking and practical use cases.

Sonneteer

Answer 1

The kind of accuracy you're looking for is pretty difficult to achieve (Since text tends to be pretty small).

IR tracking is actually pretty easy to accomplish. A few IR LEDs and an IR camera (which is really just a normal camera with different filters) and your pupil lights up (This can be done with under $100, more if you want a better camera though).

It's the head tracking that might be more of an issue.

You end up with quite a few degrees of freedoms that you need to track and your inaccuracies will just build up.

Phidippides

Thanks. Would wearing a head-mounted camera negate most noise issues? – Sonneteer

The head mounted camera would increase accuracy of the eye tracking. The issue is, to track where on a screen one is looking, you have to track the head as well. – Phidippides

Can’t we use gyros for that? Or, build a set of infrared LEDs into the headset as well, and then use a second, stationary camera to track those (thus giving us independant data-points for head-location/orientation and eye-orientation) … – Bushweller

You can, but now you're introducing more variables to track and more noise. – Phidippides

Answer 2

I'm pretty sure there is no out-of-the-box solution for problem, but on eyewriter.org there are really nice instructions how to build your own eye-tracker. It's accurate enought to let someone "draw" graphities using only his eyes. so it should be possible to convert the eye-movements into mouse-events.

Corri

Answer 3

It can be done reasonably accurately (al la this article on how people read code) but i've never seen a commercial product that does what you're asking for

Maybe take a look at Emotiv's headsets, they use thought patterns to perform tasks. They're designed for games but you can probably repurpose it for normal tasks

Trip

Answer 4

Re text cursor placement, Lightning (While I have not worked on this particular feature, I have previously contributed to the Text 2.0 project as a student) which is described in this paper:

Universal eye-tracking based text cursor warping

will place the text cursor at the most salient target in the neighborhood of the gaze position reported by the eye tracker.

However, you need a Tobii eye tracker that supports the TET API. You might want to contact Tobii to verify that the Tobii X2-30 eye tracker which costs < $10k is compatible.

Matrimony

Answer 5

Just use vim. Do more with the keyboard, less with the mouse.

Infante

Do everything with the keyboard. – Draughty

I don't understand how this would take away from using the keyboard... If you could just gaze at a place on the screen, and then hit some mapping like, <leader><leader very quickly, and have your cursor move to that spot, wouldn't you still only be using the keyboard? – Unbolt

Ah yes, the web development with vim. Eventually you have to move to a web navigator and click some buttons. – Williamsen

Answer 6

Personally I had an issue always having to reach for a normal mouse, looked at various option eyetracking/voice/touchscreen and ended up changing the keyboard to a IBM Trackpoint end result being my hands never leave the Keyboard and my typing speed and accuracy improved due to not having to reposition my right hand.

Famed

Think I'm going to look into this. – Sonneteer

@FreshCode What did you end up going with? – Expellant

Answer 7

Eye Tribe has a $99 consumer-level eye tracker that is available now.

“Using a webcam or high resolution video camera seems to be the most viable option.”

Eye Tribe is a spinoff of Gaze Group, a research group located at the IT University of Copenaghen. The people of Gaze Group developed the open-source ITU GazeTracker software, which allows people to turn low-cost webcams into eye trackers.

http://www.gazegroup.org/downloads

Upon looking at the “downloads” section for Gaze Group, it seems that there are already some eye tracking applications to do some basic actions.

melhosseiny mentioned the Text 2.0 framework for creating eye tracking apps using HTML, CSS and JavaScript, and the Universal eye-tracking based text cursor warping feature for placing the text cursor at the most salient target.

Eye Tribe has its own SDK, but those things above could help if they work with Eye Tribe.

Of