The Art of Eye Tracking

The Art of Eye Tracking

Eye-Tracking as an interaction isn't as intuitive as you would think.

Eye-tracking as an interaction isn't as intuitive as you would think. The human eye is BIZARRE AS HELL, and the magic required to make it intuitive is fascinating:

On my journey I get exposed to some wild R&D— close to 10yrs ago I was creating interactions for an early eye-tracking VR headset. For me, It involved a lot of research (and introspection) into how the eye works.

First of all, our eyes most common movement is to rapidly dart around. These precise and instantaneous movements are called SACCADES. When you walk into a new space, your eyes don't smoothly track around. Instead, your eyes have a series of saccades, darting from point to point to build a mental "map". Also, saccade is French for "jerk”.

The actual path our eyes take when scanning a sentence.

Even when reading, you would expect your eyes to smoothly and precisely follow a line of text from left-to-right, but instead it’s total chaos— your eye zig-zagging from word 7 back to to word 3 then to 9 etc… But our eye does what it wants, and our brain smooths it all out.

Ever been in a scenario, and felt it was uncomfortable, or even stressful to control your eyes?
Don't make eye contact on the subway! Don't stare at your crush! Don't glance at a distant distraction when being spoken to!

Our eyes WANT to roam free . To control your eyes feels... unnatural.
So how do you leverage gaze as a key control input?

As you might already imagine, you don't want to just turn gaze into a mouse cursor— imagine that your mouse control was 50% shared with your subconscious. Stay away from that link… don't like that post… don't even look at the delete button!

So rather than precise feedback on the jittery position of the gaze, you can use some "softening" or averaging. Forgiveness rather than precision. Not unlike Ken Kocienda’s original iPhone keyboard design which dynamically changed touch target sizes to anticipate and forgive errors— without the user knowing.

With eye-tracking, the visual feedback from your gaze needs to be subtle— almost invisible.
Something that very gradually reveals itself to you. Above all else: Don’t make the user THINK about their eyes. No cursor tracking the eyes movement, graphical feedback that reminds us that our eyes don’t drift, but SNAP to position.

If the user isn't overly conscious of their free-roaming eyes, you can create subtle and beautiful interactions. It’s relatively easy to create scenarios like "the item I was thinking about is drifting towards me”. When done thoughtfully, the UI can feel... TELEPATHIC.

Eye Tracking is a cornerstone of the Apple Vision Pro's interaction design

The best way to leverage eye-tracking is with a light touch— Apple's Reality Pro appears to be doing this well. But this approach might not always be the way...

If gaze interactions become ubiquitous, we may become more comfortable with exerting continuous precise control over our gaze. Not unlike the way that experts speed-type, or a gamer handles a controller.

It could completely re-wire our relationship with our eyes.

2024 blackbox infinite
designing positive futures

2024 blackbox infinite
designing positive futures