GDC 2018: Is Eye-Tracking the Future of Virtual, Augmented Reality?

Virtual Reality and Augmented Reality continue to grow in importance both within and outside of the game industry. Though lacking a killer app, companies have continued to develop new and better ways to experience VR and AR. One such company is Tobii, who is the largest eye-tracking company in the world. Tobii has been working on their tech for years now and have recently begun scoring partnerships with major companies such as Qualcomm. Curious if eye-tracking is as big a game changer as Tobii claims, we sat down with them to check out the tech.

Tobii’s tech is integrated right into headsets and reads where you’re looking at, feeds that data to the engine, and then replicates it on screen. The first demo featured two pictures of avatars, one with and the other without eye-tracking. While the one without stared back blankly at us, the one with the tracking replicated every movement. To Tobii, this tech could vastly increase the potential of social VR experiences as people would be able to convey emotion.

The demo shifted to a field of bottles where we were asked to grab rocks and try to knock them over. Without tracking, the task was hard to complete due to questionable aiming skills, which has always been a problem in VR first-person shooters. With eye-tracking, the software knows where you’re looking and what the intent of the user is, making it simple to pick up a rock, look at the target bottle and then throw. It was amazing how accurate it was and it’s easy to see how this could improve VR FPS titles.


The focus then shifted to a room with multiple controllable objects. There were two robots, a store, a TV and a window shutter. Merely glancing at the object we wanted to interact with allowed us to interact with it. Without the tracking, we had to move our head precisely to make sure we were using the object we wanted. The same held true when playing a movie. With eye-tracking, we only needed to look at the video and then press the Vive Wand’s button to activate it. Without, we had to reach forward to get it to play.

According to Tobii, what makes eye-tracking work so well is that it removes the complexity of interactions. Oscar Werner, Business Unit President, compared eye-tracking to what Apple did with the smartphone. With the smartphone, Apple made the process of interacting with computer hardware simple by eliminating the dragging of a mouse to open software. Similarly, eye-tracking removes the need to move the head as current VR and AR headsets require.

Tobii’s tech is also aimed at making devices better by employing a technique known as foveated rendering. Using this, a game only renders what the player is looking at in high resolution. Combined with eye-tracking, developers can use this to change the resolution of what’s displayed as the player moves their eyesight. Foveated rendering significantly reduces resources, which is something that Qualcomm is excited about.


Qualcomm recently announced they’re partnering with Tobii for this very reason. The company is using its new Snapdragon 845 to develop their own standalone VR/AR headsets. Using a combination of foveated rendering and Tobii’s eye-tracking, Qualcomm and other hardware providers can build standalone headsets with decreased power usage that will help keep the device cool. After all, nobody wants something hot that close to their eyes.

Werner also touched on some of the projects Tobii is involved in outside of the game industry, as this technology has the potential to usher in changes across various industries. Car manufacturers are looking at it to measure the attention of drivers and warn them if they’re getting distracted. Doctors are utilizing it to detect autism in children. Law enforcement can use it for lie detection purposes. There’s just so much potential here.

As for the games industry, Werner confirmed that the first headsets with eye-tracking could arrive by late 2018 or early 2019. It’ll be exciting to see how VR developers adopt this tech into their games, as we can see it improving numerous genres. From aiming in first-person shooters to party games like Job Simulator, plenty can be done with eye-tracking. The sky is the limit for Tobii and it’s exciting to see where the tech goes.