HoloLens 2: How Microsoft’s AR Headset Uses Advanced Sensors to Track Hands

Microsoft’s upgrade to the HoloLens has arrived. The company’s augmented reality headset, first introduced in 2015, received a major upgrade on Sunday at Barcelona’s Mobile World Congress. HoloLens 2 focuses on better comfort, improved support for users, and greater immersion — the latter of which has received a big boost from sensors that can track the wearer’s hands.

It’s a big breakthrough for the headset, which looks set to continue its focus on business applications. The original HoloLens used a combination of gesture and gaze, where users would move their head to aim at the target and pinch their fingers in front to “click.” The new version offers a far simpler approach, where users can directly interact with virtual objects with their hands, picking it up and moving it around or even directly pressing buttons. This is done with a new Azure Kinect time-of-flight sensor located in the band above the eyes, combined with what Microsoft refers to as “built-in A.I. and semantic understanding.”

See more: Will the Highly Anticipated Microsoft HoloLens Drop at MWC 2019?

Time of flight sensors send out beams of either lasers or near-infrared light to bounce off nearby objects. They measure the distance either through pulsing the light and measuring the time it takes to return or measuring the phase change of the wave as it returns. The technology works similar to Microsoft Kinect, the peripheral for Xbox 360 and Xbox One that beams infrared light to enable interaction with video games. Kinect developer Alex Kipman helped create the original HoloLens. Another implementation, developed for the iPhone X, scans a user’s face by beaming 30,000 infrared dots.

Beyond the hand sensors, Microsoft has made a number of improvements to HoloLens. It offers a much wider field of view, with 47 pixels per degree of sight. It also has eye scanners pointed at the wearer that enable Windows Hello iris recognition for faster logon, eye distance measuring to adjust for greater comfort, and improved interaction with holograms. It also uses vapor chamber technology to make the headset more comfortable to wear, while sliding over so users can keep their glasses on during use.

Microsoft plans to sell the HoloLens 2 later this year for $3,500. It’s also offering an Azure Kinect developer kit for preorder starting now at $399, which combines the HoloLens’ time of flight sensor with a high-definition RGB camera and seven-microphone array to enable developers to use the technology outside of the headset.

All this technology means that beyond understanding hands, HoloLens 2 will move beyond and scan a room for objects like couches and people. As developers get their hands on the new kits, its most exciting uses could come in the future. Update 2/26 4 a.m. Eastern: An earlier version of this story claimed the HoloLens 2 price was $35,000. It has now been corrected to $3,500.