How the A.I. Chip in Microsoft's HoloLens Could Make Wearable Tech Work

The next generation of Microsoft’s augmented reality headset, HoloLens 2, will feature a custom-made artificial intelligence chip, the company announced Monday. The new chip will be capable of analyzing speech and visual data directly without having to send the information to a cloud server for analysis. That change is meant to speed up performance for the Hololens 2 and make it work better as a mobile device.

The new A.I. “coprocessor,” a silicon chip designed but not manufactured by Microsoft, is meant to move the HoloLens further in terms of accurate recognition of different visuals and sounds being collected by the device. Typically, the HoloLens and other devices (like Siri on the iPhone) have to use cloud-based servers equipped with deep learning networks to make sense of the data the user provides. While this ensures more accurate output, this also has the potential of slowing the device’s functionality.

An A.I. chip integrated into the device itself allows for on-site analysis of data. The original HoloLens chip is, according to the researchers, a custom multiprocessor called the Holographic Processing Unit (HPU), which sifts through all of the device’s onboard sensors. That includes “time-of-flight depth sensor, head-tracking cameras, the inertial measurement unit (IMU), and the infrared camera.” An A.I. component on the chip simply allows the device to take this data and make sense of it almost immediately.

While such a chip pushes the image and speech recognition abilities of virtual and augmented reality applications to more efficient standards, it’s also a highlight of a larger trend among tech developers to develop customized A.I. hardware for specific needs. Onboard A.I. processors can cater to very specific functions better than a one-size-fits-all cloud server might be able to. Other companies like Apple are looking into the same kind of approach for their own mobile devices.

“This is just one example of the new capabilities we are developing for HoloLens, and is the kind of thing you can do when you have the willingness and capacity to invest for the long term, as Microsoft has done throughout its history,” wrote Marc Pollefeys, the director of science for the Hololens team. “And this is the kind of thinking you need if you’re going to develop mixed reality devices that are themselves intelligent. Mixed reality and artificial intelligence represent the future of computing, and we’re excited to be advancing this frontier.”

Of course, a single piece of hardware has its limitations. The HPU for the Hololens will be cutting edge — until it becomes obsolete. Microsoft won’t be able to ditch relying on a cloud server outright. The new chip, however, does it make it more possible to enjoy wearable technology without the tethering need for a good internet connection at all times.