Science

Help Us, Augmented Reality. You're Our Only Hope. 

The rise of AR communication brings Star Wars-style holograms one step closer to reality. 

by Russell Stein
A woman using smart glasses for augmented reality phone calls.

Of all the sci-fi wonders in Star Wars, augmented reality phone calls — even though nobody called them that then — offered a vision of the future that was incredible. While FaceTiming with a friend who lives across the ocean is cool, Princess Leia stunned audiences back in 1977 when she beamed her holographic one-way message to Obi-Wan Kenobi. Best of all, the space-age tech that mixes the real with virtual world is just around the corner.

“Using Skype video chat, we’re confined to a boxed area where our head and shoulders take up most of the viewed video feed real estate,” Matthew Szymczyk, CEO of LA-based augmented reality startup Zugara, tells Inverse. “I think we can agree that it will be much cooler if we’re literally standing side by side with someone virtually while having a conversation.”

The possibilities for AR could fundamentally change communication. While video calling made the world feel smaller and enabled far-flung business meetings, AR can make people feel as if they’re in the room. If the smartphone first made people feel constantly connected a decade ago, it’s almost unfathomable how AR in Google Glass-style headsets will reshape our understanding of relationships.

This technology has been a long-time coming: Filmmakers have been experimenting with mixing worlds ever since The Great Train Robbery wowed audiences in 1903. More than a century later, Apple’s Mac OS X Leopard introduced real-time iChat effects in 2007; they enabled users to switch their background for something more exotic, fooling your friends into thinking you were on a high-intensity rollercoaster (sort of).

Zugara took this a step further with its Zugstar augmented reality video calls back in 2010. It was here that the real-world applications became apparent: doctors can explain procedures with diagrams, friends can try on overlay dresses and share opinions, and kids can play with building blocks using their hands and talking to friends.

As processors get faster and smaller, augmented reality can process even more complex effects — and the next generation will blow away what we have today.

“I think with mobile devices, we could see holographic displays becoming more prominent,” Szymczyk says. “There was a good example of this in the recent Marvel movie, Black Panther.”

Microsoft has given the world a glimpse of how we get from technologies like Zugstar to the ones used in the fictional world of Wakanda. In 2016, the year after it first unveiled the HoloLens augmented reality headset, Microsoft showed off something even better: a 3D communications system called Holoportation that scans users and presents them to the other user as a three-dimensional object inside the recipient’s headset — complete with demo reel of a virtual guy playing guitar to his friends across the world.

The system uses just two cameras and a connection speed between 30 and 50 megabytes per second, and the team hopes to bring the requirements down to connect over cellular data. In May, Microsoft announced a key step on the journey to Holoportation: Remote Assist, which places a video call inside the augmented world.

“Microsoft Remote Assist is focused on commercial use cases at this time, primarily for firstline workers where we see immediate benefits such as remote instruction,” a spokesperson for Microsoft’s mixed reality division, tells Inverse. “Forward-looking technologies like Holoportation are more complex but will likely become more prevalent as mixed reality mainstreams.”

Remote Assist is still confined to that small Skype video chat, but the hands-free design and the mixing with the real world make it ideal for problem solving from a distance. Workers can use it on the factory floor to troubleshoot an issue, IT experts can assist with hardware issues, or someone can walk a family member through programming their DVR. These sound like small uses, but getting Remote Assist in businesses could fuel more ambitious projects.

“Microsoft HoloLens is on a multi-year journey and we are currently focused on developer and business scenarios,” Microsoft’s spokesperson says.

Princess Leia won’t start beaming messages overnight, though: And as Microsoft rolls out Remote Assist, Apple has taken its own steps by expanding those iChat effects onto the iPhone X. Users can send Animoji messages that augment faces with unicorn masks, similar to Snapchat filters, while last year’s iOS 11 included an “ARKit” to make developing augmented reality apps easier.

“The example I always like to use is the cartoon with the thought bubble of a light bulb going off when somebody gets an idea,” Szymczyk says. “That’s visually what we’ll be seeing in the near future with communication.”

The Star Wars-style future is coming, and the dog face filter could be the next step on that journey.

Related Tags