I Saw Inside the Human Brain in Augmented Reality

Protesters poured into the streets of Park City, Utah on Saturday, participating in an offshoot of the massive marches that took place in major cities across the world. The march, which served to protest President Donald Trump’s history of sexism and the divisive policies he has promised to enact, virtually shut down the streets and highways that lead to the Sundance Film Festival; with the busses all screwed up, I jumped at a chance to catch a ride with a few people who work for the tech startup Meta. And during the course of a 90-minute ride down a route that usually takes 15, we spoke about the company’s new augmented reality technology and their plans to fundamentally change the way we consume information and interact with the world.

While much of Sundance’s New Frontier focuses on virtual reality entertainment, Ryan Pamplin, the company’s VP of Sales and Partnerships, is betting on A.R. to have a broader impact both on creativity and education — and soon. The Meta 2, the company’s new headset, and its Unity-based development kit, are now being shipped to early adopters who are willing to shell out the $949 to get a jump on the new technology.

Meta, which launched with a Kickstarter in 2012, has raised over $75 million in venture capital from major VC firms and corporations, including Comcast, Dolby, and China’s Tencent Media. It is one of the major three augmented reality companies, along with Microsoft (which makes the HoloLens) and Magic Leap (which has a more advanced hand-tracking system). So there is real competition, but also plenty of avenues for potential profit; Pamplin is focused on navigating those opportunities, having comes from the advertising world, as a co-founder of BrandAds.

But at Sundance, they’re focusing on selling people on an educational proof of concept. Pamplin and his team, including David Oh, the company’s head of developer relations, were on their way to the installation of their new exhibit, Journey to the Center of the Natural Machine. It’s an educational short about the human brain, and it shows off the technology’s early capabilities.

When we finally got to Main Street, they gave me an early look at the installation. The Meta 2 headset is a bit like an Oculus or Vive, except it has plexiglass-like lenses that allow you to see everything around you, instead of enclosing you in a dark space. It’s like wearing bike helmet with big glasses, instead of a high-tech blindfold. Ryan sat across from me, wearing his own Meta 2, and we watched the three-minute video together … and took turns interacting with the images we saw on our own glasses.

The lenses have a 90-degree field of view, though I had to move my head up and down a bit to see everything on screen. That was a minor inconvenience, though, as dynamic 3D images were being projected right in front of me, which was way better than watching a movie on my laptop, as I had earlier that morning. After a brief overview of the functions that each part of the brain performed, we were tasked with grabbing and pushing different lobes toward the model in the “center of the room.” It took a few tries to get the pieces to move, but we were operating off a slow tethered internet connection due to a DDOS attack on the festival’s wireless internet.

Still, it was a much more dynamic learning experience than looking at a book or even a video on a flat screen; had I been able to use augmented reality in school, I may not have been such an awful science student. (That’s probably not true, but it would have helped me conceptualize in three dimensions the stuff I still didn’t understand). Pamplin says that he’s had a meeting with the head of China’s Ministry of Education, who told him that they “have more students than [the United States] has people,” and that they envision teaching many of them with augmented reality in the near-future.

Technically, anyone with a Meta 2 can sign on to the platform, which is run on Windows right now, and download the first wave of applications. The company is both working in-house on applications and working to “evangelize to as many developers and creators as possible.” They hired Eran May-raz and Daniel Lazo, the directors of a viral short film called Sight, as an in-house filmmaking team; the short is a bit of a cautionary tale about a future deluge of augmented reality, but they moved from Israel to help Meta use the technology to create new kinds of visual art and storytelling.

Meta built a whole video studio in its offices in the Bay Area, and this spring will begin to invite filmmakers to make use of the facility. While 360 cameras are becoming more and more common, Meta is using a lightweight, real-time rendering volumetric camera developed by engineers at Carnegie Mellon; May-raz and Lazo will be in charge of helping filmmakers at all different levels learn to use the technology.

That might be the most flashy application of the technology, but Meta is really focused on 3D modeling, automotive clients (automakers could teach technicians how to fix a car without having to send models across the country), and healthcare clients. It’ll be a lot easier for a surgeon to explain to a patient what they’re doing — and for the surgeon to practice the procedure — if there is an interactive augmented reality model readily available.

Right now, the Meta 2 is tethered to a computer — Windows-only at the moment, but it’ll be Mac-compatible soon — with a nine-foot cord. The goal is to have a “prosumer” model out in 2018, and glasses on the market in 2020. That portability will be essential to achieve the Meta’s goals, which include eradicating the need for a smartphone. For example, instead of looking down at Google Maps for directions, they want to highlight the trail right in front of your eyes. And being able to watch TV across from someone on a semi-transparent screen, and move the display around to track your head, is another idea that will soon be possible, even without the glasses.

Getting the glasses to market will be essential, as well, given that HoloLens is already hands-free. But there is also a sense, at least at Meta, that a rising tide lifts all boats, and Pamplin says he is very happy to see third party developers use their platform to make products that are also used on HoloLens and Magic Leap. Estimates are that the A.R. market could reach $120 million by 2020, making 2017 a crucial year for the company.

Just think: If the next presidential election goes like this past one, protestors across the world might be able to put on their Meta glasses and really feel like they’re part of the mass marches in cities as big as New York and as small as Park City.