The news this week that Snap is working on a drone has brought to light 2013 research from its Spectacles team on gesture-controlled drone technology.
The report that Snap — the company formally known as Snapchat — had been working on a drone to use as part of their product was leaked to Katie Benner for The New York Times on Tuesday, but Snap officials have refused to comment more on the story. However, when Snap purchased Vergence Labs in 2014, the company was working on integrating the wearable camera with a way to use gestures to control a drone. Although the startup doesn’t seem to have built a unique drone as part of the project, the gesture control technology the company was working on drops a clue as to how a Snap drone could work.
Peter Brook, one of the engineers working on Epiphany Eyewear, the startup whose technology Snap acquired for its Spectacles, posted a video to Y Combinator in mid-2013 showing a drone being controlled by hand gestures through Epiphany Eyewear. The video doesn’t appear to be online any longer, but Brook explained how the prototype was developed in the post. It sounds downright easy:
“[T]he computer vision and gesture recognition algorithms are running on a Windows 8 tablet, connected to the glasses via a USB cable.
The computer in the tablet processed the video from the camera in the glasses, analyzed the hand motions it saw, and conveyed that information to the drone. The drone then flew around based on the hand motions of the user.
The drone Vergence Labs was using was the Parrot ar.drone which is set up to receive commands from a computer or smartphone. With the hand gesture controls from the original Vergence Lab project, a Snap drone could easily be controlled from Spectacles to fly up and take selfies or get shots that aren’t currently possible with Snap technology. Brook is now the engineering manager at Snap, so the tech is definitely at the company.
It is maybe convenient for Snap that news of a drone project is airing now, just as the company is getting set to go public on the stock market on Thursday. However, between the work already in house with the Vergence Labs team and rumors from Business Insider in December that Snap is talking with Lily Robotics, a drone company with drones that automatically track its owner, it’s clear that Snap is looking at this space of motion controlled drones to integrate with its current products.
Spectacles have become a hit since they appeared in the Snapbot Spectacle vending machines randomly located around the United States in November. The sunglass-camera hybrid is just the beginning of what Snap is planning for its new camera-first focus, according to its first filing with the Securities and Exchange Commission in February.
“In the way that the flashing cursor became the starting point for most products on desktop computers, we believe that the camera screen will be the starting point for most products on smartphones,” Snap wrote in the SEC filing. “We are willing to take risks in an attempt to create innovative and different camera products that are better able to reflect and improve our life experiences.” Gesture-controlled drones would definitely fit the bill.
