Stunning A.I. results show how drones could become the next auteurs

You could soon be watching a movie that was partially directed by an autonomous drone. Researchers at Carnegie Mellon University have created a system that could allow a drone to learn how to capture scenes for a film from above. What’s more, these smart drones would shoot with artistic flair.

These researchers had drones film scenes from different perspectives and then had real people rate which shots they thought looked best. Based on this feedback, the drone was able to learn what kind of shots people thought were the most artistic and generally appealing.

Their work is outlined in a research paper titled “Can a Robot Become a Movie Director? Learning Artistic Principles for Aerial Cinematography,” which appears on arXiv, the repository of academic journal articles that haven’t yet been published in peer-reviewed journals.

Rogério Bonatti, who is pursuing his Ph.D. in CMU’s School of Computer Science, tells Inverse that his team learned a lot from the feedback they received from humans when it came to informing A.I. drones which angles made for the most epic shots.

“We had people rate the quality of several shots in different environmental contexts, and the system automatically learned which types of shots make the resulting movie more visually appealing. The system even learned that keeping the same shot type for too long makes the movie boring and that it should switch often, but not too often, otherwise the images become too confusing.” —Rogério Bonatti

The system also learned to avoid objects that could get in the way of a good shot. The drone was equipped with LiDAR, which uses pulsing lasers to determine proximity to nearby objects, so it could map its surroundings.

Bonatti did a lot of research before beginning this project. He says he “read several books on photography and cinematography” and interviewed several professionals from the entertainment industry. He wanted to learn what difficulties filmmakers face when they’re making a movie.

One problem filmmakers often have to deal with is getting good shots with drones, because it can take a whole team of people operating the drones to get the shot right. With an autonomous drone that can learn how to get the best shot without any help, things could become much simpler.

This video shows how a drone learns to use angles appreciated by humans to shoot video.

“The largest impact of our technology will be in democratizing professional cinematography to any person, anywhere,” Bonatti says. “High-end Hollywood productions will probably continue to be done using large groups of highly skilled professionals, because they involve scripted scenes and no margin for errors.

“Our system, in contrast, allows a common user to have the creative power of a filming crew while filming activities such as radical sports or social events.”

See also: Drones are Programmed to Flock Exactly Like Birds in New Breakthrough Study

This research focused on how to get good shots for a film with one autonomous drone, but Bonatti says the next step could be looking at how to get multiple drones filming a scene at once.

“We’re looking into using multiple drones as future work, which is a very exciting direction,” Bonatti says. “Besides extra caution on safety to avoid collisions between drones, an interesting challenge that arises in this case would be minimize how much one drone sees the other one in the final footage.”

Bonatti’s research has been accepted by the Journal of Field Robotics, and will be presented at the 2019 International Conference on Intelligent Robots and Systems in November in Macau, China.