How This Robot Learned to Conduct an Orchestra

Watch out, orchestra conductors. Even your job may not be safe from the threat of automation.

A robot named YuMi made its conducting debut Tuesday at the iconic Teatro Verdi theater in Pisa, Italy. Leading a full orchestra — including famed classical tenor and recording artist Andrea Bocelli — at a charity concert for more than 800 guests, YuMi impressed both audiences and the seasoned musician.

“It showed that a robot could really conduct an orchestra, but only with the excellent work of very talented engineers and a real maestro,” Bocelli said following the performance.

Andrea Colombini, an Italian conductor who worked with ABB, the company behind YuMi, explained in a blog post just how challenging this feat was.

Conducting — let alone leading a world-class orchestra — is a delicate art that relies on subtlety and precision. Teaching YuMi how to conduct was a two-part task.

“First, my movements were captured with a process called ‘lead-through programming,’ where the robot’s two arms are guided to follow my motions with great attention to detail,” Colombini wrote in the post. “The second step involved fine-tuning the movements in ABB’s RobotStudio software, where we made sure the motions were synchronized to the music.”

By the end of the process, Colombini says that the robot was able to achieve “a very high level of fluidity of gesture” typically unseen in the way traditional robots are able to move. Because of the way YuMi was programmed, with its elbow, forearm, and wrist capable of both operating independently and interacting with one another, it was even able to break up specific motions for upbeats and downbeats, which is key to successful conducting.

“Of course, YuMi is good when it comes to technique but is ultimately not gifted with human sensitivity,” Colombini wrote. “The robot uses its arms, but the soul, the spirit, always come from a human.”

While that might sound like someone worried about automation trying to convince everyone human conductors are still needed, he actually foresaw potential for a valuable partnership between humans and machines.

“I imagine the robot could serve as an aid, perhaps to execute, in the absence of a conductor, the first rehearsal, before the director steps in to make the adjustments that result in the material and artistic interpretation of a work of music,” he wrote.

Colombini even went so far as to write he and YuMi became good friends once they understood each other. Maybe there’s hope yet for human-robot relations, at least in the world of classical music.