This rapping robot named Shimon makes its own rhymes in real-time

Earlier in 2020, around February, Georgia Tech professor Gil Weinberg introduced an unlikely contender in the musical arena: Shimon the robot. The lean machine was presented as a non-human collaborator when it came to playing the marimba, singing songs, and now in the latest update to Shimon, it sounds like it can rap in real-time. Possibly even roast you.

The Scientific American reports that Shimon is capable of making its own lyrics, rhymes, and becoming a "rap-battle opponent." Shimon's antics are endearing. For one, the robot has metallic eyebrows that wiggle while it raps, making the whole deal a lot more amusing. It even has its own verified Spotify page, with songs like "Into Your Mind," "Under My Thumb," "Biological Inclusion," the intriguing "Gospel in Space," and "Consciousness Performs." Which, frankly, sound more like psychedelia song names than rap, but who are we to judge?

Shimon, a lyrical (and software) genius — Among other skills, rapping requires a smooth ability to improvise. It's a creative mode that demands quick adaptation to words and context. In other words, no stumbling allowed. For Shimon the robot, understanding rap comes down to text analysis combined with computing power.

Shimon's software fields the verbal cues from human opponents and converts those lyrics into concrete text. That text carries a rich amount of information for Shimon to parse. Shimon then uses that textual data as learning material. Historical data comes from Jay Z, Lil Wayne, and other iconic rappers to provide Shimon with broader context.

Most interestingly, Shimon relies on phenome data. It listens to how these words are pronounced, which amounts to distinctive audio information. By breaking down the text and phenome information it receives, Shimon then works on its own idea of a rhyme in real-time. And that's where the fun stuff happens. The robot rapper has to juggle quality and quantity, without compromising either. For this reason, his Georgia Tech innovators have kept its vocabulary limited to 3,000 words. It's not a lot but it still helps Shimon craft impressive comebacks.

At its core, Shimon is dabbling in speech-to-text and text-to-speech transitioning, but there is potential for it to get even stronger at rapping. Maybe one day it'll outshine human rappers. For now, though, Shimon's creators want to see if it's capable of rap-dueling with another robot. We're hoping it doesn't turn out to be creepy or offensive as we've witnessed with bots in the past.