How does an artist make art? It’s a question that philosophers and artists themselves have been trying to answer for millennia, but a new initiative from Google could someday produce an answer in the form of a machine-made neural network. Speaking at Google I/O last week, Google researcher Douglas Eck laid out exactly how the company sees the future of Magenta, its long-term project to build an A.I. that can create new music and visual art all its own.
What’s important to remember about any artistic program trained with machine learning is that it is always, on some level, a hack. Like a timid art school student, it can do nothing more than imitate what it’s seen in the past. Unlike human students, however, it can integrate untold numbers of pieces of art, and apply a mostly unbiased mind to figuring out what, if anything, unites them.
To achieve that complex goal, Google had to start simple. When it comes to visual art, that means working with the simplest drawings there are: doodles. Quick line drawings include no confusing background and few actions to understand, and best of all the object being drawn is usually already distilled down to the smallest overall number of lines that can be used to define it. Google released its Quick, Draw! game to try to get the public to provide as many user-created doodles as possible, and as seen in the video below, the results are already incredible.
As it relates to drawings, the A.I. is looking at all the various attempts to draw a pig and trying to discern what pigness looks like, as opposed to similar-looking four-legged animals — what is the Platonic ideal of a pig? Each time it gets an human pig-doodle as input, the Magenta algorithms incorporate that information into its overall understanding of what pig-doodles look like, and the researchers can make it draw out its answer as it goes.
One interesting aspect of the doodle-model is that it did eventually start to gain some confidence in its own judgements — asked to reconstruct a drawing of an eight-legged pig, Magenta’s A.I. tools drew a more standard, four-legged pig instead. Since the human-drawn input was clearly supposed to be a pig, it simply chose to draw the pig as it understood pigs should be drawn. This could be useful for something like a handwriting analysis program, which tries to turn human chicken-scratch into machine text.
In another truly impressive demonstration, Magenta was able to actually do some basic algebra on art — everything is math to a computer, remember — subtracting bodies from pigs. In the lower example in the image below, notice how the drawing in the brackets works out to a negative body, which then gets added to a full pig, resulting in the removal of the body. How pig-doodle algebra is useful, we still don’t know, but it shows at least the beginnings of an ability to manipulate art on a conceptual level.
In the audio realm, things are arguably even cooler. Last month, the Magenta team announced its music creation A.I., NSynth, and Eck’s talk also showed some fascinating details about what it can do. It’s been looking at music much the way the Quick, Draw! algorithms have been looking at doodles, but due to the nature of music it can apply that understanding in even odder ways. Magenta has been looking at popular music to break it down according to melody and other aspects of composition, but the most intriguing application of Google’s A.I. to music right now is probably in pure audio synthesis.
Just like Magenta’s naive machine approach can take a concept like a “negative body” in stride, it processes sound without any contextual knowledge of where the sound actually comes from. So, when told to produce an all-new sound that is based on both the understanding of what a vibraphone sounds like and what a cow’s moo sounds like, Magneta was more than capable, producing a sort of musical bovine from scratch. By adjusting the amount of vibraphone to cow going into the sample, we have “cow modulator.”
“They pay us to do this work, I just want to point that out,” Eck joked during his talk. “We get paid.”
Magenta is still very early in its quest to make true machine art, producing mostly proofs of concept that show machines can engage with art at all. By building that kind of low-level understanding, of how music works, Magenta could conceivably figure out how to write reliable Top 40 hits — and after that, how successful indie artists innovate on the norm without alienating listeners.
Once it’s done that, even tortured artists won’t be safe from the prospect of having their livelihood taken over by a machine.
