April Fools "DeepDoggo" A.I. Can Tell If Your Dog Is a Good Dog
Who’s a good dog? This is one of the fundamental questions for modern computing science, and Carnegie Mellon researcher Ben Lengerich decided to use the incredible power of deep learning neural networks to tackle it head on — and make a comment on the state of A.I. hype, while he was at it.
DeepDoggo is a service that allows users to upload a photo of any dog and learn whether science deems this dog to be a good dog or a bad one, based on the picture. It is accompanied by a paper published for this year’s Special Interest Group on Harry Q. Bovik SIGBOVIK conference, an annual April Fool’s event at Carnegie Mellon (if you reload the SIGBOVIK site, it generates a new Q-based middle name for their conference founder each time).
“In the DeepDoggo paper,” Lengerich wrote in an email to Inverse, “I wanted to poke some fun at the hype around deep learning.”
Lengerich’s main point about A.I. was that machine learning is only as good as the processes and standards it’s based on. If you start with nonsense, the relative “accuracy” of a system’s predictions don’t matter. Lengerich used a totally stock downloadable neural network (the paper declares it is the correct model to use because it “is easy to download in Tensorflow”), and provided the “true” judgment of whether a dog was good and bad based on Google searches.
“I used Google Image search to generate two different datasets of images,” he explains. “Those that Google Images labels ‘good dogs’ and those that Google Images labels ‘bad dogs’.”
Their trained model achieves a stated 73% accuracy in predicting which group a picture would fall into — which certainly sounds pretty good until you remember that it’s all based on a meaningless standard.
“When we rely on predictive accuracy to evaluate neural networks, we can get a false sense of confidence about what the network is doing. Just because it has 90% accuracy doesn’t make it a ‘good’ model. While a neural network can learn to differentiate these classes of images, the classes don’t really mean anything, so analyzing the results doesn’t mean much either,” Lengerich said. “But it can still be fun.”
Still, we decided to give it a little test with three images of good, bad, and ambiguous doggos. (When asked just what a doggo actually is, Lengerich replied: “A big ol pupper.”)
Lassie. Truth: Very good dog. DeepDoggo Prediction: Bad dog! (Good Doggo Score: 0.3708851773040486).
Carface Carruthers. Truth: Very bad dog. DeepDoggo prediction: Good dog! (Good Doggo Score: 0.6058659181957677).
My Friend Clinton’s Stupid Dog. Truth: I guess he’s alright. DeepDoggo prediction: Bad dog! (Good Doggo Score: 0.4422879308476644).
Truly, these are very terrible results — though that’s to be expected. Though Lengerich did actually train the neural network for the paper, it won’t actually get hooked up to the site itself until the weekend, so the “Good Doggo Scores” are currently a big, bad scam. If you check the site in a few days, the numbers might start to make more sense, but not that much sense, of course. The goodness of a doggo is something every human or A.I. must determine for themselves.
