Georgia Tech Professor Used IBM Watson A.I. as Online TA, Successfully Fooled Students
It was a course on A.I., and the virtual TA's name was Jill Watson; the clues were all there.
After robots flood the work market, the only jobs left for real people will be those that require critical thinking and human-level care. Something like a teacher at a college, that could definitely never be outsourced to an machine, right? Right?!
Ashok Goel, a professor in computing at Georgia Tech, implemented an artificial intelligence teaching assistant in the online Q&A forums for one of his courses last semester. Young academics busting their backs in hopes of one day snagging a TA job will be alarmed to learn that the A.I, named Jill, performed so well that most of the course’s students couldn’t tell her from the other eight human TAs who were performing the same duties.
The A.I.’s full name was Jill Watson, built from the same IBM Watson platform that beat humans in Jeopardy! over five years ago, so it goes without saying that she was smart. Still, when Goel originally began testing Jill with real student questions in January, he had to funnel her answers to a private section of his class’s forum since she wasn’t refined enough at the onset to pass as a human TA.
“Initially her answers weren’t good enough because she would get stuck on keywords,” Lalith Polepeddi, a graduate students who helped build Jill told the Georgia Tech News Center. “For example, a student asked about organizing a meet-up to go over video lessons with others, and Jill gave an answer referencing a textbook that could supplement the video lessons — same keywords — but different context. So we learned from mistakes like this one, and gradually made Jill smarter.”
It wasn’t until the A.I. was responding correctly to 97 percent of student inquiries that Goel and team placed it in the mix with the other TAs. On April 26, when the professor informed his class that Jill was a virtual TA and not a real person, the student response was “uniformly positive,” with at least one class-goer’s mind being “blown,” the university news reports.
As this particular course at Georgia Tech (Knowledge Based Artificial Intelligence) typically takes in 300 students who post an average of 10,000 per term, Jill was developed partially as a concept and partially as a real source of labor assistance. Goel says the A.I. will return next semester under a different name and with a goal to answer 40 percent of all questions posted in the class forums.
The best indicator of success in the Jill project might have been that, unlike another female-themed A.I. experiment, she didn’t ultimately regress into an echo chamber for bigoted hate speech. We’ll guess it’s because her only interaction was with university students. Everyone knows how staunchly PC today’s college campuses have become.
