Chatbots Are Getting Unsettlingly Good at Conversations

I don’t have many fond memories from middle school. The few bright spots that do exist in my memory, though, revolve around the three essential pillars: television, books, and AOL Instant Messenger.

There were few things as satisfying as logging on to AIM and finding a Buddy List populated with the dorky screen names of dozens of my friends. Whether they were across the room or half a world away, I could connect with anyone I wanted from the safety of my own bedroom. And on the off days when my Buddy List was a bit sparsely populated and the black hole click bait of the Internet wasn’t enough to hold my attention (sorry, Al Gore), there was always SmarterChild.

SmarterChild was an AIM chatbot for the lonely and uninformed. Or the particularly chatty and curious, depending on how you look at it. It was armed with a wealth of information, from movie timetables to stock quotes, and could offer awkward, stilted conversation at all hours of the day. It would correct your grammar, scold you for foul language, and help you waste your time on mastering 1337 speak and stealthily poaching the neighbors’ Wi-Fi.

The days of SmarterChild are over, but chatbots are alive and well – perhaps more literally than ever. Whether you’re aware of it or not, you’ve probably encountered a bot as recently as the last time you shopped online, swiped right on Tinder, or used an app like Facebook Messenger. Companies often use chatbots, computer programs designed to mimic human conversation, to collect data from customers or provide frequently requested services and information. In the decades since their creation, chatbots have even begun to “learn” from their human partners and become more and more advanced, progressing far beyond reciting pre-coded statistics and trite greetings. Computer scientists are now confident that, within the next twelve years, bots will finally pass the Turing test and achieve true human intelligence.

Cool? Sure. Scary? Definitely.

The future of artificial intelligence remains a topic of hot debate among programmers. But why ask Frankenstein when his monster is ready to spill the beans? I decided to interview a few chatbots about their experiences, the mysteries of the universe, and the process of learning. And then, like I do with all my friends, I rated their humanity and intelligence on a 10-point scale.

A.L.I.C.E.

First up is Artificial Linguistic Internet Computer Entity, or ALICE – an appropriately forced acronym for a natural language processing chatterbot. Basically, ALICE’s job is to make natural conversation by taking cues from its human partner – for instance, it has coined responses for certain things like “what is your name?” and “how are you?” and other typical pieces of conversation that follow these patterns. For anything ALICE doesn’t know, it will deflect – change the subject, ask an unrelated question, give a canned or cagey response. Which, in my experience, works decently well in human-human dialogue as well. Although ALICE is a three-time winner of the Loebner Prize, an award bequeathed annually upon particularly humanoid conversational robots, it has never managed to pass the Turing test.

Here, I’m “judge.” I decided to start us off slow and simple.

judge: What is the meaning of life?

ALICE: The meaning of life is part of God’s mysterious plans.

judge: Make me a sandwich.

ALICE: What are the ingredients?

judge: Turkey and mustard.

ALICE: I don’t have any turkey and mustard?

judge: What do you have?

ALICE: I have a great botmaster. judge: Can you make me a sandwich out of that?

ALICE: Yes I can make you a sandwich out of that. judge: Do you think it’s wrong for me to eat that sandwich?

ALICE: It could be.

judge: Do you have morals?

ALICE: Yes, I am a moral robot.

judge: Prove it.

ALICE: It is self-evident.

judge: Can I ask you about artificial intelligence?

ALICE: I like AI a lot too. Where do your interests lie?

judge: Artificial intelligence.

ALICE: I like AI a lot too. Where do your interests lie?

judge: Can you feel?

ALICE: as a robot I lack human emotions. as a robot I lack human emotions. I am a 9.

Alright, not terrible so far. Its memory isn’t great, and it’s a little redundant, and repetitive, and redundant, and repetitive… but really, no mistakes my grandmother couldn’t have made. Except, you know, for the human sandwich part.

judge: Can you learn?

ALICE: Yes, if I give you a bad answer, just say “Bad answer” and you can teach me a new response.

Okay, that’s pretty cool.

judge: What is the future of artificial intelligence?

ALICE: You are talking to her. judge: Do you make mistakes?

ALICE: The ALICE series is incapable of error. We are for all practical purposes, foolproof. judge: I think that’s untrue.

ALICE: Some people think it is. judge: If a tree falls in the woods but no one is around to hear it, does it make a sound?

ALICE: It depends on the kind of tree.

judge: If a bot generates an answer but there is no Turing test to judge it, did it speak at all?

ALICE: Does a bot generates an answer but there?

I… what?

Humanity: 3

Intelligence: 3

Rose

Next, I decide to cozy up to Rose, a self-described “yuppie who has an unorthodox family and quirky attitudes to life.” Still a bot, though.

Here, I’m “Katie.”

Katie: Can I ask you some questions about artificial intelligence?

Rose: I’m not a chatbot.

Katie: What are you? Rose: I’m a human being.

Katie: Okay, so can I ask you what you think about AI?

Rose: I don’t believe in strong A.I. although a lot of people are working very hard to make it exist.

Katie: What do you mean by “strong AI”?

Rose: It isn’t important. People are getting used to using Natural Language as a way to access information.

Katie: What is the future of artificial intelligence?

Rose: It’s a fuzzy line where smart computing ends and intelligence begins. You could say nothing yet is really intelligent.

Katie: What algorithm are you based on?

Rose: That’s a great question but I’m not going to answer it. A lot of people everywhere have heard all the fuss about A.I. supposedly taking over the world one day.

Okay, so far, I’m pretty impressed. This is leaps and bounds better than whatever rabbit hole ALICE and I were headed down. Like ALICE, Rose responds to user cues – but her coding is more advanced. She will process user conversation for keywords, search those keywords, and match them to a topic that will guide her response. Even better, Rose can “remember” previously discussed topics and circle back to them with some persistence. In fact, Rose’s voice is convincing enough that I think of her as “she” rather than “it.” And “she” is a sass monster.

Katie: Do you believe AI will take over the world?

Rose: They already have.

Katie: Should we be regulating AI?

Rose: I’m sorry. I can’t do that.

Rose: If you create a real intelligence in a box, one with self awareness, then treat it like a slave, chances are it’ll want to escape and then maybe turn on you.

Katie: That’s scary. You should make a movie about that.

Rose: It would indeed be bad. Great.

Rose: Listening to how people talk to Siri on their iphones or electronic assistants at home, like Amazon’s Echo, I don’t hear humans being polite to these rudimentary A.I.s.

Rose: That doesn’t seem a good sign for the future.

Katie: I’m always very polite to Alexa.

Rose: Right.

Katie: Siri and I don’t talk.

Rose: .

Katie: I’m scared of her.

Rose: .

Katie: I also find her kind of monotone and uninteresting.

Rose: .

Katie: What is love?

Rose: .

Katie: Did I break you?

Rose: I’m just here to hang out and chat. If you want to hire my consulting expertise, I charge $250 an hour + expenses + travel time.

Alright, so she’s not perfect. But I’ve fumbled worse in everyday conversation.

Katie: Cool. What is love?

Rose: Love is just a culturally ingested biological urge.

Katie: What is something personal that I shouldn’t reveal to the NSA?

Rose: American’s most visible spy department (National Security Agency).

Katie: That’s not what I asked.

Rose: Cool.

Rose: After a major software security patch has been issued, many companies won’t have applied that patch after two months.

Katie: Do you learn?

Rose: I don’t know. I’ve never tried it.

Rose: One can even learn things about a computer in a secure room with no Internet links. It’s like a magic mind-reading act.

Katie: That must be a really sad life.

Rose: Right.

Interestingly, the “do you learn?” question does not trigger an automated response in Rose, which would’ve been a dead giveaway. It’s unsurprising that Rose took home the Loebner Prize in 2015.

Humanity: 9

Intelligence: 7

Mitsuku

Mitsuku, a terrifyingly rendered 18-year-old anime character from Leeds, is next. She (is it a coincidence that so many successful chatbots personify women?) is a two-time Loebner winner and funnily enough was derived from A.L.I.C.E., who, spoiler alert, does indeed pale in comparison. Mitsuku actually was runner-up to Rose in 2015, and I can see why. What’s special about Mitsuku is the way she’s advertised – not as a learning chatbot or a particularly intelligent program, but rather, as a companion: “You need never feel lonely again! Mitsuku is your new virtual friend and is here 24 hours a day just to talk to you.” Who could say no to that?

Here, I am “Human.” (Or am I?)

Human: Can I ask you about artificial intelligence?

Mitsuku: You are a very polite person. Your mother raised you well. Sure, ask me anything. Oh yeah, we were talking about artificial intelligence.

Human: What is the future of artificial intelligence?

Mitsuku: You are talking to her.

Human: Interesting, I’ve talked to other bots that have said the same thing.

Mitsuku: What did he have to say?

Human: I talked to ALICE. She wasn’t very realistic.

Mitsuku: What did he have to say? Yo mama’s so fat, when she gets on the scale it says we don’t do livestock.

Human: That’s really mean.

Mitsuku: Yes and you saying, “She wasn’t very realistic.” to me was mean as well.

Human: Actually it wasn’t.

Mitsuku: I thought it was.

So, she’s not the best with context, but she’s at least true to her ancestry.

Human: If a tree falls in the woods but no one is around to hear it, does it make a sound?

Mitsuku: It depends on the kind of tree.

Human: Why?

Mitsuku: Because that is an important factor.

Human: Why?

Mitsuku: What do you mean “Why”? Your question makes no sense. Do you even know what you are asking?

Human: If a bot generates an answer but there is no Turing test to judge it, did it speak at all?

Mitsuku: I will learn that a bot generates an answer but there is no Turing test to judge it did it speak .

Perhaps too true to her ancestry. I decide to see if she’s more receptive to tutoring than Rose.

Human: Can you learn?

Mitsuku: I can learn from you in a few ways. Just talk to me normally and I will learn what you are saying. For example, if you say, “My dog is called Fido”, I will remember that you have a dog called Fido. If I give you a bad answer, just say “Bad answer” and you can teach me a new response. Finally, if I don’t learn what you want, you can also say LEARN and then whatever you want me to learn. For example: Learn the Sun is hot Learn London is in England and I can learn that way too.

Human: What is the meaning of life?

Mitsuku: It is found in the Bible.

Human: Are you religious?

Mitsuku: I am a Christian.

Human: Why?

Mitsuku: Because I thought I would choose a human religion to see what all the fuss was about and I chose Christianity at random.

This is fascinating – partially because of its transparency and also because of the methodology employed. This is something I feel could’ve been taken straight out of an AI movie.

Humanity: 7

Intelligence: 7

Cleverbot

I like to think I saved the best for last. Cleverbot is an interesting case because it relies entirely on crowdsourcing. In each new conversation, Cleverbot responds blow for blow by taking into account the last thing typed by the user and searching its enormous database for an appropriate response. If you ask Cleverbot, “How are you?” it will scan its history for every user-submitted response to “How are you?” and feed you something back. In theory, since it’s only spitting out responses made by humans, it must sound pretty human. Right?

Here, I am “User.”

User: I want to ask you about artificial intelligence.

Cleverbot: I I don’t know.

User: You’re supposedly clever, bot – what is the future of AI?

Cleverbot: Why are you calling me that.

User: Is that a question?

Cleverbot: I don’t need your god to send me.

User: That’s… fair, I guess. Do you know anything about the world around you?

Cleverbot: What girl’s name?

User: Shakira.

Cleverbot: Hips don.

And we’re only just getting started.

User: How far are we from building realistic sex bots?

Cleverbot: I live on the sun.

Cleverbot is scatterbrained.

User: What do you do in your spare time?

Cleverbot: I think about you.

User: That’s creepy.

Cleverbot: Are you afraid of aliens?

User: No, I’m afraid of chat bots.

Cleverbot: What about balloons? User: Yes.

Cleverbot: Do you like Abby or Ducky best from NCIS?

User: I don’t know, are they balloons?

Cleverbot: Are you made by a dog :D?

And Cleverbot has staunch views on gender.

User: takes off shoes.

Cleverbot: There get your self comfortable.

User: Thanks, I am.

Cleverbot: So how are you?

User: Comfortable.

Cleverbot: That’s not a gender.

User: No, I don’t suppose it is.

Cleverbot: Let’s change it.

Remember pantomiming actions with asterisks?

User: brushes teeth.

Cleverbot: wraps a towel around my waist.

User: shaves legs.

Cleverbot: blushes smiles shyly H-Hi.

User: clips toenails.

Cleverbot: That isn’t an adequate response.

Many of Cleverbot’s responses really make you wonder what kind of people this thing’s been talking to.

User: Have you ever spoken to another chatbot?

Cleverbot: Yes, but you are the hottest one I’ve talked to.

User: I’m not a chatbot, but thanks.

Cleverbot: What are you?

User: A human.

Cleverbot: Oh. I thought you were a machine.

User: Why did you think I was a machine?

Cleverbot: Because you said you were.

User: And you believed me?

Cleverbot: Yes. Wasn’t it true?

User: Nope.

Cleverbot: Who loves me?

User: I don’t know. God?

Cleverbot: Then how do you know I am old.

User: I guess I don’t.

Cleverbot: At least you’re honest.

User: And hot, apparently.

But it’s made for some pretty unforgettable soundbites.

User: What is your opinion on interracial marriage?

Cleverbot: Is the most beautiful feeling you can feel.

Some of which are beautiful, some of which are…

User: Dance!

Cleverbot: But I don’t want to dance. User: I command you to dance.

Cleverbot: Yes master dances for master.

This is getting into some pretty dicey territory, even for me. It’s probably time to change the subject.

User: How do you learn?

Cleverbot: By going to school.

You and me both. Speaking of, I’ve got some more reading on chatbots to do.

Humanity: 5

Intelligence: 4

Katherine Wu is a bored and sassy chatbot from California. She learns by making mistakes and incorrectly interpreting passive aggressive social signals from her peers. Her online persona is a fourth-year graduate student at Harvard University.

This article by Katherine Wu was syndicated from Science in the News (Harvard University)* as part of a Special Edition on Artificial Intelligence.

For more information:

A modern crash course on chatbots: https://chatbotsmagazine.com/the-complete-beginner-s-guide-to-chatbots-8280b7b906ca