Microsoft Wants Its A.I. to See and Describe the World to the Blind
Ordering food through a bot is cool, but what other uses could come from this innovation?
By far the biggest news out of the first day of Microsoft’s annual Build developer event was its introduction of a bot ecosystem, in which companies can develop their own bots designed to respond to specific customer service needs, take orders for deliveries, or even plan vacations — from transportation to hotel and back again.
The bots communicate with the Cortana Intelligence Suite and run on what the company calls the Microsoft bot framework. Microsoft demoed the tech by showing a chatbot follow-up with a user about hotel arrangements after a remote meeting was scheduled. Cortana was then able to fill in location and arrival time data through its ability to learn preferences.
Microsoft also showed how the bots could be used to order a Domino’s pizza and cupcakes.
Ordering food via chatbot is pretty cool but Seamless, Postmates, DoorDash, and the new UberEATS already make that task pretty painless, even if you can’t chat at them. This is to say nothing of the plethora of travel services a chatbot system would have as its competition.
With those territories covered, where can it make an impact? Here’s a rundown of promising options for Microsoft’s chatbot technology.
Describing the World With a Chatbot
The company showed off the impressive Microsoft Cognitive Services, which utilizes chatbots to help the blind “see” the world around them. The tool allows users to take snapshots of their surroundings and have a bot describe to them what he or she is hearing.
In a demo video for the project, Microsoft engineer Saqib Shaikh – who has been blind since the age of seven – takes a photo of a skateboarder doing a trick and has it described to him by a bot. He’s also able to snap a picture of a restaurant menu with his phone and have a bot read it aloud to him. The system is also able to read and describe the faces, expressions, and relative ages of people in the room, so Shaikh can know how they are reacting to his presentation.
Medical Research
More and more apps are allowing users to stay out of the doctor’s office, and even though many flock to services such as WebMD or just Google, sites like those are not able to give real-time responses to a user’s medical ailments.
But what if bots could?! Simply tell the A.I. some of your symptoms and it may be able to give you some baseline treatments before it inevitably has to say “please consult your doctor.” Perhaps, one day you could even take a photo of an affected area and receive advice on how to deal with it.
It would save a lot of patients time and money, especially those who may not be able to afford a doctor’s visit.
Artificial intelligence is already present in other technical and engineering fields such as Amber, the assistant that can help vendors buy parts from suppliers. A whole suite of bots could further help streamline these sorts of systems and provide vital information, along with delivery.
Microsoft shows off the security features on its bot program.
Customer Service
Time Warner Cable and Comcast have some of the lowest consumer satisfaction ratings in the country, so no service sector is in more need of a chat bot system than these telecom companies.
Imagine a day when you don’t have to wait on the phone to get your cable box fixed, or allowing an installation bot to communicate with Cortana to set up an appropriate delivery time, not a five-hour time window.
Both companies already have live chat services, but they haven’t done much to help the services’ image. One Comcast Xfinity subscriber famously chronicled their conversation with a “live” agent, but it became obvious this was either a broken computer program or a very bad communicator.
A customer shows reports a conversation had with a real or fake chat service.
Perusing the top complaints from Time Warner Cable users, it’s clear their needs haven’t been met either.
One customer explains his dissatisfaction with Time Warner Cable.
Maybe with this new program Microsoft can help us arrive at a future where A.I. is better at assisting us instead of just being racist. (You didn’t think we’d end without mentioning Tay Bot, did you?)
