This year’s Google I/O revealed that the tech company is nearing the completion of an artficially intelligent secretary that can book appointments without you having to lift a finger. But the science fiction dream of a robotic housekeeper, like Rosie from The Jetsons, remains elusive.
That has a lot to do with the fact that in many respects A.I. remains incredibly dumb when asked to operate in the real world. Sure, it can ridiculously complex calculations at blazing fast speeds — a narrow application — but when confronted with mundane tasks like making your bed, A.I. perpetually comes up short. This is exactly why a group of computer scientists led by the Massachusetts Institute of Technology is teaching machines to complete household tasks using a The Sims-inspired “VirtualHome” simulator.
The researchers first crowdsource a large knowledge base of household tasks (top). Each task has a high level name, and a natural language instruction. They then collected "programs” for these tasks, (middle left). Where the annotators “translate” the instruction into simple code. They implemented the most frequent interactions in a 3D simulator, called VirtualHouse, allowing them to drive an agent to execute tasks defined by programs. They proposed methods to generate programs automatically from text (top) and video (bottom) thus driving an agent via language and a video demonstration.
In a new paper that will be presented at the Computer Vision and Pattern Recognition (CVPR) conference beginning June 19, the researchers describe how they broke down tasks into simple steps that even a bucket of bolts could understand. This allowed their A.I. to complete a thousand chores, like “turn on the TV” or “pour milk into a glass.” The idea is that video game-like environments could be the key to training the household robots of the future.
“This line of work could facilitate true robotic personal assistants in the future,” said Qiao Wang, a research assistant in arts, media, and engineering at Arizona State University in a statement. “Instead of each task programmed by the manufacturer, the robot can learn tasks just by listening to or watching the specific person it accompanies. This allows the robot to do tasks in a personalized way, or even someday invoke an emotional connection as a result of this personalized learning process.”
Bringing Robot Butlers to Life
This would only be the first step in creating real-life robot butlers. Transferring what A.I. can do inside of a video game to the physical world will require some pretty high-tech hardware. Partnerships seem likely: robotics companies like Boston Dynamics, for example, could software like this to train their army of bots.
In the shorter term, the authors say this research could help companies like Amazon or Google develop a robotic system built around their smart speaker technology. Since the Google Assistant will soon be able to set up appointments for you, research like this could help enable A.I. that’s sophisticated enough to actually pick up your phone and make a call on your behalf.
“You can imagine a setting where robots are assisting with chores at home and can eventually anticipate personalized wants and needs, or impending action,” said the lead author of the study, Xavier Puig. “This could be especially helpful as an assistive technology for the elderly, or those who may have limited mobility.”
So get ready to kick back, relax, and have a robotic butler sweep your floor, make your bed, and even dream up new ways to help you better organize your life. Hopefully, it’s not as mouthy as Rosie.
