Ian Bach

Viewing conflicts through the eye of Counterinsurgency COIN – Since 2007

Robots Learn by Watching Videos

Autonomous robots can learn and perform complex actions via observation

Imagine having a personal robot prepare your breakfast every morning. Now, imagine that this robot didn’t need any help figuring out how to make the perfect omelet, because it learned all the necessary steps by watching videos on YouTube. It might sound like science fiction, but a team at the University of Maryland has just made a significant breakthrough that will bring this scenario one step closer to reality.

UMD computer scientist Yiannis Aloimonos (center) is developing robotic systems able to visually recognize objects and generate new behavior based on those observations. Photo: John T. ConsoliResearchers at the University of Maryland Institute for Advanced Computer Studies (UMIACS)partnered with a scientist at theNational Information Communications Technology Research Centre of Excellence in Australia (NICTA) to develop robotic systems that are able to teach themselves. Specifically, these robots are able to learn the intricate grasping and manipulation movements required for cooking by watching online cooking videos. The key breakthrough is that the robots can “think” for themselves, determining the best combination of observed motions that will allow them to efficiently accomplish a given task.

The work will be presented on Jan. 29, 2015, at the Association for the Advancement of Artificial Intelligence Conference in Austin, Texas. The researchers achieved this milestone by combining approaches from three distinct research areas: artificial intelligence, or the design of computers that can make their own decisions; computer vision, or the engineering of systems that can accurately identify shapes and movements; and natural language processing, or the development of robust systems that can understand spoken commands. Although the underlying work is complex, the team wanted the results to reflect something practical and relatable to people’s daily lives.

“We chose cooking videos because everyone has done it and understands it,” said Yiannis Aloimonos, UMD professor of computer science and director of the Computer Vision Lab, one of 16 labs and centers in UMIACS. “But cooking is complex in terms of manipulation, the steps involved and the tools you use. If you want to cut a cucumber, for example, you need to grab the knife, move it into place, make the cut and observe the results to make sure you did them properly.”

One key challenge was devising a way for the robots to parse individual steps appropriately, while gathering information from videos that varied in quality and consistency. The robots needed to be able to recognize each distinct step, assign it to a “rule” that dictates a certain behavior, and then string together these behaviors in the proper order.

UMD researcher Cornelia Fermüller (left) works with graduate student Yezhou Yang (right) on computer vision systems able to accurately identify and replicate intricate hand movements. Photo: John T. Consoli“We are trying to create a technology so that robots eventually can interact with humans,” saidCornelia Fermüller, an associate research scientist at UMIACS. “So they need to understand what humans are doing. For that, we need tools so that the robots can pick up a human’s actions and track them in real time. We are interested in understanding all of these components. How is an action performed by humans? How is it perceived by humans? What are the cognitive processes behind it?”

Aloimonos and Fermüller compare these individual actions to words in a sentence. Once a robot has learned a “vocabulary” of actions, they can then string them together in a way that achieves a given goal. In fact, this is precisely what distinguishes their work from previous efforts.

“Others have tried to copy the movements. Instead, we try to copy the goals. This is the breakthrough,” Aloimonos explained. This approach allows the robots to decide for themselves how best to combine various actions, rather than reproducing a predetermined series of actions.

The work also relies on a specialized software architecture known as deep-learning neural networks. While this approach is not new, it requires lots of processing power to work well, and it took a while for computing technology to catch up. Similar versions of neural networks are responsible for the voice recognition capabilities in smartphones and the facial recognition software used by Facebook and other websites.

While robots have been used to carry out complicated tasks for decades—think automobile assembly lines—these must be carefully programmed and calibrated by human technicians. Self-learning robots could gather the necessary information by watching others, which is the same way humans learn. Aloimonos and Fermüller envision a future in which robots tend to the mundane chores of daily life while humans are freed to pursue more stimulating tasks.

“By having flexible robots, we’re contributing to the next phase of automation. This will be the next industrial revolution,” said Aloimonos. “We will have smart manufacturing environments and completely automated warehouses. It would be great to use autonomous robots for dangerous work—to defuse bombs and clean up nuclear disasters such as the Fukushima event. We have demonstrated that it is possible for humanoid robots to do our human jobs.”

In addition to Aloimonos and Fermüller, study authors includedYezhou Yang, a UMD computer science doctoral student, and Yi Li, a former doctoral student of Aloimonos and Fermüller from NICTA.


About Ian Bach

Independent Online Terrorist Hunter I teach people how to hunt down and shut down the Bad guys web sites. I also teach about the various countries and cultures. Like most cases it is a small group of bsd eggs that in this case call themselves Muslims but in actual fact they are more like how KKK call themselves ",True Christisians". But in both cases / groups they preach a perverted and twisted view a religion. In the case of ISIS, all Qaeda, al Nusra, and the rest of the terrorists who claim to be true Muslims most of these groups follow the Wahhabi teachings. They are almost all Sunni and their goal is global domination. Yet they must be very bad at math and history. Since most Muslims prefer a separation of church and state and also mist are against Shari's Law. Esp the twisted and overly exaggerated form of Sharia Law that the Wahhabi and other bad guys use. I have studied terrorism, insurgencies, and the best tried and proven methods that work to fight terrorism. My Blogs have many links and articles that can show you who are the best and most knowledgeable people in the fields or counterinsurgency and counter terrorism. When I find great practitioner's I always listen to them to find out who they learned from and who they respect and admire. Thus I am always learning new stuff from the best and most successful in their fields of knoeledge. I strive to be an open and ethical source of information, I have met many awesome, kind, caring, and loving wonderful people many who I am close friends with now from Afghanistan, Iraq, Syria, Turkey, Iran, and many from S.E. Asia which also has a high percent of their populations that are Muslim.We must always strive to be aware people are not any one particular religion via that's what they chose to be, instead most people are a particular religion because that's what their parents and/or county is. I was raised Catholic but because I became Interested in magic ,(illusion - smoke n mirrors) and science which lead me to study many religions, and I would call myself an atheist. Yet sometimes when I lose my keys ZI find myself praying "Hail Marys" and a few "Our Fathers" which most always aides me in finding my keys. My belief is that if I just frantically look around for my krys, good luck it takes me for ever. But by saying these prayers it is like s sort of meditation and my mind becomes more calm, which is why it helps my find my keys.

Leave a Reply or Comment here. Thanks

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


This entry was posted on January 17, 2015 by in Uncategorized and tagged , , , , , , , .
%d bloggers like this: