Robots Learn by Watching How-to Videos
January 4, 2016 | Cornell UniversityEstimated reading time: 2 minutes
When you hire new workers you might sit them down to watch an instructional video on how to do the job. What happens when you buy a new robot?
Cornell researchers are teaching robots to watch instructional videos and derive a series of step-by-step instructions to perform a task. You won’t even have to turn on the DVD player; the robot can look up what it needs on YouTube. The work is aimed at a future when we may have “personal robots” to perform everyday housework – cooking, washing dishes, doing the laundry, feeding the cat – as well as to assist the elderly and people with disabilities.
The researchers call their project ”RoboWatch.” Part of what makes it possible is that there is a common underlying structure to most how-to videos. And, there’s plenty of source material available. YouTube offers 180,000 videos on “How to make an omelet” and 281,000 on “How to tie a bowtie.” By scanning multiple videos on the same task, a computer can find what they all have in common and reduce that to simple step-by-step instructions in natural language.
Why do people post all these videos? “Maybe to help people or maybe just to show off,” said graduate student Ozan Sener, lead author of a paper on the video parsing method presented Dec. 16 at the International Conference on Computer Vision in Santiago, Chile. Sener collaborated with colleagues at Stanford University, where he is currently a visiting researcher.
A key feature of their system, Sener pointed out, is that it is “unsupervised.” In most previous work, robot learning is accomplished by having a human explain what the robot is observing – for example, teaching a robot to recognize objects by showing it pictures of the objects while a human labels them by name. Here, a robot with a job to do can look up the instructions and figure them out for itself.
Page 1 of 2
Suggested Items
Gartner Forecasts Worldwide IT Spending to Grow 8% in 2024
04/17/2024 | Gartner, Inc.Worldwide IT spending is expected to total $5.06 trillion in 2024, an increase of 8% from 2023, according to the latest forecast by Gartner, Inc. This is an increase from the previous quarter’s forecast of 6.8% growth and puts worldwide IT spending on track to surpass $8 trillion well before the end of the decade.
Argonne, RIKEN Sign a Memorandum of Understanding in Support of AI for Science
04/16/2024 | BUSINESS WIRELeaders in high performance computing in the U.S. and Japan have signed a memorandum of understanding (MOU) establishing a cooperative relationship in support of artificial intelligence (AI) computing projects.
RAICo Fellowship to Widen Participation Within the Field of Robotics and Artificial Intelligence
04/12/2024 | RAICoWOMEN and those early in their academia career who are passionate about robotics and artificial intelligence are encouraged to apply for a RAICo Fellowship.
Indium to Host Free Webinar on Lead-Free Solder Paste for Automotive Applications
04/09/2024 | Indium CorporationIndium Corporation’s Senior Research Metallurgist, Jie Geng, Ph.D., will conduct a webinar on low-voiding, high-reliability, lead-free solder paste for automotive applications.
SMTA Announces Program for High Reliability: Strategic Technology Advancement Research Forum
04/08/2024 | SMTAThe SMTA announced the finalized program for the High Reliability: Strategic Technology Advancement Research Forum which takes place on May 2, 2024 in Olathe, Kansas, USA. The event, now in its second year, addresses challenges for electronics manufacturers in the high reliability sector.