Toward Machines that Improve with Experience

Reading time ( words)

Self-driving taxis. Cell phones that react appropriately to spoken requests. Computers that outcompete world-class chess and Go players. Artificial Intelligence (AI) is becoming part and parcel of the technological landscape—not only in the civilian and commercial worlds but also within the Defense Department, where AI is finding application in such arenas as cybersecurity and dynamic logistics planning. 

But even the smartest of the current crop of AI systems can’t stack up against adaptive biological intelligence. These high-profile examples of AI all rely on clever programming and extensive training datasets—a framework referred to as Machine Learning (ML)—to accomplish seemingly intelligent tasks. Unless their programming or training sets have specifically accounted for a particular element, situation, or circumstance, these ML systems are stymied, unable to determine what to do. 

That’s a far cry from what even simple biological systems can do as they adapt to and learn from experience. And it’s light years short of how, say, human motorists build on experience as they encounter the dynamic vagaries of real-world driving—becoming ever more adept at handling never-before-encountered challenges on the road. 

This is where DARPA’s new Lifelong Learning Machines (L2M) program comes in. 

The technical goal of L2M is to develop next-generation ML technologies that can learn from new situations and apply that learning to become better and more reliable, while remaining constrained within a predetermined set of limits that the system cannot override. Such a capability for automatic and ongoing learning could, for example, help driverless vehicles become safer as they apply knowledge gained from previous experiences—including the accidents, blind spots, and vulnerabilities they encounter on roadways—to circumstances they weren’t specifically programmed or trained for. 

“Life is by definition unpredictable. It is impossible for programmers to anticipate every problematic or surprising situation that might arise, which means existing ML systems remain susceptible to failures as they encounter the irregularities and unpredictability of real-world circumstances,” said L2M program manager Hava Siegelmann. “Today, if you want to extend an ML system’s ability to perform in a new kind of situation, you have to take the system out of service and retrain it with additional data sets relevant to that new situation. This approach is just not scalable.” 

To get there, the L2M program aims to develop fundamentally new ML mechanisms that will enable systems to learn from experience on the fly—much the way children and other biological systems do, using life as a training set. The basic understanding of how to develop a machine that could truly improve from experience by gaining generalizable lessons from specific situations is still immature. The L2M program will provide a unique opportunity to build a community of computer scientists and biologists to explore these new mechanisms. 

“Enabling a computer to learn even the simplest things from experience has been a longstanding but elusive goal,” said Siegelmann. “That’s because today’s computers are designed to run on prewritten programs incapable of adapting as they execute, a model that hasn’t changed since the British polymath Alan Turing developed the earliest computing machines in the 1930s. L2M calls for a new computing paradigm.” 

The four-year L2M program features two technical areas. The first aims to develop ML frameworks that can continuously apply the results of past experience and adapt “lessons learned” to new data or situations. Simultaneously, it calls for the development of techniques for monitoring an ML system’s behavior, setting limits on the scope of its ability to adapt, and intervening in the system’s functions as needed. The research will encompass network theory, algorithms, software, and computer architectures. The second technical area, which derives from Siegelmann’s longstanding interest in biological learning mechanisms, will focus specifically on how living systems learn and adapt and will consider whether and how those principles and techniques can be applied to ML systems. “Life has had billions of years to develop approaches for learning from experience,” Siegelmann said. “There are almost certainly some secrets there that can be applied to machines so they can be not just computational tools to help us solve problems but responsive and adaptive collaborators.”


Suggested Items

OFFSET 'Sprinters' Encouraged to Share Innovative Ideas for Swarm Systems

10/16/2017 | DARPA
DARPA’s OFFensive Swarm-Enabled Tactics (OFFSET) program envisions future small-unit infantry forces using small unmanned aircraft systems (UASs) and/or small unmanned ground systems (UGSs) in swarms of 250 robots or more to accomplish diverse missions in complex urban environments.

DARPA’s Drive to Keep the Microelectronics Revolution at Full Speed Builds Its Own Momentum

08/28/2017 | DARPA
To perpetuate the pace of innovation and progress in microelectronics technology over the past half-century, it will take an enormous village rife with innovators. This week, about 100 of those innovators throughout the broader technology ecosystem, including participants from the military, commercial, and academic sectors, gathered at DARPA headquarters at the kickoff meeting for the Agency’s new CHIPS program, known in long form as the Common Heterogeneous Integration and Intellectual Property (IP) Reuse Strategies program.

DARPA's Assured Autonomy Program Seeks to Guarantee Safety of Learning-enabled Autonomous Systems

08/17/2017 | DARPA
Building on recent breakthroughs in autonomous cyber systems and formal methods, DARPA today announced a new research program called Assured Autonomy that aims to advance the ways computing systems can learn and evolve to better manage variations in the environment and enhance the predictability of autonomous systems like driverless vehicles and unmanned aerial vehicles (UAVs).

Copyright © 2018 I-Connect007. All rights reserved.