Explained: Neural Networks
April 17, 2017 | MITEstimated reading time: 7 minutes
By the 1980s, however, researchers had developed algorithms for modifying neural nets’ weights and thresholds that were efficient enough for networks with more than one layer, removing many of the limitations identified by Minsky and Papert. The field enjoyed a renaissance.
But intellectually, there’s something unsatisfying about neural nets. Enough training may revise a network’s settings to the point that it can usefully classify data, but what do those settings mean? What image features is an object recognizer looking at, and how does it piece them together into the distinctive visual signatures of cars, houses, and coffee cups? Looking at the weights of individual connections won’t answer that question.
In recent years, computer scientists have begun to come up with ingenious methods for deducing the analytic strategies adopted by neural nets. But in the 1980s, the networks’ strategies were indecipherable. So around the turn of the century, neural networks were supplanted by support vector machines, an alternative approach to machine learning that’s based on some very clean and elegant mathematics.
The recent resurgence in neural networks — the deep-learning revolution — comes courtesy of the computer-game industry. The complex imagery and rapid pace of today’s video games require hardware that can keep up, and the result has been the graphics processing unit (GPU), which packs thousands of relatively simple processing cores on a single chip. It didn’t take long for researchers to realize that the architecture of a GPU is remarkably like that of a neural net.
Modern GPUs enabled the one-layer networks of the 1960s and the two- to three-layer networks of the 1980s to blossom into the 10-, 15-, even 50-layer networks of today. That’s what the “deep” in “deep learning” refers to — the depth of the network’s layers. And currently, deep learning is responsible for the best-performing systems in almost every area of artificial-intelligence research.
Under the hood
The networks’ opacity is still unsettling to theorists, but there’s headway on that front, too. In addition to directing the Center for Brains, Minds, and Machines (CBMM), Poggio leads the center’s research program in Theoretical Frameworks for Intelligence. Recently, Poggio and his CBMM colleagues have released a three-part theoretical study of neural networks.
The first part, which was published last month in the International Journal of Automation and Computing, addresses the range of computations that deep-learning networks can execute and when deep networks offer advantages over shallower ones. Parts two and three, which have been released as CBMM technical reports, address the problems of global optimization, or guaranteeing that a network has found the settings that best accord with its training data, and overfitting, or cases in which the network becomes so attuned to the specifics of its training data that it fails to generalize to other instances of the same categories.
There are still plenty of theoretical questions to be answered, but CBMM researchers’ work could help ensure that neural networks finally break the generational cycle that has brought them in and out of favor for seven decades.
Page 2 of 2Suggested Items
Gartner Survey Finds 61% of Organizations Are Evolving Their D&A Operating Model Because of AI Technologies
05/01/2024 | Gartner, Inc.Sixty-one percent of organizations are forced to evolve or rethink their data and analytics (D&A) operating model because of the impact of disruptive artificial intelligence (AI) technologies, according to a new Gartner, Inc. survey.
Real Time with… IPC APEX EXPO 2024: Operational Excellence and Smart Factory Initiatives
04/30/2024 | Real Time with...IPC APEX EXPOOperational excellence and operational efficiency are defined in this interview with Koh Young General Manager Joel Scutchfield. He touches on automation, AI, and collaboration as solutions to resource limitations. Koh Young's data-driven approach uses AI for process adjustments, data analytics, and supply chain enhancements. The discussion underscores the shift toward smart factory initiatives and the future of manufacturing, with a focus on reshoring, nearshoring, and technology utilization.
IDTechEx Report on Quantum Technology: Nano-scale Physics for Massive Market Impact
04/30/2024 | PRNewswireThe quantum technology market leverages nano-scale physics to create revolutionary new devices for computing, sensing, and communications. Across the industry, quantum technology offers a paradigm shift in performance compared with incumbent solutions.
NASA’s Optical Comms Demo Transmits Data Over 140 Million Miles
04/30/2024 | NASA JPLNASA’s Deep Space Optical Communications experiment also interfaced with the Psyche spacecraft’s communication system for the first time, transmitting engineering data to Earth.
Real Time with… IPC APEX EXPO 2024: Insight into Summit Interconnect's Success
04/30/2024 | Real Time with...IPC APEX EXPOShane Whiteside, CEO of Summit Interconnect, discusses the company's recent recognition as one of the best PCB fabricators in the industry by receiving IPC's Peter Sarmanian award. Whiteside touches on the impact of changes in the marketplace, such as the Defense Production Act and presidential determination, on their growth. Whiteside also shares the company's focus on mechanical and data automation to enhance manufacturing processes and anticipates more automation and evolution in the industry.