The hidden geometry of learning: Neural networks think alike
New research by Penn engineers illuminates the inner workings of neural networks, opening the possibility of developing hyper-efficient algorithms that could classify images in a fraction of the time.
Penn Engineers have uncovered an unexpected pattern in how neural networks—the systems leading today’s AI revolution—learn, suggesting an answer to one of the most important unanswered questions in AI: why these methods work so well.
Inspired by biological neurons, neural networks are computer programs that take in data and train themselves by repeatedly making small modifications to the weights or parameters that govern their output, much like neurons adjusting their connections to one another. The final result is a model that allows the network to predict on data it has not seen before. Neural networks are being used today in essentially all fields of science and engineering, from medicine to cosmology, identifying potentially diseased cells and discovering new galaxies.
“Suppose the task is to identify pictures of cats and dogs,” says Chaudhari. “You might use the whiskers to classify them, while another person might use the shape of the ears—you would presume that different networks would use the pixels in the images in different ways, and some networks certainly achieve better results than others, but there is a very strong commonality in how they all learn. This is what makes the result so surprising.”
The result not only illuminates the inner workings of neural networks, but gestures toward the possibility of developing hyper-efficient algorithms that could classify images in a fraction of the time, at a fraction of the cost. Indeed, one of the highest costs associated with AI is the immense computational power required to develop neural networks. “These results suggest that there may exist new ways to train them,” says Chaudhari.
In Senegal, the ambitious Dakar Greenbelt project seeks to create an extensive network of ecological infrastructure in and around the city to sustainably address environmental concerns and enhance urban life. With support from David Gouverneur and Ellen Neises, Ph.D. candidate Rob Levinthal in the Weitzman School of Design led two courses that included a field trip to Dakar, that culminated in students presenting their visions for parts of the Greenbelt.
From a desert to an oasis: Penn engages in ambitious greening effort in the Sahel
Students from the Weitzman School of Design journeyed to Senegal to help with a massive ecological and infrastructural greening effort as part of their coursework. The Dakar Greenbelt aims to combat desertification and promote sustainable urban growth.
As part of an undergraduate course, Penn faculty and students curated an Arthur Ross Gallery exhibition of works from the Neumann family’s extensive collection of modern and contemporary art.
The University’s nexus for technology transfer supports researchers in their innovative efforts, from CAR T to mRNA advancements that have dramatically reshaped the world.
