Penn researchers created a fleet of robots to navigate unknown underground environments as part of the Defense Advanced Research Projects Agency’s Subterranean Challenge.
A legged “dog” robot, made by Penn spinoff Ghost Robotics, navigates through the Number 9 mine in Lansford, PA. The robots were designed to detect artifacts, like backpacks and dummy personnel, to mimic a real-life search-and-rescue mission underground.
Earlier this semester, a group of Penn students, postdocs, and faculty travelled to an experimental mine near Pittsburgh to participate in the first round of the Subterranean (SubT) Challenge hosted by the Defense Advanced Research Projects Agency (DARPA). Led by Camillo J. Taylor, researchers partnered with Penn spinoff companies to build a team of robots to navigate and explore unknown underground environments.
DARPA challenges are meant to inspire innovative solutions to complex problems, and the goal of SubT was to develop robots that could be sent into places too dangerous for humans, like a subway tunnel after a fire or a mine where workers are trapped. These robots could then send back reports on what it saw to a person who could remain in a safe location.
Members of the Pennsylvania Laboratory for Underground Tunnel Operations team, PLUTO for short, had experience working with drones in challenging settings, but going underground meant that another type of platform was needed to carry heavy equipment while navigating narrow spaces. By combining Ghost Robotics legged “dog” platforms with Exyn technologies drones, PLUTO united prior areas of expertise with the benefits of a legged system.
To traverse a mine successfully, robots need to be able to see what’s around them so they can avoid obstacles while also sharing what they see with human operators. Each dog carried a light detection and ranging device that uses laser light to create a 3D map, stereo RGB cameras to see artifacts, thermal cameras to detect heat signatures, and an onboard computer to process the data.
PLUTO’s robots also need the ability to decide where to travel next, recognize artifacts, and relay information to other dogs and to the humans outside the mine. Programs developed by Ph.D. student Anthony Cowley were used to generate a map of the robot’s location based on images collected by the sensors, while artifacts like backpacks and phones were detected using a program developed by Ph.D. student Shreyas Shivakumar.
The team’s prep space at the SubT challenge. (Image: Shrevas Shivakumar)
Communication is particularly challenging underground since radio waves can’t travel through thick cave walls. PLUTO’s strategy, led by master’s student Fernando Cladera, was to create a “bucket brigade” system that allowed robots to share data with one another. That way, if a robot couldn’t get back to the entrance, the data it collected could still be relayed by other dogs to the base station.
Bringing all of these abilities together requires high-level autonomy to allow the robots to plan their exploration strategies without direct human input. Ph.D. student Ian Miller led this effort and helped make sure that all the sensors, hardware, and algorithms worked together.
PLUTO spent time at the Number 9 Coal Mine and Museum in Lansford, Pennsylvania, over the summer and at the Colorado School of Mines experimental mine earlier this year to see how their automated systems performed underground. Many months of preparation led up to the first round of the SubT challenge in August against ten other teams a test mine near Pittsburgh.
Each team completed two mine courses and had two attempts through each course to find artifacts, ranging from backpacks, cell phones, fire extinguishers, and dummy personnel, with a one-hour time limit. None of the team members was allowed to travel inside the mine, and only Miller was allowed to interact with the robots while they were collecting data.
While their robots did not spot as many artifacts as hoped, the PLUTO team is pleased with how the system performed in such challenging and unknown environments. Components that worked well include how the dogs detected and explored tunnels, their ability to recognize objects, and sharing data through the bucket brigade. Several detections were provided by “fallen” dogs, showing how their system was resilient even when individual robots took a stumble.
Members of the PLUTO team, including Penn students, postdocs, and faculty, tested their platforms over the summer at the Number 9 Coal Mine and Museum in Lansford to make sure their robots would be able to perform in complex underground environments.
Adarsh Kulkarni, a master’s student who also works at Ghost Robotics, says that he’s happy with the dogs’ mechanical stability and how well they performed even after multiple falls. “This was by far the hardest we’ve run the robots and the worst environments they’ve been in,” says Kulkarni. “The mine is its own beast.”
“They took a beating every single day, and they were working the next morning,” Shivakumar adds. “That’s really commendable.”
Although some of their design had been overfit to challenges specific to Number 9, including sensors designed for narrow, textured walls while SubT had wider, smooth walls, the experience showed the team first-hand how difficult it is to engineer robots destined for unknown environments and was a rare opportunity to test robots in a new setting. “It’s very different from the normal academic workflow,” says Miller. “Going from an algorithm to something that works in a place that you’ve never seen before are very different problems.”
PLUTO’s fleet of dogs line up outside of Pennovation Center, ready to head to Pittsburgh. (Image: Shreyas Shivakumar)
This “last 10%” in robotics, making an automated system that is robust and reliable, is a challenge that is often solved through a combination of state-of-the-art technology and pragmatic perspectives. “Sometimes these systems are not novel per se, but what is novel is their implementation in an untested environment,” says Cladera. “The novelty is how to solve all these problems so that the robots are reliable in harsh environments.”
The team is still discussing their plans for future rounds of SubT, with the next round happening in February in an urban setting, meaning more human-made structures and shapes like sharp angles, smooth walls, and staircases. Regardless of what lies ahead, creating and sending robots into challenging real-world environments is fundamental for progress both at Penn and in the field of robotics as a whole, especially for a future where automated systems could be tasked with a wide range of challenging tasks from driving cars to searching for survivors.
Researchers at Penn’s General Robotics, Automation, Sensing, and Perception Lab are well-placed to address these challenges, thanks in part to a culture that encourages collaboration and communication. “It’s something that we instill in everybody that walks into our facility,” says Taylor. “If you are not worried about having a project that fits only within your area of expertise, if you are willing to be broad about how you think, it enables you to do bigger things.”
The PLUTO team was supported by funding from the U.S. Defense Advanced Research Projects Agency.
The Number 9 Coal Mine is a non-profit organization operated by the Panther Creek Valley Foundation. Zachary Petroski serves as the foundation’s president and director of the mine.
Ghost Robotics was co-founded by Penn alumni Avik De and Gavin Kenneally.
In Senegal, the ambitious Dakar Greenbelt project seeks to create an extensive network of ecological infrastructure in and around the city to sustainably address environmental concerns and enhance urban life. With support from David Gouverneur and Ellen Neises, Ph.D. candidate Rob Levinthal in the Weitzman School of Design led two courses that included a field trip to Dakar, that culminated in students presenting their visions for parts of the Greenbelt.
From a desert to an oasis: Penn engages in ambitious greening effort in the Sahel
Students from the Weitzman School of Design journeyed to Senegal to help with a massive ecological and infrastructural greening effort as part of their coursework. The Dakar Greenbelt aims to combat desertification and promote sustainable urban growth.
As part of an undergraduate course, Penn faculty and students curated an Arthur Ross Gallery exhibition of works from the Neumann family’s extensive collection of modern and contemporary art.
The University’s nexus for technology transfer supports researchers in their innovative efforts, from CAR T to mRNA advancements that have dramatically reshaped the world.
