Last fall, a team of undergraduates developed a high-tech solution to help the city target one of its persistent problems: the illegal dumping of construction and trash debris.
From left, Penn computer science students Abhinav Karale, Ameya Shiva and William Kayat presented their project targeting illegal dumping at the City of Philadelphia in April. They developed a tool that uses computer vision to flag dumping on surveillance video. (Image: Abhinav Karale)
Poverty and litter often go hand in hand, something Philadelphia knows all too well. The nation’s poorest big city has struggled for decades to rein in its trash problem and sweep away the old nickname “Filthadelphia.”
Last fall, an innovative collaboration with a group of computer science undergraduates helped introduce the city to a high-tech solution for one of its persistent problems: the illegal dumping of construction debris and trash.
It began with political science professor Daniel Hopkins, who has been working with the city and other academic partners in the region to leverage the resources within Penn and other universities to better serve the city. Part of that work was identifying policy areas where researchers could help, one of them being the city’s Zero Waste and Litter Cabinet. Hopkins has served on the cabinet for several years, helping brainstorm ways the city could reduce trash.
“In those conversations, we realized that the city wanted stepped up enforcement targeting illegal dumping,” he says. “I knew that the city was interested in automated analyses of its video cameras. The Philadelphia police are way too busy to just watch video feeds to look for acts of littering. But if we could write an algorithm or develop a computational approach that could say ‘hey, you don’t have to watch the entire feed, just watch this minute.’ We thought that could be really promising and cost saving.”
That’s where the team of then seniors, Abhinav Karale, Ameya Shiva, and William Kayat came in. They had been on the lookout for something to do their senior design project on when they spotted Hopkins’ video camera project on one of their boards.
“The project was rare in that we worked with real stakeholders, we had the ability to engage the city and use our computer science skills to make a difference for them,” says Karale.
The idea was to solve the problem of slogging through hours of surveillance footage to find instances of dumping, and to do it using artificial intelligence, computer vision, and machine learning.
Almost a full year before the team took on the project, Hopkins and colleagues in SAS computing had been working to lay the legal groundwork and set up a secure environment in which to house and analyze the videos.
“One of the challenges that I face as a social scientist is in making sure that we are both teaching the technical skills but also having the ethical conversations,” he says.
Once the legal, ethical and security issues had been addressed, the city gave the team a handful of 24-hour video camera footage to use, and they worked from September 2018 through April on developing the tool.
The students essentially built a computer brain that could learn what certain items were in the video feed and taught it to search for and flag those items, like black trash bags, wooden beams from a house demolition, and tires. The algorithm could very quickly look through the footage to find those things.
It also became a framework that would make the analysis of the video straightforward, so that future teams could build on it. Building in a human aspect, allowing users to correct the tool if it made a mistake, was a way to help the model keep learning and get better and better, Karale says.
For the city, the tool was a solution for something they’d been struggling to address.
One of the challenges that I face as a social scientist is in making sure that we are both teaching the technical skills but also having the ethical conversations.
Daniel Hopkins, political science professor
“Even though it is a huge quality of life issue that people hate around the city, as a crime dumping doesn’t rank high, compared with shootings and burglaries,” says Nic Esposito, director of the city’s Zero Waste and Litter Cabinet. “We needed to make it as streamlined as possible. We have a lot of footage to look through and if you can pinpoint in a second rather than looking through an hour, it’s a real benefit.”
He said many at the city were “enthralled” by what the Penn team’s tool accomplished, and said they hope to be able to incorporate the tool into official use at some point.
“I know there are some people in the city who are excited to get their hands on this technology,” Esposito says.
For Karale and his teammates, the project was a chance to take everything they had learned in their courses and put it all to use in the real world.
“We got incredibly positive feedback, and after the presentation someone commented that it seemed like magic, which as a computer scientist, that is really something you want to hear,” Karale says. “It was awesome, and one of the coolest experiences I had in college.”
Karale, Shiva, and Kayat all graduated from Penn last May. Karale earned an undergraduate degree in finance from the Wharton School and in computer science from the School of Engineering and Applied Science (SEAS). He is now working towards a master’s degree in data science in Penn Engineering. Shiva earned an undergraduate degree in in engineering from SEAS and in economics from Wharton. Kayat earned his undergraduate degree in computer science and math in SEAS.
Penn physicists led by Bo Zhen have created hybrid light-matter particles that interact strongly enough to compute, pointing toward ultrafast, low-energy optical AI hardware.
Penn’s newest supercomputer is transforming research
Penn’s first campus-wide HPC and AI cluster, “Betty,” is expanding access to powerful computing, enabling groundbreaking projects, and fostering new collaborations across disciplines.
A bioengineered bean gum from the lab of Penn Dental’s Henry Daniell is found to reduce the levels of three microbes associated with head and neck squamous cell cancer to almost zero, without affecting the beneficial bacteria normally found in the mouth.
Fighting oral cancer with bioengineered chewing gum
Research led by Penn Dental’s Henry Daniell shows that antiviral and antibacterial chewing gums reduce the levels of three microbes linked to worse outcomes in oral cancers, paving the way for more effective and affordable therapies.
The performing arts at Penn: Process, practice, and purpose
In the vivid tapestry of performing arts groups at Penn, students prepare for their performances while simultaneously enriching their college experience.
