In a new paper, researchers at Penn Engineering highlight the need to develop more thorough frameworks for ensuring that AI-enabled robots embody a ‘do no harm’ core principle.
In a collaborative paper in Science Robotics, the researchers highlight the need to develop more thorough frameworks for ensuring that AI-enabled robots embody a core principle famously articulated by science fiction author Isaac Asimov: “A robot may not injure a human being.”
“There has been substantial progress in alignment research when it comes to AI-enabled chatbots,” says George J. Pappas, UPS Foundation Professor of Transportation in Electrical and Systems Engineering at Penn Engineering and the paper’s senior author. “But the same cannot be said for robotics.”
Indeed, according to research conducted by Pappas and others and cited in the new paper, the vulnerabilities of chatbots to “jailbreaking” attacks pose extreme danger to humans when those AI systems are allowed to control robots. In one instance, the researchers note, framing instructions as movie dialogue persuaded a chatbot to deliver an explosive device, despite its manufacturer’s efforts to provide guardrails that limited the robot’s behavior.
In recent years, AI researchers have focused on the need to “align” AI systems with human values. But, as the authors of the new paper point out, those efforts have almost entirely focused on chatbots—disembodied systems that cannot interact with the physical world.
“Most of today’s AI breakthroughs live in a digital sandbox—language and images, with guardrails designed for pixels, not physics,” says Vijay Kumar, Professor in Mechanical Engineering and Applied Mechanics, Nemirovsky Family Dean of Penn Engineering, and a co-author of the paper. “But when those same foundation models step into the real world through robots, the consequences are no longer virtual. The guardrails that work online are simply not sufficient when actions are associated with inertia, momentum and irreversible effects.”
When the Schuylkill swallowed the city: Lessons from Hurricane Ida’s historic flood
New Penn research shows that Hurricane Ida wasn’t a once-in-a-century anomaly but a preview of how climate change, urbanization, and aging infrastructure are rewriting flood risk.
The Fed explained: What it does and why it matters
Former Philadelphia Fed President Patrick Harker and financial historian Peter Conti-Brown, both Wharton professors, unpack the central bank’s origins, its unusual structure, and the quiet ways it shapes the economy
How population changes are impacting primary education worldwide
Research from Penn sociologist Emily Hannum and colleagues reveals regional trends in whether school-age populations are increasing, plateauing, or decreasing—and shows how different countries are responding.
A bioengineered bean gum from the lab of Penn Dental’s Henry Daniell is found to reduce the levels of three microbes associated with head and neck squamous cell cancer to almost zero, without affecting the beneficial bacteria normally found in the mouth.
Fighting oral cancer with bioengineered chewing gum
Research led by Penn Dental’s Henry Daniell shows that antiviral and antibacterial chewing gums reduce the levels of three microbes linked to worse outcomes in oral cancers, paving the way for more effective and affordable therapies.
