Real or fake text? We can learn to spot the difference
Penn computer scientists prove that people can be trained to tell the difference between AI-generated and human-written text. Their new paper debuts the results of the largest-ever human study on AI detection.
The most recent generation of chatbots has surfaced longstanding concerns about the growing sophistication and accessibility of artificial intelligence.
Fears about the integrity of the job market—from the creative economy to the managerial class—have spread to the classroom as educators rethink learning in the wake of ChatGPT.
Yet while apprehensions about employment and schools dominate headlines, the truth is that the effects of large-scale language models such as ChatGPT will touch virtually every corner of our lives. These new tools raise society-wide concerns about artificial intelligence’s role in reinforcing social biases, committing fraud and identity theft, generating fake news, spreading misinformation and more.
A team of researchers at Penn’s School of Engineering and Applied Science is seeking to empower tech users to mitigate these risks. In a peer-reviewed paper, the authors demonstrate that people can learn to spot the difference between machine-generated and human-written text.
Before you choose a recipe, share an article, or provide your credit card details, it’s important to know there are steps you can take to discern the reliability of your source.
The study, led by Chris Callison-Burch, associate professor in the Department of Computer and Information Science (CIS), along with Liam Dugan and Daphne Ippolito, students in CIS, provides evidence that AI-generated text is detectable.
“We’ve shown that people can train themselves to recognize machine-generated texts,” says Callison-Burch. “People start with a certain set of assumptions about what sort of errors a machine would make, but these assumptions aren’t necessarily correct. Over time, given enough examples and explicit instruction, we can learn to pick up on the types of errors that machines are currently making.”
The study uses data collected using Real or Fake Text?, an original web-based training game.
This training game is notable for transforming the standard experimental method for detection studies into a more accurate recreation of how people use AI to generate text.
In standard methods, participants are asked to indicate in a yes-or-no fashion whether a machine has produced a given text. This task involves simply classifying a text as real or fake and responses are scored as correct or incorrect.
The Penn model significantly refines the standard detection study into an effective training task by showing examples that all begin as human-written. Each example then transitions into generated text, asking participants to mark where they believe this transition begins. Trainees identify and describe the features of the text that indicate error and receive a score.
The study results show that participants scored significantly better than random chance, providing evidence that AI-created text is, to some extent, detectable.
“Our method not only gamifies the task, making it more engaging, it also provides a more realistic context for training,” says Dugan. “Generated texts, like those produced by ChatGPT, begin with human-provided prompts.”
Griffin Pitt, right, works with two other student researchers to test the conductivity, total dissolved solids, salinity, and temperature of water below a sand dam in Kenya.
Griffin Pitt’s upbringing made her passionate about water access and pollution, and Penn has given her the opportunity to explore these issues back home in North Carolina and abroad.
Helping robots work together to explore the Moon and Mars
Penn Engineers, NASA, and five other universities tested robotic systems designed to help unmanned explorers cooperate in the dunes of White Sands, New Mexico, paving the way for Moon and Mars exploration.
From framework to actions: Provost John L. Jackson Jr. talks Penn Forward
In a Q&A, Provost John L. Jackson Jr. explains the relationship between the strategic framework In Principle and Practice and Penn Forward—a new University-wide process and action plan that will advance Penn forward for the next decade and beyond.
