How AI tools can help assess verbal eyewitness statements

In a new paper in Psychological Science, researchers from the Quattrone Center for the Fair Administration of Justice at the University of Pennsylvania Carey Law School demonstrate how artificial intelligence (AI) can improve the accuracy of the criminal adjudication process.

View large image

The paper, “Assessing Verbal Eyewitness Confidence Statements Using Natural Language Processing,” outlines how the researchers trained a large language model (LLM) to parse eyewitness confidence statements in an effort to help reduce misidentifications, one of the largest known contributors to wrongful convictions.

“Of the over 3,000 exonerations recorded to date by the National Registry of Exonerations, more than 900 are due to eyewitness misidentifications,” says Paul Heaton, academic director of the Quattrone Center, professor of law, and co-author of the paper. “To help address this problem, we developed a new tool to enable attorneys and police investigators to better distinguish accurate identifications from faulty ones.”

After an eyewitness selects a suspect from a lineup, best practices instruct police officers to ask how confident they are in their identification. Prior research demonstrates that witnesses who are highly confident at the time of the identification from an unbiased lineup tend to be correct much more frequently than those who express uncertainty. But in the field, police typically make a subjective judgement as to whether any particular witness was confident, which can introduce error into the process. The AI developed by the researchers offers a neutral method for assessing such statements free of contextual bias, leveraging information from thousands of prior eyewitness decisions.

Read more at Penn Carey Law.