Artificial intelligence methods developed by researchers at the University of Pennsylvania, University College London and the University of Sheffield accurately predicted the results of judicial decisions from the European Court of Human Rights 79 percent of the time.

The method is the first to forecast the outcomes of a major international court by automatically analysing case text using a machine learning algorithm. The researchers published their study in PeerJ Computer Science.

“We don’t see AI replacing judges or lawyers, but we think they’d find it useful for rapidly identifying patterns in cases that lead to certain outcomes,” said UCL’s Nikolaos Aletras, who led the study. “It could also be a valuable tool for highlighting which cases are most likely to be violations.”

It’s a new take on the field for Daniel Preotiuc-Pietro of Penn’s Positive Psychology Center in the School of Arts & Sciences. “For me as a computer scientist, it was interesting because we applied our work into a completely different domain and found something that our colleagues [in the law profession] find very interesting,” he said.

In developing the method, the team extracted case information published in the court’s publically accessible database. They then identified English language data sets for 584 cases relating to Articles 3, 6 and 8 of what’s known as the European Convention on Human Rights. Article 3 prohibits torture and inhuman and degrading treatment; Article 6 protects the right to a fair trial; and Article 8 provides a right to respect for one’s “private and family life, his home and his correspondence.”

For those three, for which the most data were available, the scientists applied an AI algorithm to the text to seek out patterns. To prevent bias and mislearning, they selected an equal number of violation and non-violation cases.

“Ideally, we’d test and refine our algorithm using the applications made to the court rather than the published judgements, but without access to that data we rely on the court-published summaries of these submissions,” explained Vasileios Lampos, also of UCL Computer Science.

Topics extracted from the circumstances mentioned in the case text turned out to be the most reliable factors for predicting the court’s decision. The “circumstances” section of the text includes information about the case’s factual background. Combining the information extracted from the abstract “topics” the cases cover and “circumstances” across data for all three articles achieved 79 percent accuracy.

“Previous studies have predicted outcomes based on the nature of the crime, or the policy position of each judge, so this is the first time judgements have been predicted using analysis of text prepared by the court,” Lampos said.

Preotiuc-Pietro said access to additional data could potentially increase that accuracy. “We were confident we could get better than chance,” he said. “But I’m optimistic this would improve even more if we had much more data.”

Funding for the research came partially from the Templeton Religion Trust.