When a 2013 study published in Science concluded that reading literary fiction for as few as 20 minutes could improve someone’s social abilities, it made quite the splash. However, when researchers from the University of Pennsylvania, Pace University, Boston College and the University of Oklahoma tried to replicate the findings using the original study materials and methodology, the results didn’t hold up.

“Reading a short piece of literary fiction does not seem to boost theory of mind,” said Deena Weisberg, a senior fellow in Penn’s psychology department in the School of Arts & Sciences, referring to the notion that describes a person’s ability to understand the mental states of others. “Literary fiction did not do any better than popular fiction, expository non-fiction and not any better than reading nothing at all.”

The research team published its results in a new paper in the Journal of Personality and Social Psychology.

Initially, Weisberg and Pace’s Thalia Goldstein wanted to repeat the original study, conducted at the New School for Social Research, to better understand how such a minimal intervention and a specific storytelling type alone could result in this response.  

“Why would literary fiction be particularly good at doing this? Why not romance literature, which is primarily about relationships? Or why not something more absorbing?” Weisberg said. “Literature is harder to absorb. Those questions made me raise my eyebrows.”

The pair followed the published study to the letter. They used the stories and materials from the original work, applying the same measures and design, including a theory-of-mind measure called the Reading the Mind in the Eyes Test, or RMET, in the hopes of drawing the same conclusion. They worked closely with New School researchers to ensure accuracy. Results in hand, they began speaking with other institutions, learning that BC and Oklahoma scientists had attempted — and failed — to replicate these results as well. They collaborated to put together the paper.

This particular outcome not only shines a light on problems with the conclusions drawn in one study but also reinforces a broader issue with which the field has been grappling.

“Psychology has been doing a lot of soul-searching lately,” Weisberg said. “There’s been a lot of attention to high-profile studies that show something of social importance. It would be amazing if we could put into place interventions on the basis of this study, but we really need to double check and not just rely on one lab, one study, before we go shouting from the rooftops.”

Weisberg doesn’t discount the idea that exposure to fiction could positively affect a person’s social cognition. In fact, she and her collaborators additionally administered the Author Recognition Test, which measures lifetime exposure to all genres of fiction: From a list of 130 names — some real authors, some foils — participants were asked to select all real writers they knew with certainty. They were penalized for guessing and for incorrect answers. The researchers then tested for relations between this measure and social cognition, once again using the RMET, which offers an image of eyes and asks participants to choose the best description of the emotion the eyes convey.

In this case, they noted a strong relationship: The more authors participants knew, the better they scored on the social cognition measure.

“One brief exposure to fiction won’t have an effect, but perhaps a protracted engagement with fictional stories such that you boost your skills, perhaps that could,” Weisberg said. “It’s also possible the causality is the other way around: It could be people who are already good at theory of mind read a lot. They like engaging in stories with people.”

The next phase of research entails looking in more detail at other variables. Does literary fiction improve social abilities for some people but not others? Perhaps other kinds of fiction work? What personality traits make someone more likely to feel the effect?

Part of the original study’s appeal came from its defense of reading literature. It’s possible such a link will one day be demonstrated, but, for now, writers will continue to stand on their own merits, and psychology will continue answering questions about what works best to engage our social-cognitive abilities.