Let’s talk a bit about the research that went into the paper.
I started the process by trying to figure out how to track the relevant media coverage. I thought The Economist article may have been the only instance of problematic reporting out there. Alternatively, there may be thousands. I started trying to find a mechanism for locating the most covered science and determining how it is reported and isolating the underlying narrative of those stories. I used newspapers to control the medium. Science is far more likely to be covered in print than in broadcast. My research team and I also looked at the reporting of major retractions. We systematically looked at those things that were suggesting something was wrong with science, including articles about retraction to determine those structures.
You ended up pinpointing three storylines, which you focused on throughout the paper: quest discovery, counterfeit quest discovery, and systemic problem structure.
For narrative one, we have a hero or heroine structure in pursuit of a quest that’s honorable. Society is the beneficiary. In the second narrative, somebody is pretending to be the hero, but in fact has perverted the science and was engaged in a false quest. The knowledge is unreliable and problematic because it violates basic science norms. The third is ‘the system is broken’ narrative. That’s a completely different structure that uses ‘broken’ or ‘in crisis’ rhetoric. The question the essay then asks is, ‘Is it justified to say that about science?’ and should we worry if that’s becoming a more prevalent narrative. If it’s justified, we’d have cause for concern. Occasional fraud or misuse of statistics should elicit narrative two, not narrative three.
You bring forth two areas of science that, in the past, had failures to replicate.
If a large-scale study fails to replicate, that means something is wrong, but not necessarily that science as a whole is broken. The first place where we’ve seen this sort of failure to replicate is psychology, and the second is oncology. I argue in both cases that there were practices in need of correction, but self-correction is in process now. In the case of oncology, the science, even during that period, was still producing reliable results. In fact, some of the breakthrough science in oncology has occurred exactly in this period when those headlines were saying ‘oncology in crisis.’ You can’t say the whole area is broken when a successful new paradigm of cancer science is actually coming on the scene.
It seems like a lot of journalists, as well as scientists, can benefit from your suggestions in the article.
Scientists who are engaged in self-correction by, for example, insisting that studies are adequately powered, need to tell us what the self-correction is. When there is a retraction, science journalists should point out that the science community caught the problem and report what they are doing to prevent its recurrence. And journalists who are covering the science of discovery shouldn’t ignore the false starts and the dead ends. Scientists deliver discovery by going through that process.
What are your recommendations for those communicating science?
Allow some space in the article to say, ‘These are the steps that it took science to get here.’ Science continues to build on itself. An article about discovery should also note the unanswered question. When one finding is replaced by a better one, the public needs to understand that a finding is a step on a path that over time leads to more reliable knowledge, and includes past contributions on which the new knowledge is built, false starts and dead ends. We rarely hit the point at which we say, ‘We are done, we know everything.’ So the recommendation for those writing the first narrative is, include the trial and error process of discovery. When writing the second narrative, the false quest narrative, the recommendation is, feature the heroes who caught the error. There are heroes in this narrative. There aren’t just false heroes. The heroes persisted in covering the flaws. Then ask what is being done to address them. The job of the journalist is not to uncritically transmit whatever the scientist says. Journalists aren’t publicists. Journalists are supposed to hold science accountable. But journalists also have the obligation to make sure that they are accurately creating a narrative about the process.
When using the last narrative, my recommendation is, don’t uncritically use the words ‘crisis’ and ‘broken.’ It takes strong evidence to justify those conclusions. Aggressively cover individual instances of wrongdoing and studies that reveal problematic patterns. Ask the scientific community, ‘What are you doing about it?’ Go back in six months, eight months, a year. Until you are sure they can honestly say they’ve implemented a change, you’re not done as a journalist.
We see the first storyline a lot more than second storyline. And it’s not that it’s incorrect, you say it’s just incomplete?
Yes, what they have is accurate, it’s just incomplete.
Why is that happening?
Humans have a basic desire to create coherent, simple narratives. Journalists aren’t getting the big points wrong. But downstream public understanding will be more accurate if journalists adopt these recommendations.
Why is it so important to have this complete narrative?
Public understanding of science and the way in which knowledge is generated by it contribute to public confidence in science. The journalistic narrative shouldn’t engender distrust. When the narrative is not only right but complete, the public understanding of the scientific enterprise will be less vulnerable to gratuitous attacks on science. When science delivers a vaccine against Zika, potentially preventing Zika transmission to pregnant women, we want the public to say, ‘I can trust the science,’ not ‘Science gets things wrong all the time. Scientists invariably have to retract.’
How does this research fit in with the APPC and its goals?
We have three areas of focus: adolescent health and risk; institutions of democracy, which is our civics work; and the science of science communication. One of our missions with the science of science communication area is helping journalists cover science in ways that ensure that science honors its norms. Another is increasing the likelihood that when journalists cover science, they increase public understanding of how science actually works, not how science works in some mythical world. When something goes wrong, when there is a failure, when there’s fraud, that it is understandable because humans are involved, but it is caught because the enterprise is fundamentally self-correcting.
Do you think journalists are being lazy when they don’t include the full narrative? Are they attempting to get clicks when they use ‘crisis’ rhetoric?
They may have too little time or space to be comprehensive or some may be naive about science. Most journalists aren’t knowledgeable about all of the areas of science they cover. Imagine covering a story about astrophysics one day and genetics the next. It’s just not plausible that a journalist has all that capacity. And a headline writer often has not written the article, and so is one step removed from the interview or the original scientific work. I don’t think the mistaken narratives are ill-intended and I don’t think the reporter is simply trying to get people to click through. But there is a tendency when trying to get attention to hype content, and we all need to fight that, including scholars.
Why is it so important science communicators take note of this piece, and your recommendations?
Journalists don’t wake up in the morning realizing that across time the stories that they write influence the way people think. Journalists are on deadline, trying to get the story right, trying to be clear, trying to find metaphors that will make sense to people. They’re not usually asking, ‘If you put all the stories together about this subject across a three-year period, what would people conclude about how science works?’ But across time public understanding about how science works is affected by journalism. It doesn’t only come from what we learned in high school or college science courses. A journalist, with very little extra effort, can increase the accuracy of public understanding and minimize public vulnerability to distortions of science. A bit of extra work creates a social benefit. Their stories matter because across their stories they will help shape how the public comes to see science.
