Trevor Butterworth is Director of Sense About Science USA, which advocates for evidence and transparency in science and technology in the public interest. He is also editor of STATS.org, a collaboration between the American Statistical Association and Sense About Science USA that promotes statistical literacy in the news media. He has been a journalist in the US for over 15 years, writing about data and statistics and how they are interpreted in our so-called "knowledge economy," especially in relation to risk and regulation. He’s written for the New Yorker online , Harvard Business Review, The Financial Times, The Wall Street Journal, and many other publications.
+ Full Transcript
Bob Long: There was a time when young people went to college for a degree so they could get their foot in the door as a newspaper reporter or a radio television reporter or anchor. We used to recognize certain reporters for being great wordsmiths, others for their interviewing skills and still others for their melodious voices. These skills may still be important,but journalists today need a new set of skills, these tools can assist them in taking a complex story based on statistical studies and help people to understand how that relates to them. I’m Bob Long and I welcome you to another edition of Stats and Stories where we explore the statistics behind the stories and the stories behind the statistics. Today we have part two of our interview with journalist Trevor Butterworth focusing on the new equation for modern journalism. Joining me for this discussion are Miami University Media Journalism and Film Chair Richard Campbell and Statistics Department Chair, John Bailer. Our guest, Trevor Butterworth is the editor of Stats.org, a joint project between the American Statistical Association and Sense about Science USA promoting statistical literacy in the media and society. He’s also written for The Financial Times, The Wall Street Journal, Washington Post, also a science writer for Newsweek. So Trevor, we welcome you to this edition of Stats and Stories.
Trevor Butterworth: Thank you for having me.
Long: I, boy I hate to do this, because we could spend a half an hour, and I don’t want to spend a half an hour, it doesn’t deserve a half an hour, but, when we talk about statistics in journalism the one thing that just drives me up the wall, and probably everybody at this table is presidential polling, because I hear all of these stories of so and so is ahead by three pip points and then I look at the margin of error and it’s about the same. So, when we talk about statistical reporting that to me, strikes me as one of the great drawbacks of American media today.
Butterworth: Well yes, and it seems to be an ineradicable problem. So, I’ve been in the, you know, I graduated from journalism school in 1998, and every election cycle that comes around you have the same old same old and then you have a raft of criticism, ‘why can’t we do it better?’ Perhaps because it’s just- pre-packaged- you know the polls give you a ready-made story that goes out with very little effort expended on making it comprehensible. And the truth is that, there’s nothing worse, as in radio or TV to have blank air time. The reality of polling is that there really is, maybe a lot of blank, and not a whole lot of insight to be gained from this sort of thing. But, the problem is so redoubtable, that I’m not so sure you can actually do anything. People like, you know, it’s a horse race and we’ve seen these trends, we’ve seen a lot of baleful trends in the media coverage of politics, from the shrinking of the soundbite to a meaningless seven seconds, from the focus on the horse race over substance, from the migration of serious news coverage on TV from the evening shows to the early morning shows. But you’ve also seen the rise of other kinds of journalism, Jon Stewart, and now David Oliver and Trevor Noah, that seems to be the proper response, is satire. So when something is particularly corrupt you get a measure of the response and I- I mean the real interesting thing would be to know whether this sort of stuff bothers the American public, more specifically the part of the American public paying attention, or do they love it? Do they love going ‘oh three points up, three points down, this is so exciting!’
Long: Richard Campbell I’ll go to you for the next question.
Richard Campbell: Well I think some of us do watch it as sport, or as entertainment and I think that knowledgeable people have some sense that these numbers don’t mean very much, but they drive larger narratives of is the bad guy winning, is the good guy winning , who’s ahead, who’s behind. I guess, one of the things I’m interested in is, one of the things that’s happened to media, particularly because of cable, we have a lot of opinion shows, and a lot of good journalism, I’m looking at the old Nightline model, that Ted Koppel had where he would start with evidence. He would actually send a reporter to a real place to talk to people, to document, to find out what was going on. And today we have, you know, in the evening it’s pretty much talking heads and there’s very little money spent on reporting, sending reporters out. But we also are living in a world increasingly where we have science deniers, people that say well, in this world, every opinion is- they’re all equal, it’s your opinion, it’s equal to anybody else’s opinion. Well, I think we all know that opinions based on evidence are actually better than opinions that aren’t. While we have this moment where we’re paying a lot more attention to the importance of numbers and quantitative literacy, we’re also living in a world where that’s at odds with people that want to say ‘well, you know, I just don’t want to look at that- that goes against my world view’ I mean how do we respond, I think?
Butterworth: Well, that’s really, I mean, first of all we respond by looking at, there’s a lot of very interesting research that's gone on in the sociology of science communication and you look at the work of Dan Kahan at Yale who has, you know, really shown that, the, kind of the big finding is that it’s not simply a case of pouring more knowledge into the empty vessel of the supposed, uneducated or uninformed American. In fact in case of climate change he’s shown that the more science educated you are the more likely you’re going to doubt the climate change narrative in some shape or form. It’s interesting, I’m not sure we’re in a worse place than we were when we had a real naïve view that all science was right, which was kind of the 1950’s, flying car model of science. I mean, if you think back to the kinds of critics that warned us about the kind of criticism, the social criticism that existed then, not entirely dissimilar to the kind of criticism you could mount now. You think of Paul Goodman and his argument that technology was really a branch of moral philosophy which was based on a view that science had sold itself out to a military industrial complex that was working with a system that could create a nuclear holocaust, you know, we had some pretty big problems back then. So, I’m not so sure that people change enormously, there is, Bruno Latour who is a French sociologist of science has made a very important claim that I think explains an absolute difference now and fifty years ago. And that is that the right, sort of on the right side of the political pendulum have borrowed a lot of the techniques and theoretical apparatus that actually used to be on the left side of the pendulum, which was a sort of critique of science, critique of epistemology, deconstruction, Foucault will to power, all of that kind of stuff and began applying it to things like the Twin Towers, you know the 9/11 truthers began to question, to see everything as narratives, as narratives that can be deconstructed and if you take that classic approach, classic, that’s the wrong kind of word, but if you take that what had been a standard approach in literary theory and you apply it to something like climate change you can find all sorts of ways in which the text subverts itself. So, and he said ‘what is the monster we‘ - we meaning sort of the left wing theory from the 60’s and 70’s -‘what is the monster we have created, this might be a monster we have created.’ a Frankenstein’s monster of theory. Again, I think Eco has the solution, it’s not an easy solution; I think it’s a very sensible solution. He said ‘we have to stop thinking about science as a body of facts. ‘You know you go into a house and it’s fully built and everything’s as it should be, but rather it’s a house that’s constantly being built and in fact what you need to teach kids in school is the scientific method, and how you infer reliable knowledge. Maybe then there’ll be change, I don’t know.
Long: You’re listening to Stats and Stories where again, we’re exploring the topic today a New Equation for Modern Journalism and our special guest is Trevor Butterworth, director of Sense about Science and editor of Stats.org. Joining me also, I’m Bob Long, are regular panelists are Miami University Statistics Department Chair, John Bailer, Media Journalism and Film Chair Richard Campbell, and John, we’ll turn to you next.
John Bailer: Thank you. You know, the point you just made that science, that understanding the process is a critical part of being able to critically consume evidence, I think of, I’ve heard, I don’t know who said this but, that statistics is the language of science, and particularly the scientific method. And if you look at the formality that’s implicit in testing, in a statistical sense it very much reflects the idea of formally evaluating hypotheses with evidence. And a key component of that, and I think the part that’s a conflict as we think about some of the points that you’ve made, is that you have to be willing to reject what you believe-
Butterworth: Yes
Bailer: and that’s a fundamental touchstone of science, and if you don’t have that, it doesn’t matter what evidence you have.
Butterworth: And that’s very interesting. Like Dan Kahan has said, it’s actually when you see things, when you see scientific beliefs cohering with social groupings, the barrier to rejecting a belief that your community accepts is actually very high. But really the issue is not- in some sense science is powerless when it comes to those kinds of allegiances, and I think, but also, in knowing that I think we can begin to reformulate some of the social problems we have. And the obvious one is that, rather than talking about climate change you talk about weather impact, and that’s something that actually works very well with farmers, who may be very suspicious of this kind of environmental, left agenda on climate change, but if you talk about resource management, you talk about the impact of the weather, then you have a conversation. So, I think these insights from sociology are actually very very interesting.
Bailer: Even with that, changing the narrative somewhat to go in that direction you still have the problem of the uncertainty and the variability that’s part of this, and that goes back to even the earlier conversation we were talking about the polling and the uncertainty and the margin of error, and I think that we have a real pushback against even the idea of considering uncertainty and variability as part of a story.
Butterworth: I think that’s a great point. Tracey Brown, who founded Sense about Science in the UK, recently gave a lecture at the, I think it was the British Library, and her entire talk was the need for us to really start taking uncertainty seriously because false certainty is a contagion that can poison people’s acceptance of science. And, in many ways, the uncertainty is very justifiable, now there is a countervailing argument that says, actually in the recent issue of Philosophical Transactions, looks at the issue of uncertainty and climate change, and says that actually uncertainty becomes an argument for doing nothing, it’s being used as an argument for doing nothing, because it’s uncertain but actually uncertainty increases risk. So, the point is that we need to talk about uncertainty more, and I think that goes, that sort of on the one hand on the other hand, was once considered classic awful journalism, the thing your editor never wanted to see. ‘Well and you know, maybe this, maybe that, and then there’s the other thing, and oh by the way, oh you know we have this, and we really need to take that into account too,’ and the result’s as clear as mud. And, that makes a terrible story.
Campbell: It works against the old ‘there’s two sides to every story.’
Butterworth: Yeah, and, but actually I think being honest, when we have these issues, where the communication is polluted, where people are arrayed against positions, not because they’ve read all the evidence but because of all sorts of other values issues, we need to be really especially honest, have an honest conversation about uncertainty.
Long: Richard, go ahead
Campbell: One of the things, and John’s brought up in this discussion about uncertainty, reminds me that we do live in a world in which the narrative is the dominant way we make sense of experience, and narratives are about a beginning, middle and an end. And uncertainty complicates- it sort of works against what the dominant symbolic structure is in our society. And this is the problem that journalists have too. You know ‘what’s the ending to my story,’ well, uncertainty. It’s not something we’re very comfortable with, and I think this is most cultures.
Butterworth: That, to a certain degree, yes I think you’re absolutely right, but you also have to look at the way people, people are changing their behavior all the time. You know, trivial examples of this would be, to look at my own country, Ireland, all but one county voted for gay marriage. Twenty years ago, ten, five years ago, you would have doubted that result, an enormous change. You look at, say, the issue of trans issues, again, a sort of freak show phenomenon back in the 80’s and 90’s or even the 2000’s, now, the internet has brought about a kind of unraveling of a key anthropological notion in, certainly wester n culture, which is the idea that gender is binary, you’re either male or female. Now we’re looking at gender as a continuum, now that may not resonate with all parts of the US, or Europe for that matter, but the fact that if you look at where this conversation is now, I mean, that’s a huge change, that’s massive. So people are capable of massive change. From a more statistical point of view, you look at the way that knowledge was divined from patterns, extraordinary occurrences of patterns, you read the entrails, and you go ‘oh my gosh the entrails look really strange today’ and then you marry that against a phenomena and then you make a prediction. And ok, took a long time to get past that, but then you began looking for ordinary patterns and ordinary things and then you develop statistics and probability. So actually, we are incredibly good at changing our minds and our behaviors. But I do think that may actually require bringing in anthropologists and cultural historians and technologists to actually create a good narrative of change. I’ve seen in the sociology of communication, I remember talking to somebody, and they were saying ‘yeah people don’t change their mind, they just vote with their opinions, their values, what their friends say, who they respect on TV, that’s all like this iron cage of determinism’ and I said ‘well how do you account for the success of a green peace campaign?’ and he says ‘well we don’t really have a good theory for that.’
Long: You’re listening to our discussion of a New Equation for Modern Journalism on Stats and Stories where we explore the statistics behind the stories and the stories behind the statistics. I’m Bob Long and I’m joined by our regular panelists Miami University Media Journalism and Film Chair Richard Campbell and Statistics Department Chair John Bailer and special guest Trevor Butterworth, director of Sense about Science and editor of Stats.org. You know one thing I want to get into Trevor, it just seems to me, we keep looking at America, everybody talks about, Newspapers are dead, radio and TV are dying, and the internet is everything but it seems like the internet really has made a huge difference in terms of, I look at a lot of what I see in traditional media, and we’ve talked about how it’s not serious, there’s so much entertainment that’s crept into the news. Do you see the internet as a way where people like you are able to really talk to people who are seriously interested in a lot of these topics that we’re discussing today?
Butterworth: I would say that, first of all, it would be wrong to think that the media wasn’t full of entertainment back in the days of the 1930’s or even the 1890’s, the entertainment was certainly different, I mean one of the fascinating aspects of British literary history and literary culture were that there were eight million editions of the penny classics, penny classic poetry, penny classic drama, which was a very particular brand in circulation in Britain in the 1890’s. So there was a mania for literacy, but there was also lots of trashy stuff, which doesn’t get preserved in the libraries or is buried out of sight and we’ve all forgotten about, we’ve all forgotten about how America was sort of enthralled to Walter Winchell, although, arguably, Winchell was a better judge of character than Walter Lippmann, I mean he did- especially when it came to Hitler, so, and predicting the rise of Nazism. So, don’t always be hard on the gossip columnists, they have insights too. So, I’ll give you an interesting example of why I think the internet has improved things. If you, so remember back to the Alar, the scandal with Alar, which was the pesticide being used on apples, and 60 minutes ran a big story on this, and Meryl Strep spoke out in front of a Natural Resources Defense Council, it was an absolute uproar. The result was, very quickly Alar was finished, the apple crop was ruined, I mean you had all sorts of hysteria, police cars chasing down school buses to retrieve apples from lunch boxes, I mean there was all sorts of nuttiness that went on. But that was it, that was a singular narrative. CBS ran the story, people either agreed with it or they ignored it. I don’t think you could really do-apart from maybe what Volkswagen did recently, which was just so flagrant, the thing about Alar was that according to people like Bruce Ames, a legendary cancer researcher and toxicologist, there was no real risk from Alar, but there was no way of counteracting, because if you were at ABC news at the time you had to say ‘well am I going to go to war with CBS?’ well, no, your editor didn’t really-he wanted you to do your own story. The internet for good and bad, has opened, has created a whole series of new forms of journalism. You have science bloggers, you have, you actually had, I remember because I was involved in setting up a site in ’98 for daily media criticism, I remember journalists saying ‘this is a waste of time, you should be out reporting stories, you shouldn’t be doing criticism’ which struck me as kind of crazy, because the media was a very powerful institution, why shouldn’t there be criticism? Now the internet enabled a massive amount of criticism, and some of it was really good. Now there were ways to share sources and spread good information. And while there’s a massive amount of rubbish, of garbage, the fact is that the amount of good stuff, I really believe prevents that kind of runaway narrative like Alar from ever happening again.
Long: John Bailer, go to you for the next question.
Bailer: I think about the sites and I wonder how much of the sites are likely to be preaching to the choir. You know, that you sort of self-select the input sources that you have, and I’m going to ask you if you have an example of a story that you think that there was some statistical insight that really made the story, or complimentary, a story where there was some statistical observation that really killed the story.
Butterworth: I think this happens all the time, I think, it’s not just the in the questioning of the design, or the correct interpretation of the statistics, it’s simply knowing what else is out there. I mean one of the hallmarks of the way that BPA has been covered, there have been thousands of stories on BPA, we’ve looked at how the media has covered certain issues on this. We did an analysis at George Mason University in which we looked- so what was the key piece of information that if you, as just an regular audience member, or viewer or reader, needed to know on this story to make sense of the competing claims. Well, you actually needed to know that the European food safety authority did a series of risk assessments every two years, you needed to know those results, you needed to know what they thought. Why Europe? Why not the FDA? Well because one of the narrative elements of the BPA risk story was that American regulation was failing because it wasn’t following the precautionary principle. And you got on CNN, Dr. Gupta coming on and saying ‘we need regulation like Europe, Europe protects its people better than we do’ from these dangerous chemicals in products. And the reality was of course that 2006, 2008, 2010, 2012, 2014, EFSA continually said ‘BPA is safe at current exposure levels, we can’t replicate the studies that claim they’re dangerous, and we’re operating under the precautionary principle.’ So that’s a pretty big chunk of information to counteract with all the endless number of scare stories. So when we actually looked at the media coverage, only 6% of the news stories mentioned EFSA’s risk evaluation, and when they were mentioned, the people mentioning them were industry sources. So they came, so EFSA’s independent evaluation was coated with spokesperson for American Chemistry Council, so of course, it never got treated, it subtly given a different credible status than the independent scientists. So, sometimes it’s actually really simple, it’s just report the story, you don’t even need the statistical data. You just need to know-you need to talk to the other side.
Long: Richard Campbell, we’ll go to you.
Campbell: This discussion reminds me of Neil Postman’s 1985 book Amusing Ourselves to Death, where he talks about that the problem in the 19th century, he’s talked about it in other books since then, that the problem of the 19th century was that we didn’t have enough information, and the problem of the late 20th century and certainly today is that we have too much information. One of the things I like about listening to you is that you’re very optimistic.
Butterworth: I’m completely optimistic
Campbell: And I feel like-
Butterworth: Well not completely, look there are winner and losers, but go on.
Campbell: I feel like one of the challenges today, and I see this with our students, is they sort of retreat to their Facebook pages and their twitter accounts, that they feel overwhelmed. I mean it’s certainly very different that there’s so much information- we talked a little bit at the beginning of just all the poll numbers, that we just turn this stuff off, and our challenge is how to help them find those resources that are reliable and valid and sort of tune out this other stuff, but they often sort of retreat into what has been called ‘the daily me’ by, I’m forgetting who coined that term, but just-
Butterworth: What’s being posted on their Facebook page?
Campbell: Yes
Butterworth: This, I mean, Yes, so many issues raised by what you just said. I think we have real problems with data literacy, you know, the research carried out by the British Museum which showed that people raised on google didn’t know how to use google effectively, kids that is. That the best way of analyzing research was to actually have had parents who were brought up in the old card(?) system, because whatever they were teaching their kids gave them an added analytical element. I think, again, an anecdote, but maybe a telling one. One of the most successful personal organizers I know is increasingly being asked to organize people’s reading habits because there’s just too much stuff. So, one potential solution to this is the one nobody wants to hear, which is that if the internet is the asteroid hitting the news media, the die-off has yet to really truly happen. And, Tim Cook’s comment that Apple are going to value privacy by using ad blockers could really be a devastating result for a lot of news organizations whose only business strategy is digital advertising. Real problems there, I can’t downplay those, but the downside of all these situations are new opportunities to build new products. I mean if you look at cutting-if you look at things that happen-National Press Foundation gave an award to somebody who I genuinely believe the future historians will see as one of the most important journalists of our era. Brian Krebs, Krebs on Security. He was downsized by The Washington Post the guy was widely regarded, I know I did a lot of background reporting on this topic, widely regarded by the top experts in cyber security as the journalist who really understood it. He has started his own- he went off and started his own site. He has broken all the major cyber security scandals, he has, and he got an award from the National Press Foundation, which has increasingly embraced digital change, and he gave a speech telling journalists not to be afraid to just, if they wanted to go and do it themselves they should come and talk to him. So, ok that’s just one person, but you know what? One person who knows the subject really well, whose providing value. So what happens ultimately? Well, even people like Nick Denton, who runs the Gawker empire has said the race to the bottom is just pointless. More ad clicks, more clicks, is not the way forward, there has to be better content. And I think what I want to say is that it’s very easy to become disenchanted or depressed about where we are in this particular moment, and not see the broader cultural movement taking place. Which is a broader movement of change, in which new platforms are being created, I mean, and one key example, I’m sorry rushing my thoughts, my thoughts are rushing in all at the same time, if you look at the concerns about privacy right now, which will only amplify, and you look at the technological solutions that are in place, new protocols, new potential platforms, you begin to see the moment we’re in as the wild west. We’re the wild west of the internet, the end of one big era, and the start of a new one and where we’re going to be in five or ten years’ time with data is going to be a very very different place, but the place that’s going to be in, the absolute thing I think we can be really certain is that journalists who have the quantitative skills, who can decode numbers into good stories, into clear communication, those people will be the ones who will take advantage of that new environment.
Long: Trevor Butterworth, we want to thank you again for joining us on Stats and Stories today to shed some insight on this new equation of modern journalism. If you’d like to share your thoughts with us too, on our program you can send your comments to StatsandStories@miamioh.edu. So be sure to listen for future editions of Stats and Stories where we always talk about the statistics behind the stories and the stories behind the statistics.