Ashley Steel is a statistician and quantitative ecologist with experience in academia, government and international organizations. She wrote “The Truth About Science: A Curriculum for Developing Young Scientists” which guides middle school students through the process of conducting research. She also designed and taught a course on statistical thinking at the University of Washington, Seattle, where she is affiliate faculty. Passionate about the value of probabilistic thinking in every-day decision making, she volunteers at science fairs and supports teachers in understanding statistics.
Megan Higgs is a statistician, freelancer, and owner of Critical Inference. She has experience in academic research and teaching, as well as consulting and scientific collaboration in many disciplines. She believes in the importance of raising awareness about limitations of current uses of statistical methods and inference in scientific practice and communication.
Episode Description
There's a lot of statistical information shared every day in news stories. Everything from COVID cases to economic data is Quantified help us better understand our world. But do news presentations really help us do that? And what do statisticians think about the way journalists are covering their work, that’s the topic behind this episode of Stats and Stories with guest Megan Higgs and Ashley Steel.
+Full Transcript
Rosemary Pennington: There’s a lot of statistical information shared every day in news stories. Everything from COVID cases to economic data, to athletic performances quantified to help us better understand our world, but do news presentations really help us do that? and what do statisticians think about the way news media presents statistical data? That’s the focus of this episode of Stats and Stories where we explore the statistics behind the stories and the stories behind the statistics. I’m Rosemary Pennington. Stats and Stories is a production of Miami University’s Departments of Statistics and Media Journalism and Film, as well as the American Statistical Association. Joining me are regular panelists John Bailer, Chair of Miami Statistics Department and Richard Campbell, former Chair of Media, Journalism and Film. We have two guests joining us today to talk about a new blog sponsored by the International Statistical Institute; Statisticians React to the News. Megan Higgs is a statistician, freelancer and owner of the blog Critical Inference. She believes in the importance of considering the philosophical and general practice issues involved in using statistical inference to inform science management and policy. E. Ashley Steel is a statistician and quantitative ecologist and considers herself a quote statistician who is passionate about statistical communication, end quote. Higgs and Steel are also the minds behind the statisticians React to the News blog. Thank you so much for being here. That was a lot of statistics and statisticians one introduction to this program. So, but we’re so happy to have you here today.
Steel: Thank you very much. We’re happy to be here.
Higgs: Yeah thank you so much, Rosemary.
Pennington: Uh, I just want to get this conversation started and ask you how you decided that you wanted to create this blog? Sort of what was its genesis.
Steel: So, I think we all read the newspaper and so much of how we understand the world is based on observations and they’re often structured into data. And that’s what statisticians do. We think about data and what it means, and I think in many ways journalism and statistics have a lot in common right? We help people understand what’s happening in the world in different ways and when I was reading the news about, you know, it was about COVID-19 and there were all these data flooding us constantly and I was spending a lot of time applying the statistical skills that I have from my professional life in understanding the news. And I started to wish also that I had other kind of information like what would a virologist say? What would an economist say? And I came to understand that our professional lens, our expertise, really helps us understand one aspect or one facet of the news, and it seemed like a great opportunity to share that; to share how statistical training and experience helps us better understand the news and the data around us. And I want to also credit Peter Guttorp , another statistician who was working simultaneously on an editorial piece and we felt- we just got to discussing how important it would be for us to better communicate how statistics can better help people understand the news.
Bailer: Are their particular skills that-statistical skills that you think are critical for consuming the news and reading the news?
Steel: I think statistical thinking skills- of course, if I’m talking about statistics, I’m going to get around statistical thinking that came really quickly. So, nobody needs to really do a lot of calculations when they’re reading the news. And I think people confuse statistics with calculations. So I don’t read the news with a calculator by my side but I read the news with this understanding of how does how many observations we saw influence how much we should believe the conclusions, but I read it with an idea of what are different kinds of probabilistic events and if we’re just looking at the last the last events in the chain what do we now about the earlier events in the chain. I read the news with all these kinds of probabilistic thinking skills and I think it helps me. There are pieces that are missing from other disciplines, but these skills help me.
higgs: Yeah I can chime in there. I feel like from my perspective there’s a maybe an understanding of the complexity that’s going on behind the scenes in terms of how the data are collected, any analysis that went on and then the interpretations based on the results that can’t really be summarized simply and neatly and briefly in a news article. And so, I feel like that’s something as statisticians reading it we have this nifty understanding. So I feel like that was one of my excitement or motivations to be involved is to kind of help get that perspective out there and not that we can explain all the complexity but just to raise or spark the curiosity about what else is there that can’t be covered in the news article but the blog could provide the perspective and knowledge that there is more there.
Campbell: Well I would like to ask a question for whatever reason you allowed John to and this is what I want to know because you start out with telling this story of our governor who has taken three tests that we know about one positive, two negative, and Ashley and Megan probably don’t know this but our governor went to Miami, most of his kids went to Miami, and I want to know if the DeWine family shows up in Oxford do I have to avoid them?
Bailer: Well, they probably would avoid you Richard. That’s a different question. I guess it’s just a- that’s one of Ashley’s posts about the conditionality of arguments, I think her recent post talks about that. well that’s, so Richard noticed that Megan and Ashley allowed me to contribute to this as well and as part of the promotion of this within ISI’s purview of the public voice and that’s one of the things that Peter Gunthorpe has been a proponent of as well. But Richard, in terms of the question, this was one of the things that the blog involved which was reacting to the news. And it was pretty amazing to see these stories on consecutive days where a positive result was followed immediately by a couple of negative results and there were different tests and there were different ideas that were going on. What does that mean in terms of statistics and I guess we could have just as easily called it deconstructing the news from a statistician’s perspective, but it was reacting to that. so, in terms of answering your question I don’t- if they are travelling here you probably don’t have to worry about it.
Campbell: Okay that’s all I need to know.
Pennington: I do have a kind of related question. So there are several posts on the blog obviously given the moment we are living in that are about COVID because it’s sort of the unavoidable topic of a lot of statistical things right now because the stats for the pandemic are everywhere and trying to understand them is sort of a mission of all of us I think. But you know six months from now maybe if this pandemic has sort of, you know, obviously not gone away but is less than some what other kinds of things would you like to see? are there other ways of thinking about statistics that you’d like to see people reading about on the blog? Or do you have things in the pipe you’d like to talk about?
Steel: Yeah sure we expected there to be a COVID-19 theme to with obviously because of the times we are in, but I also feel like COVID posts that have- sorry let’s stop that for a second. So the COVID-19 related posts that are already published have some underlying messages that can really be applied to other stories as well but coming up shortly we have we’ll be talking a little bit more about official statistics, so there is world statistics day, so we’ll be covering more about that, we’ll also be covering more about election statistics I believe, when we get closer to November and the US Presidential election. Hopefully for other countries as well. One of the really important things is having this international perspective that comes from, you know that’s an expectation because of sponsorship of the International Statistical Institute but also a really exciting part in something that really drew me to wanting to be involved as well, so we have had posts so far from Brazil, Italy and Sweden, and have commitments from contributors from Palestine, New Zealand, Puerto Rico, Philippines, Thailand, Nigeria, Switzerland. So, we really are trying to get out. So, what news items will come up hopefully will be country dependent, as well, which is fun and a new perspective.
Bailer: So one aspect that you discuss about this blog is having topics that might be thought of as timely, as reactive to a particular news story, and others that might be evergreen is sometimes said n this business, that would be themes that continue on. Can you talk a little bit about that distinction between the timeliness and that mix and what are some of those kinds of evergreen topics that you could envision covering here?
Higgs: That’s a great question. So, we originally thought of these neat categories of these timely versus evergreen, or a specific story versus a general reaction which is how we’ve categorized them on the blog. You know as things go, nothing is turning out to go as neatly as categorized, so most of them have a particular story that they are referencing and talk about but then they go on to provide general takeaways and lessons so we learn. So, it’s been really hard for me to actually categorize those. But I hope that for some of the evergreen topics that we’d hoped to have more kind of general statistical thinking type ideas like Ashley mentioned in the beginning, some really underlying things to spark curiosity in people about how they’re reading news has statistics or statistical information related in it but I think it’s great to hone in to have that one reference in articles as well.
Higgs: I think if the blog evolves and there are more and more stories we’ll start to see links between stories and that will naturally lend itself to more and more topics that are more general and I think that will be exciting because it will demonstrate to readers how one statistical idea can help us understand climate change, racial inequity, immigration patterns and sports. It’s one foundational idea and I’m excited about that.
Campbell: I wanted to ask about- and if Megan wants to follow up but I’ll forget my questions if I don’t ask them right now. So, it’s a two-part question, one, do you know if journalists are reading the blog yet? And two, what can they do better? What can journalists do better?
Megan: Do you want to go first Ashley?
Campbell: Go ahead.
Ashley: Okay well I don’t know; you and Rosemary might be the first journalists to read the blog.
Campbell: That’s sad.
Higgs: I hope that’s not the case but you’re the first- you’re the only two that I know of for sure who have read it. But yeah, part of what we’re doing is trying to promote to a wider audience, maybe Ashley can talk more about audience later, but I’ll answer my version of your question about what I think journalists can do better. I think that my kind of main frustration lies within what happens in journalism regarding reporting of statistical information is the simplification that must happen. You have to tell a story in a very limited number of words and so what journalists and scientists as well tend to do is oversimplify and lose the complexity and I think that has to happen to some extent to be able to do our jobs, but I think that we can be a lot more mindful about the language that we are using in order to not oversell the simplicity, not present it as answers and like we’ve used statistics to reduce n=uncertainty which is not what we have done in any of these cases right. But that’s how this is kind of used is that calculator get us an answer type of way. So, I think a little example beforehand, it’s really common to use the word determine in reporting because that’s kind of a nice term to grab on to. That was a term I outlawed from my classes. We do not determine anything with statistics. So I think- and that’s a qword that creeps in to journalism a lot and I understand why and most of just don’t, you know, once you think about it once you’re like oh, that makes sense that I wouldn’t want to use that term, but even- I’m going to stop for a second while I look at my little example that I made, because I- so for example you might say researchers determine that X causes Y. right a very exciting headline, where we could easily change that to still be something short that’s more like early research suggest that X may cause Y in some people. So, like those two to me are just so different, and I think that when I read that as a statistician, I take away something very different. So, I’m kind of hoping that’s kind of the little pieces of curiosity in those type of reporting that we can promote.
Steel: And I think there’s also a demand issue, so if people are demanding really certain exciting results then it’s very hard for journalists write in a way that expresses how science moves with a bit of slowness and awkwardness and we have uncertainties that we’re constantly revising and if this blog can contribute to a better market for journalists so that if people want to consume news that has those uncertainties, they don’t just write it off as oh, we don’t know yet, they read it as an exciting step, that would be fantastic.
Pennington: You’re listening to Stats and Stories and today we’re talking about- oh, I’m going to start that over again I was going to say statisticians, for some reason I keep switching those words around. Three two one Charles. You’re listening to Stats and Stories and today we’re talking about statistics and news with Megan Hicks and Ashley Steel. Now you both have sort of expressed a commitment to sort of the public communication of statistical information. You have the critical inference blog Megan and Ashley you said this is something you’re passionate about. Why? I mean because it feels like it’s a step beyond traditional training. So you know to teach journalism is difficult because I know how to do it but then when you sort of step outside the actual doing to the discussion about it it sort of a more difficult place to sort of- I said sort of a lot- to manage or to be in. so I wonder what was it that made you feel so compelled to take that step to try to push for clearer communication of statistical information?
Higgs: We’ve also been working on a course at the University of Washington on statistical thinking and it includes statistical communication and all of that work is really because I believe it’s maybe too light of a word. I know that people can make better decisions in their personal and their professional life if they understand probabilities and data. And so the more- somehow that has been handed to the statistical community, that kind of knowledge, and I think we kind of need to do a better job at spreading it out again and sharing with people why is it interesting? Why do you need to think about this? How could you make a wrong conclusion or a wrong decision and how could you improve on your decision making? Even from parenting you have to take all these different kinds of decisions, they’re all probabilistic. Your own health decisions about whether to go on vacation, now we’re living in a world where every decision should you go grocery shopping at four o’clock or five o’clock is a probabilistic health related decision and I think the world can be a better place if more people understand statistics.
Bailer: In terms of parenting I always used random reinforcement.
[Laughter]
Bailer: You know keep them guessing. So the question that I’m going to suggest that Megan and Ashley you guys might have a question for Richard and Rosemary let’s put the journalists on the spot and lets say play journalists, you have a chance here to say what are some topics or stories that you would like to se covered as part of this blog?
Campbell: Well one of the things that has been one of my pet peeves as we have interviewed a lot of scientists and statisticians is the question of random sampling and why that wasn’t done more often and I think that we would know a lot more I think, that’s one story although again I think as you guys have pointed out this is a hard story for journalists to do because most of them don’t have the training or background to do this. I think from what I’ve learned it’s a lot better than it used to be. I think there’s so much good journalism out there that does things that are helpful to me. So that’s one story that I would like to see tackled, and journalists go after. I think I’ve heard a lot of complaints about this, but I haven’t really seen good reporting on that particular story.
Pennington: For me, so I did, in my past life medical and science stories were one of my beats and so I did a science literacy shop for journalists a million years ago out in San Francisco, and one of the most useful things I ever got was that a statistician came in and walked us through how to read a study. And so I think maybe down the road it might be worthwhile, especially if you’re thinking of you know journalists or the public coming to the blog and reading about statistics , like how do I read a study if I see something in the news I think looks interesting and I want to go and read the study how do I read it in a way where I don’t need to know maybe all of the layers of the statistical analysis but can get enough to understand whether there’s some validity or something worth trusting in this data is something that could help journalists as well as laypeople.
Higgs: Yeah that’s a great idea.
Steel: That’s a fantastic idea and one of those posts that I hope will arise, maybe even regularly, are posts that are of the category- five questions to ask yourself when you’re reading a news story. Because helping people know where to even focus- it’s very similar to what you’re saying, how to read a study, but ask yourself, did they mention sample size? Did they use a word like determine, prove, so I think we can make a lot of progress there but it’s going to take I think first a collective of statisticians to come together about all these different news stories so that we can agree- well maybe not. Statisticians don’t agree very easily, but we can come together with a good pol of questions that we can organize and provide in a very useful and efficient format.
Campbell: Well I want to turn the – go ahead.
Steel: Oh I was just going to say that to me it’s really important even if a person, you know we can give a person the skills to look into something or give them advice for how to look into a study and how to look further, but the reality is that most people are not going to have to do that it’s something Even just the recognition that there is more to this story. Like there is a rest of the story that is not being covered here I think is incredibly important. And to just get people to sit with that and realize it and not let that part kind of gloss over.
Higgs: That’s a great- I love the description of thinking about the resto of the story, and I’m going to hold that close. One of the phrases that I use a lot is efficient skepticism. Like we’re so skeptical that we can’t read anything without thinking it’s false or it’s wrong; that won’t work, we can’t get any information. But we shouldn’t be reading it- we should have some principles for how we can efficiently have enough skepticism to guide our understanding and I think that that’s a good principle also.
Steel: Yeah that’s moving skepticism to using curiosity, which I really- I used to use like healthy skepticism, but I think I really want people to be curious about the rest, like what else is there?
Campbell: Id like to go back and return the focus to statistician.
Steel: Yes.
[Laughter]
Campbell: So, Ashley you talk about the course on statistical thinking and you do a pre-course survey. You talk about in the stuff that I’ve read about this you talk about things that even statisticians get wrong, in terms of statistical thinking and I want an example.
[Laughter]
Bailer: I think that’s an idea that works for journalists.
Steel: I think there’s a lot of, of course statisticians get things wrong all the time, that’s how we lean. Like any other profession we learn and then we figure out what we did wrong last time and how to improve it the next time. There’s a lot of ways that the human brain just thinks about probability wrong. A lot of these were described by Daniel Conman and his Nobel Prize winning work, and I heard an interview by him where he described how he’s been studying these things for 20 years and he still makes incorrect assumptions. And I think that kind of humility- and I’ll get to an example in a minute, but what I think that kind of humility that we are naturally going to make certain kinds of mistakes is really helpful we are naturally going to see a cloud of dots make a pattern tell a story. We’re just going to do it. And we have to have a little red flag in our minds that says wait, wait wait. How many dots are there? How likely is that story? Could I have seen a similar cloud with just a random chance? And I think of it a lot like optical illusions. So, we see optical illusions and we see that one with the three lines and the arrows and we instantly think this is an illusion, this is a trick. I have to be careful. And I think there’s a lot of things like this about probability that we just can learn that when you see a small cloud of points in a pattern we should think wait, stop, let me check. When we see a probability we can think wait, stop, conditional on what? There’s a lot that we can do like that.
Higgs: I think that humility is a great point and kind of bringing it back to the blog too, one of my visions, I really don’t want the blog to become a statistician as hero narrative, or a scientist is hero narrative. I think that’s one of the problems that underlies communication and trust in science and journalism. So, but I think that’s all related to what Ashley was saying.
Bailer: I really like that characterization. That’s really nice. I mean the collaboration between statisticians and journalists I think is generally something you might find accepted and celebrated in this group.
Steel: Absolutely. I want to say something about the audience because in this interview was the first time I’ve thought of journalists as audience because I haven’t been thinking that we’ve somehow been telling journalists what to do. I’ve been thinking of it as a collaborative process. And I think there are so many audiences. Like one I’ve had was teachers, and I’ve imagined with this typical thing when some poor kid has to learn calculus, instead of probability which would be so much more useful, and they say you know like, why would this ever be useful? I’ll never need this in real life. So not only teachers are teaching statistics and some kid says I’ll never need this in real life. They have just an arsenal of ammunition all the ways that statistics can help them understand real life. And I think somebody made a comment on the blog about how they hope this can also enliven statisticians and help them remember the joy of communicating and the importance of- I mean we all get a little tired of our work over time and I think that it can help statisticians remember how valuable our work is and how much we need to communicate it better. So, I think there’s quite a lot of different audiences. It’s not meant to be a corrective blog; it’s meant to be an expanding ideas blog.
Higgs: Yeah I think the words that we came up with were instead of narrowing it down to groups in the audience it was scientifically literate and curious or something like that. maybe even scientifically literate is too restrictive. But yeah just curious about another perspective on the news.
Steel: So, I have a really funny story because I posted this on my personal Facebook page of course, when I wrote a blog and I didn’t expect personal friends who aren’t’ statisticians to have too much to say about it and I got the following comment back, and it said what’s the risk of a slightly overweight 36-year-old Egyptian drummer will be hospitalized from COVID if he gets a vanilla milkshake later tonight? And I was thrilled because of course it’s silly; it’s super funny, but it also says this person read the blog, thought about how those ideas that are about a newspaper article can also be applied to real life decision making and I was- to me, that’s the audience for the blog. For people to make these connections and to sort of highlight statistics is for so many personal decisions.
Pennington: I was certainly one of those kids who was like I hate math, I don’t understand, you know I was like, what am I going to use this for and then I took statistics in grad school and I had to learn regression by hand and all of these things, which was not fun but at the same time oh wait a minute, this applies in so many places and there was like that practicality in the way it helped me understand things. And I was like if someone had been able to convince me that this was math could lead me to down the road, maybe I would have cared a lot sooner in my youth, and have paid a bit more attention in those classes, but it was like this really a-ha moment in grad school like oh, stats is actually really great. So yeah it would be great if the blog could provide those moments for other people like maybe Richard.
[Laughter]
Bailer: Were going to have to do a golf episode or a golf article.
Campbell: Hey I want to know when we are going to drive calculus out of the high school curriculums and put in statistics, because calculus turned me off of math.
Steel: Any day. I am right there with you.
Campbell: Why aren’t we doing that?
Steel: There are some- it depends on the high school, but they do have-
Campbell: Well a lot more of our students are coming with statistics courses, but still I bet it’s fewer than half.
Steel: I hope that this blog can help fix a item bit of that because one of the problems is we don’t have enough teachers who love statistics at the high school level who know how to do statistics and can communicate it well. So, my daughters’ statistics teacher was the drama teacher. And that just made me bananas. And so, I think we need to get people excited from all kinds of different career paths about statistics and then we’ll have teachers and then we’ll have students. Megan Sorry.
Higgs: No, I just wanted to say something after. So, I think that is so important, but I want to put a huge caution warning that you don’t have to come through math to become a statistician. So, I did not come from a math background, I came from a science background. I started another Ph.D. before actually realizing like why are we using statistical methods like this like what is going on? Like I don’t have the background to do this and why am I being asked to? And then went back to stats. I mean I didn’t mind that, but I liked that part of it, but it doesn’t have to come from that. so even thinking about high school education I would almost rather my kid’s science teacher teach stats than a strict mathematician because that feeds into this calculation get an answer myth of stats that we’ve got to get rid of that. we’ve got to be in that dirty gray area and embrace that rather than the get the right answer mentality.
Steel: It’s interesting because I also didn’t come from a math background. I liked it enough and then I switched majors and I went on and then in graduate school I had to go to a statistical consultant. And it was such a horrific experience. They spoke to me in words I didn’t understand. They told me everything I was doing was wrong, and I left the room and I said I’m never going to be subjected to this again I’m going to learn it myself. And of course it’s fascinating but it’s a reminder to me when I go and I’m trying to communicate with people how easy it is to turn people off and scare them and there’s so much more to statistics than the very careful calculation that you get taught very early.
Pennington: Well that’s all the time that we have for this episode of Stats and Stories. Thank you guys so much for this conversation.
Bailer: This has been fun thank you.
Steel: Thank you that was really fun.
Higgs: Yeah it was great.
Pennington: Stats and Stories is a partnership between Miami University’s Departments of Statistics and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple Podcasts, or other places where you can find podcasts. If you’d like to share your thoughts on the program send your emails to statsandstories@miamioh.edu or check us out at statsandstories.net and be sure to listen for future editions of Stats and Stories, where we explore the statistics behind the stories and the stories behind the statistics.