Jena Barchas-Lichtenstein is a linguistic anthropologist who leads the media research at Knology, as well as Associate Editor of Public Anthropologies at American Anthropologist. They lead Knology's participatory collaborations with news organizations, including PBS NewsHour. Their research interests center on the relationship between scientific authority and questions of epistemic and probabilistic certainty. Their doctoral research focused on media circulation and socialization into the global community of Jehovah’s Witnesses.
Laura Santhanam is the Health Reporter and Coordinating Producer for Polling for the PBS NewsHour, where she has also worked as the Data Producer. Santhanam uses narrative and numbers to tell stories. Her work at the NewsHour merged her career as a newspaper reporter at the Chattanooga Times Free Press and the Arizona Republic with her work as a media analyst at Pew Research Center. She previously worked as a senior climate researcher at Media Matters for America, where she wrote blogs that examined climate change and managed data-driven projects on the media’s coverage of issues related to energy and the environment.
+Full Transcript
Rosemary Pennington
Just a reminder that Stats and Stories is running its data visualization contest to celebrate its 300th episode, you can grab data about the show to analyze and submit your entry at statsandstories.net/contest. Tour entry has to be there by June 30.
The news landscape is continually in flux as new media technologies are developed and the audience needs to shift. This mix of new tech and new needs has highlighted the importance of ensuring audiences understand quantitative information. A research partnership between Knology and PBS Newshour is studying how people consume news and numbers. That's the focus of this episode of Stats and Stories where we explore the statistics behind the stories and the stories behind the statistics. I'm Rosemary Pennington. Stats and Stories is a production of Miami University's Department of Statistics and media journalism and film, as well as the American Statistical Association. Joining me is regular panelist John Bailer, emeritus professor of statistics at Miami University. Our guest today, or guests, are Jena Barchas-Lichtenstein and NewsHour’s Laura Santhanam. Barchas-Lichtenstein is a linguistic anthropologist who leads the Media Research at Knology. They're also associate editor of public anthropologies at American anthropologist daily, and acknowledge participatory collaborations with news organizations, including PBS NewsHour. Santhanam is the health reporter and coordinating producer for polling for the PBS NewsHour, where she's also worked as the data producer, thank you both so much for joining us this morning.
Jena Barchas-Lichtenstein
Thank you for having us.
Laura Santhanam
Thank you for having us.
Rosemary Pennington
Could you just explain how this partnership started?
Jena Barchas-Lichtenstein
I have been in my current role since 2017. And the partnership predates me. So I want to say it dates back to about 2014 or so maybe 2013, not 100%? Sure. Johnny Fraser, who has knowledge he's the founding CEO, and Patty person managing producer for NewsHour, met at a conference and were very taken with one another's ideas. And in some sense, the rest is history. I think we have about 12 grants collaboratively between the two organizations at this point over those last 10 or so years.
John Bailer
Well, that's quite a success story. So when they were telling us, I was taken with each other's stories and their perspectives. Can you maybe give each in turn, maybe talk a little bit about what NewsHour’s story and perspective was and Knology, the story and perspective? And why was that this natural pairing?
Laura Santhanam
Sure, I'll jump in there. You know, me, I feel like the PBS NewsHour really values, and we feel our audience also values, just the the power of context, right, like, you know, very few things in life in the world, and just in general happen just out of the blue, right, there's always a lead up, just sort of a natural arc of a story and event, you know, a discovery, right. And so, you know, we really like to connect the dots, but in our reporting, our audience seems to appreciate that as well. And so, I feel like that definitely holds true when we're thinking about numbers, right? Well, whether we're talking about climate change, and x degrees equals like, absolute doom, you know, in temperature rising, or if we're talking about, say the last few years where it's been kind of weird, and there has been a Coronavirus pandemic, you know, numbers absolutely have played a critical role, both in terms of just how we're planning for the future, but also how we're, thinking about living our day to day lives, right, thinking about numbers on so many levels. So, you know, I feel like that really was sort of the impetus for the work that we've been doing most recently, together with this latest project,
Jena Barchas-Lichtenstein
Zooming out a little bit, Knology is a nonprofit research institute whose mission is practical social science for a better world. What that means is we do nearly all of our work in close partnership with practitioners of various types. As Rosemary mentioned, I lead our media research, so I collaborate with news organizations. And from our perspective, everyone at NewsHour at every level is so committed to the idea that news is a public service to inform and educate people. And I think a lot of the common vision starts there, because they're in it to help adults understand the world. And we had been talking for a really long time from a prior grant that was looking at science among young adults, and particularly young adults who were not currently in school, right. So people were starting to form those adult information habits and figure out what their informational needs were when they weren't in formal education anymore. And we had a lot of conversations about math and numbers and statistics as underpinning all of the fields of STEM and being a real weak point, in terms of both comprehension and reporting. And so our current National Science Foundation funded project really was looking at numbers across a broad set of coverage areas, looking at what the public can and can't draw from them. And I think a lot of it comes back to what Laura was saying about context being the thing that's most often missing.
Rosemary Pennington
So I wonder if you guys could explain how exactly you are studying how quantitative information is communicated and received? It seems like so much with media research, and this, you know, I can be guilty of this as well as a media researcher, is that we end up being very focused on the content and don't always get access to the audiences. And I wonder, given that you're looking at issues around quantitative literacy, like how are you studying audience perception? And what have you found so far?
Jena Barchas-Lichtenstein
We've done a bunch of different activities with Publix. So we have done several large scale surveys, primarily using Prolific, which is an online panel provider, we've done a fair amount of experimental testing. So we'll have, say, three groups of 80 people, each of whom sees a slightly different version and looking at the inferences they draw, also looking at their confidence in how much they know. And then we've also done a fair number of focus groups and discussions. And for those in particular, we've really primarily worked with NewsHour audiences specifically rather than the general US public, the way we did the recruitment was through, in most cases, NewsHour social media, newsletters, Twitter accounts, you name it. So putting out the call directly from the NewsHour and saying, Hey, do you like our stuff? Will you give us a little bit of access to how you think about it?
Laura Santhanam
Totally. And it's been, I feel like just a fruitful partnership. In those respects, it's really sort of inviting a kind of, like experimentation in real time, right, like you will, whether we're looking at, you know, we represented this bar chart and this play with this color scheme. And then hearing from the focus groups and hearing from members of the public, how are they then interpreting the data? Are we sticking the landing on what we intended to represent, or in just sort of, in totally unintended ways, is our color scheme, sort of implicitly, because it becomes implicitly confusing, because maybe it sort of translates to the reader in a way that we hadn't even intended, right. So, that sort of back and forth has been really helpful over the years, both in terms of sharpening not only our sort of like day to day like, this is how we should, you know, present these data, but also like thinking more strategically, and also, how is this going to be consumed? How are these data going to be consumed? How are these numbers going? Are we going to get the message across that we're hoping to share with our audience so that they better understand why this particular poll number is important or at this point in time? Or you know, why this story about the economy needs to be like, at the front of our homepage? And yeah, so it's been really helpful.
John Bailer
I think it's really cool that you're doing this experimentally, that you kind of are probing the different options that you have for for exploring this information and trying to figure out if the consumers are hearing what you hope might be conveyed by the message, as you think about some of the work that you've done, and some of the studies that you've conducted? Is there something that sort of just jumped out as a really surprising result? Something that kind of ran counter to your intuition? I mean, so I'm just always curious about the unusual, what kind of really runs against what I expect?
Jena Barchas-Lichtenstein
I'm curious what Laura's are. One of mine comes from the experimental studies, and that one was, it's widely assumed that nobody likes uncertainty. And one of the things that we found was that showing visually, showing confidence intervals or margin of error rate, which is both speak for a confidence interval, but showing it visually increases trust in the news organization, and decreases trust in the precise number, the point estimate, which is great, actually, that's exactly what you want. They trust you more, they trust the specific number less when that was unexpected. The other one, and as a mixed methods researcher, I want to point to the focus groups more exploratory open ended work to something we were not looking for, we're not asking about but that came out was red and blue are terrible contrast of colors in the sense that everyone thinks they're about politics, right? Because in the US, it is so ingrained for the last, since the 2000 election, that red is Republican and blue is Democratic, that if you use red and blue contrastively for anything else, people assume there is a political subtext. And that was something that just came out, we showed people figures about COVID. And the vaccinations were one color and the deaths were another and they thought that was political. We showed them numbers about something else. And it was rural and urban. And they thought that was political. And it just, it came up spontaneously in so many different contexts that we went, maybe we should only use these colors for politics. And I, for one, was just surprised at how many times it came up independently with different groups of people.
John Bailer
Right before you start the next response, I gotta say, it's not fair to say nobody likes uncertainty. Jena has job security. I mean, for heaven's sake, let's not go there.
Unknown Speaker
Right. But as a statistician, you also read me too, I'm with you.
Laura Santhanam
I just want to echo what Jena was saying. And, you know, I mean, I guess, thinking about data visualization and the power of colors, we had always kind of had like, the long standing, you know, well, if you're talking about death, avoid red, you know, that's kind of grim. And not like, way to macabre for a bar chart. But, you know, I felt like this was a real great eye opening moment for us. We now generally steer clear of red and blue. I mean, like, in part, it was, yeah, there were all kinds of reasons why we had been doing it. But we were presented with this evidence that it was ultimately counterproductive, which being put in front of our audience members were like, well, of course, we're going to do something else. Because that's the last thing we want to do is inadvertently confuse people. And then they walk away, you know, more sort of befuddled than they were before they sat in front of our chart. That's the last thing we want. So it was really helpful.
Rosemary Pennington
Yeah, you're listening to Stats and Stories. And today, we're talking about a research partnership between Knology and PBS NewsHour focused on quantitative literacy and the news. You highlighted something in this sort of discussion like, the colors that I think are really interesting, but I wonder, what are some other challenges that you have faced, and particularly you are facing as you're trying to figure out how to communicate, you know, numerical information? Well, and maybe are there other things that have come out of this research partnership that have helped?
Laura Santhanam
Totally I mean, earlier, Jena sort of touched on just sort of relaying uncertainty, right? In how, you know, having precise numbers, whether it's in the body of a story, or the headline of the story at caption, how certainly a data graphic can have precise, leaning too much on precise numbers can leave people with perhaps, an ultimately not totally correct understanding of what's going on. So I feel like, you know, that's that sort of broad, like, sort of truth has definitely found a place in, the way that we now think about headlines again, yeah, we're putting out one of our polls that we do, like, pretty regularly, when we're thinking about poll headlines, where we're not, putting an exact overall percent of US adults who say they think x about the Supreme Court or whatever, we'll lean more on, about half, roughly a third, having sort of hedging language, because you're not going to be able to say, plus or minus 3.5 percentage points in a headline, but you can nod to that and then, working with graphics that have those sort of shaded, we call them in house like a shaded bar chart, but it's sort of like it visually depicts the margin of error above and below the percent for a given answer to sort of further support that understanding that this is our best estimate in this snapshot of public opinion on these questions. Right. So it's definitely helped us think more with greater statistical literacy for sure. When we're crafting headlines, because those two are so important for reasons that may seem obvious, but you know, I'll go ahead and state them. You know, for many people, that's the first and last time they may see that story. So we definitely want them to have an accurate understanding of what that story is about. And we don't want them to walk away and think like, oh, 17% of Americans think blah, and you know, that the Alpha and Omega have their understanding of whatever that issue is, because that may be not altogether accurate, right?
Jena Barchas-Lichtenstein
I'm reminded of something you said, Laura, maybe even in the kickoff meeting for the grant, where you said, and forgive me, I'm sure I'm misquoting you, because it's been four years, but you said, I want us to come out of this grant with everybody convinced that those 4,000 people are not interesting, except in what they can tell us about everyone. And fundamentally, the point estimate only tells us about those 4,000 people who care.
Laura Santhanam
Right, right. Okay, that sounds about right. Me from four years ago, thanks.
John Bailer
You know, you're talking about just sort of this exposure, this story might be the only time they see this number, I've often thought about when people are taking a stat at a stat class, it may be the only stat class they ever see that, that in some sense, you know, how does someone build upon maybe their only exposure, and from the perspective of kind of exposition, you want to make sure there's a good base, but the importance of journalism, on building and expanding that seems like there's a role. So I see in what you're doing that literacy, you know, you've been involved with numeracy efforts. So you have to think about functional relationships. If you want to talk about, you know, this kind of saving for retirement, you have to talk about, you know, magnitude, if you're talking about parts per billion or trillion, or billions and trillions of debt. So you have all of these challenges, the numeracy challenges of conveying this. Like you, we had a guest many, many episodes ago that talked about numbers as plot elements, which resonated with me. And when you just said earlier, Laura, about the power of context, that seems like these are two elements of putting statistics in this supportive role for the story that you tell, so I'm rambling here because my questions are trying to percolate to the surface. But I find that you know, you've got this guidebook now that you've put out, you're now trying to communicate storytelling with numbers. What are some of the things that you hope this guide book will achieve? What's the purpose and role of this guidebook?
Laura Santhanam
I mean, I guess, one thing that I hope I mean, we all hope, it is a drone that has been beaten far too often, but still remains true is that, you know, we hear about newsrooms, we hear about quarterly layoffs, you know, we hear about like these, these places, where, which their duty is to be the eyes and ears and just be the watchdogs for society. And they're being further and further depleted, right. And yet, they're tasked with, again, being the watchdogs of society, and their job is so incredibly important. And I feel like throughout the formation of this guidebook, which we hope is helpful for people, you know, just thinking about people who are, maybe they're the one reporter in their newsroom, maybe they're just starting out with their career, and they're already overwhelmed. Or maybe they're, you know, just just trying to do their best in really complicated times. And we're all human, we don't always get it right. But we can try every day, right? And this guidebook, I feel, is sort of like our best effort to, you know, either if someone finds themselves suddenly on a new beat that they never reported on board, there's suddenly the healthcare reporter, the bit local econ business reporter, and probably wear five other hats, if they're suddenly plunged into doing that, like this guide book, I feel is a great sort of foundational understanding of how to tell those kinds of stories. You know, and the important thing is so that they can then relay accurate, well founded information to their audiences so that their broader audience, the broader public has a solid understanding of what's going on. And, you know, whether it's about climate, whether it's about health, again, economy, you know, if it's an election year what these polls mean, who is up ahead or what does this even mean? Is this significant? You know, so that's sort of what I felt was the overall marching order for this for this project.
Rosemary Pennington
Jena, how did you get interested in studying this issue, particularly as it relates to news media?
Jena Barchas-Lichtenstein
One of the through lines of my career has been questions of uncertainty and expertise and authority. And I'm a little bit of a weirdo in the sense that I am the rare person who mostly studies numbers, qualitatively. But I think that's really important. And one of the things that research has shown for a while is that getting back to sort of our earlier conversation about certainty, right, people tend to think that experts shouldn't be certain, even though experts actually hedge way more than people who know just enough to be dangerous. So that's one piece of it that, for me, has sort of informed this question. I've been for a really long time just interested in how people become informed and learn through media and sort of what media and news in particular can do to support that. And there's also a question here. I don't have hard data on this. But I would certainly say anecdotally, from reading a lot of news through this project, and to some extent, from some of the landscaping work we did, there's this ironic problem where the better the journalist understands the statistics, the harder it is for them to explain it in the sense that they don't always realize what other people do and don't know at the point at which they're really an expert in something. I see this a lot in kind of business and economic reporting, where the problem is not at all the journalists literacy that journalists are extremely literate, but they're so literate, that they forgotten that not everyone knows how this thing is calculated what it does, and doesn't mean, what it actually is just the very fact that any and all official statistics are, by their very nature estimates, and they can't be perfectly precise. And they know that too well. And so there's a lot of hand wringing about like, oh, journalists can't do math. And I think that's wrong. I think, if anything, the problem is often that journalists are quite good at math, even if they think they're not, you're trying to kind of cram a lot of information into very little space. And there's a real temptation not to pack it.
John Bailer
You know, I love that you had this partnership with research and journalism, and I also really appreciate this mixed method approach that you're taking there, just because you're exploring different dimensions with different tools and techniques. So I just want to say that that's really cool. And that's really awesome. And it's something I, you know, we're kind of this strange partnership that we have in our studio is something that reflects kind of this interest in trying to define these connections. What's, you know, I guess, one thing I would be interested in is, you know, maybe Laura, what's something that you've learned from Jena, and Jena, within that, I'll change it, what are some of the the insights maybe that you've learned from this collaboration with journalists?
Laura Santhanam
Sure. I mean, I feel like this partnership has been a really wonderful reminder that it is okay to embrace the uncertainty of life. And, boy, we really chose a moment for that launch, with a pandemic and everything. So yeah, it was sort of a, it was a very regular reminder that, you know, it felt like not only embracing it, but being transparent about that, both internally in conversations with editors, through story pitches, but most importantly with our audience as well, so that they understand this is our best understanding of this moment, whatever it is that we're talking about, and these are the things that we don't know, this is what we can say at this point in time. And this is what remains shrouded in, in questions. Right. And these are the ways that people are going about trying to answer those questions, right. Yeah, you know, though that's such a powerful takeaway, both as a as a reporter or a storyteller, but also thinking about how those stories come through to our audience and feel they appreciate that that honesty, that transparency,
Jena Barchas-Lichtenstein
The biggest thing that I've learned from working with Laura, so much humility, because I could never do it. She does the speed at which they have to work. Honestly, it's incredible what a good job journalists do. And one of the moments for me that was really eye opening was Laura walking me through the timeline of an opinion poll where you could be drafting questions Monday morning, it's in the field Monday afternoon for 48 hours, you're pulling an all-nighter and it's live at 9am Thursday.
John Bailer
Oh, Mercy.
Jena Barchas-Lichtenstein
So much right? Under, right, really incredible pressure, journalists in general, I mean, to say, and I just have so much humility, because I could never work at that speed. And I really admire it.
Rosemary Pennington
Well, I'm so glad you were both here. And before you leave, I'm looking at your Reporting with Numbers guide right now. And I teach a multimedia journalism class where we do some work with data. And I'm actually going to use this in my class in the fall. So I'm so thrilled that we had you on today, because now I'm going to be better prepped for my class in the fall. So thank you both for being here.
Jena Barchas-Lichtenstein
You know, we're thinking of Reporting with Numbers as a living document. We would love feedback suggestions, if you or your students have thoughts about what is missing, or what you would do differently, like, reach out, we are super excited to hear from anyone.
Laura Santhanam
No, I'm so glad that you brought that up. Because absolutely we don't want this to you know, sort of in any way. The you know, sort of like an edict trouble on high because no, and I feel like we have these sort of conversations like that is just so much more fruitful, and potentially hopefully impactful than just sort of like shoving a document off onto the internet and then walking away forever and pretending, I don't feel like that's that's helping people. I think, the more we have these conversations around what sort of reporting techniques around statistical literacy in numbers is working and perhaps even more meaningful aid, you know, what's not working, right? And how can we address that together? What can we do better? Because, you know, we again, we do see here poll after poll and so many focus groups like so, you know, some of this confusion and any How can what can we do collectively to address hopefully dispel that confusion and bring greater clarity around the numbers that are driving our world.
Rosemary Pennington
Thank you so much for having us. Stats and Stories is a partnership between Miami University’s Departments of Statistics, and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple podcasts, or other places you can find podcasts. If you’d like to share your thoughts on the program send your email to statsandstories@miamioh.edu or check us out at statsandstories.net, and be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.