Dr. Francesca Tripodi is a sociologist and media scholar whose research examines the relationship between social media, political partisanship, and democratic participation, revealing how Google and Wikipedia are manipulated for political gains. She is an assistant professor at the UNC School of Information and Library Science (SILS), a senior faculty researcher with the Center for Information, Technology, and Public Life (CITAP) at the University of North Carolina at Chapel Hill, and an affiliate at the Data & Society Research Institute. In 2019, Dr. Tripodi testified before the U.S. Senate Judiciary Committee on her research, explaining how search processes are gamed to maximize exposure and drive ideologically based queries. Her research has been covered by The Washington Post, The New York Times, The New Yorker, The Columbia Journalism Review, Wired, The Guardian, and The Neiman Journalism Lab.
Episode Description
Researchers around the world are tracking misinformation and disinformation as they move through media ecosystems. The explosion of disinformation in particular is not an accident. In fact, one researcher argues that it has been weaponized by some on the right to erode democracy. That claim is the focus of this episode of Stats and Stories with guest Dr. Francesca Tripodi.
+Full Transcript
Rosemary Pennington
Just a reminder that Stats and Stories is running its data visualization contest to celebrate its 300th episode, you can grab data about the show to analyze and submit your entry at statsandstories.net/contest. Your entry has to be there by June 30.
Researchers around the world are tracking disinformation and misinformation as they move through media ecosystems. The explosion in disinformation in particular is not an accident. Instead, one researcher argues it's been weaponized by some on the right to erode democracy. That's the focus of this episode of Stats and Stories, where we explore the statistics behind the stories. And the stories behind these statistics. I'm Rosemary Pennington. Stats and Stories is a production of Miami University's departments of statistics and media, journalism and film, as well as the American Statistical Association. Joining me is regular panelist John Bailer, emeritus professor of statistics at Miami University. Our guest today is Francesca Tripodi. Tripodi is a sociologist and media scholar whose research examines the relationship between social media political partisanship and democratic participation, revealing how Google and Wikipedia are manipulated for political gain. She's an assistant professor at the UNC School of Information and Library Science, a senior faculty researcher with the Center for Information Technology and public life at the University of North Carolina at Chapel Hill, and an affiliate at the data and Society Research Institute. Her new book, The propagandist playbook, how conservative elites manipulate search and threaten democracy is out now from Yale University Press. Francesca, thank you so much for joining us on the show again,
Francesca Tripodi
This is so fun to come back. Thank you for having me.
Rosemary Pennington
Why are you using the term propaganda in your book and not dis or misinformation, sort of defray what you're doing? And how are those all different?
Francesca Tripodi
Sure. So within the field, there is a differentiation between mis, dis and then also some researchers refer to malinformation, misinformation, disinformation misinformation, it's all about the role of intent. So for misinformation, these are just false claims. Disinformation, are known falsities that are spread with the intention of deceiving and malinformation is information that might be factually true, but has been framed in a way or manipulated in a way so that it is deceiving in its intent. And I opted for this concept of propaganda because I was less interested in the veracity of claims or whether or not these truths could be confirmed or denied. And I was more interested in how political parties unite citizens in a common conversation. So this role of branding of politics, and steering the public into one way of thinking over another?
John Bailer
You know, so you had a couple of different strategies for doing this investigation that you describe, and in your book, one aspect of it is ethnographic observation. And the second part was scraping metadata as an answer to harvesting insight from that. Could you just give us kind of a quick overview of what those two types of data collection strategies mean?
Francesca Tripodi
Yeah, absolutely. So I consider myself a mixed methods researcher, although my sociological background is very much rooted in qualitative methods in ethnography. So ethnography draws from anthropological roots. And it is about embedding yourself into a community or a group so that you can see the culture around you referring to what anthropologists call this thick description. And so effectively, I was able to gain access to the College Republicans group and a women's Republican group. And I spent many, many hours going to their group meetings, as well as any other events that were tangentially related to these groups. And then in addition to participant observation, I was also able to conduct in depth interviews with persons who were directly involved with these groups or, again, who I met through these groups events. And what I think the real strength of ethnography is that allows people to speak for themselves. But something that ethnographers haven't done as much as what I did in this study is really engaged in the media and information environments that those who I had the opportunity to engage with in person were doing when I wasn't there, right. So as an ethnographer, I really can't be there all the time. But what I added to this study and kind of the field of ethnography is a concept I refer to as media immersion. So a lot of the news and information sources that my respondents had indicated were trustworthy sources of news, I was actually not very familiar with. So in addition to my in person ethnography, I also purposely refrained for four months from my traditional sources of news and information. And I replaced those with news and information that my respondents had indicated were trustworthy sources of news. And I felt that doing this was really important because after that four month process, I did a content analysis of the same strokes. So I went and I downloaded the transcripts. And I looked for themes using more traditional content analysis. But before I did that, by allowing myself to really embed myself in that media environment, I had just a much deeper understanding of what were the main concerns that people who I was studying thought were of utmost importance. And so that was the ethnographic side of the study. Also, through the informed consent process, I created a Facebook account that was exclusive to this research project. Now, this was easier for me, because I didn't have any private Facebook account. So I was able to maintain confidentiality a lot easier. But through the informed consent process, I asked if those who I interviewed would let me follow them on Facebook. And so after the interview was over, I was able to see what are the news stories that they liked, or shared or commented on? And then I would read those news stories as well, just to get a better idea of what is resonating and why is it resonating? And why is this considered really valid sources of news and information for persons who had the opportunity to observe. And then the data scraping that you refer to that was like, really fun. So that was after the ethnography had ended, I had the opportunity through data and society, I connected with this amazing data scientist, Leon Yin, who's now at the markup. And he had a script that was able to pull from YouTube, how content creators tagged their content. And tagging is just a word that's basically like search algorithms work on relevancy. And in order for an algorithm to know that the content you're looking for is relevant, it's trying to best match your keyword or your query with those same words and phrases that the content has been tagged within. So it's basically a way for your content to be readable by the algorithm. An algorithm can't read. But it can match, if that makes sense. So the content creators on YouTube will tag their content with a bunch of key words and phrases, so that when you search for something in YouTube, YouTube's algorithm will be like, oh, here, this super matches what you want to see. And with the help of this data scientist, I was able to pull the top 10 progressive and top 10 conservative YouTubers, and this was just based on engagement metrics. So we were trying to find youtubers that had multimillion kind of followers behind them. And we were able to look at what are the key words and phrases that youtubers on the right and youtubers on the left use to engage with their audiences. And effectively through this, I found that conservative YouTubers are just much more savvy when it comes to tagging their content, anticipating their audience's needs, as well as anticipating needs that would be outside of their audience. So they would actually tag a lot of their content, using words and phrases traditionally associated with progressive ideas, like feminism, for example, or social justice or environmentalism. And so by looking at that, I really had a more profound understanding of how well content creators on the right understand the way the internet works much more than most normal people do. Most of us really have very little idea of how information flows. But through this metadata analysis, I was able to see that this ability to connect with audiences, much more purposeful than people might realize.
Rosemary Pennington
There's been a lot of work on algorithms and sort of how algorithms are not these innocuous things. I think sometimes people who are not deeply you know, entrenched In technology or media studies understand them to be. But why search? Why was “searched for” the sort of entryway for you to tour to explore this?
Francesca Tripodi
For me, I think search is a really understudied way that people spread disinformation. So there's been a lot of focus on social media platforms, like, you know, especially, I would say, within the last seven, eight years, there's been a real emphasis on trying to fact check and look at how disinformation or fake news quote unquote, is spread on Twitter or Facebook or Instagram or whatever social media. And what I kind of bring to the table is saying, well, at least in my initial research, I found that people don't necessarily trust what they see on social media. And often people are inclined to fact check what they see on social media on their own. And so people will take these claims and put them into a search engine and kind of trust search as this arbiter of truth or this intermediary between falsities that they might see on social media and other people in their other networks. This is also true offline. If people hear something from a friend and they're like, I don't know if that's true, they're very inclined to go to search engines, and try and validate those claims for themselves. And what I wanted to demonstrate through my research is that there's been a lot of really great work that documents how search and other information systems don't necessarily have consumers best interests, right. Safiya Noble’s "Algorithms of Oppression”, I think, really documents that much better than me. But what I wanted to also kind of shed some light on is the role users play in starting that journey. So we kind of think about Google as this like window into a wider world. And what I document in my book is that it can actually serve much more as a mirror reflecting these biases back to us. Now, that's not to say there aren't these corporate interests that also can elevate some information more than others. And we can talk about that, as well. But a great example I use in my book is just like the color of the sky. Do you want me to get into that?
Rosemary Pennington
I don't know. Yeah, that'd be great.
John Bailer
Well, it's always great here. It's got to be blue in Chapel Hill.
Francesca Tripodi
Sure, sure. Well, it's yeah, it's Carolina blue. Right. Um, so I think what's fascinating is that we think about the color of the sky being blue, as though it's like this universal truth. I mean, it's the color of a crayon, right? There's a blue crayon called Sky, you know, you can't get more true than that. And if you searched, the sky is blue, you very quickly validate those truths. But there's some great sociological or knowledge research that documents how the color of the sky isn't always blue. And, you know, through content analysis like, Homer's Iliad, or the original translations of the Bible, they were describing the sky is red, or the sky is gray. And then you find blue does not really exist in nature, as much as we think it's a highly synthetic color. There's this great podcast, actually, that was showing how blue was initially manufactured out of this rare rock. And that's why the Virgin Mary was always painted blue in Italy, because it was so expensive. It demonstrated, like the most beautiful, like the most beautiful thing we could possibly value, Virgin Mary. And so if you even look at other cultures, where blue is not as widely available, synthetically, talking about the sky being blue, it's just when people ask them, like, What's the color of the sky? That question doesn't even translate. And so I take that notion, and then I push it into Google. So we're a search engine, right? The sky is purple, the sky is gray, the sky is red, or the sky is not blue. And, all these queries will verify these alternative realities. And so I love that example, because it's completely apolitical. But there is this role that users play in driving that reality. And I think a lot of research on filtering information or thinking exclusively on algorithmic accountability, doesn't shed enough light on us to the way that we drive those.
Rosemary Pennington
You're listening to Stats and Stories and today we're talking with Francesca Tripodi about her new book The Propagandists Playbook.
John Bailer
I just want to follow up on that image that you had there of, of search as window versus mirror, I, you know, I don't think I'm going to do another browser search without thinking about that. So I appreciate you causing me discomfort. But, you know, one aspect of what you also talked about in your book, and you mentioned it before when you visited us, was the idea of this IKEA as what you called the IKEA of misinformation, the idea of “discover for yourself and build it for yourself.” And, somehow that really kind of crummy table in bed is great, because you've made it so good. Can you talk a little bit about what kind of what you're thinking in terms of or what you're trying to express? When do you talk about the psyche of misinformation?
Francesca Tripodi
Sure, sure. So what I explained in my book is that even though everyday information seekers might not be aware of how search engines function, that propagandists or conspiracy theorists, or political pundits have a very acute understanding of how information flows. And that you can really activate an audience around a keyword and phrase, and by focusing attention around that keyword and phrase, you can drive search traffic around that phrase. So this concept coincides with what internet researchers at Microsoft Research refer to as a data void. So when little to nothing exists online, it's very easy to create a lot of attention around that keyword or phrase and trick search engine heuristics without having really any kind of programmatic expertise at all. So you effectively create a lot of digital first content, and radio I was finding. And this was also happening on TV or through podcasts around this one key concept, and then driving a lot of questions around it. Well, who is this person? Why do we care about them? Don't trust us? Google it, Google it for yourself, right? And we're even seeing there's been really interesting documentation about how a lot of prominent conservative hosts have now said, don't use Google, use DuckDuckGo. And effectively what this means is that DuckDuckGo doesn't have as much content moderation around information integrity. And so it's very easy. It's easier, right? For problematic content to remain at the top. Because there's no one that I don't know of the same level of information integrity work going on. Through DuckDuckGo. So I mean, DuckDuckGo go Google, they have their own problems. But I think my worst fear is like after attending my talk, people will be like, This is exactly why I use DuckDuckGo. And they say, well, DuckDuckGo is actually easier to exploit, then something like Google because it just has less people using it. And actually DuckDuckGo doesn't create its own index system, they borrow from Bing, and Google anyway. So it's not like you're getting some like fresh, new insights into the world. They're just not necessarily tracking and selling your query. So if you care about data, privacy, DuckDuckGo is fantastic. Good for you. Use DuckDuckGo. But if you're thinking that somehow DuckDuckGo provides you with more quality information or better information, that's not the same thing as data privacy.
Rosemary Pennington
In the book, you talk about this concept of scriptural inference, as you talk about sort of the experiences of these individuals you worked with. Could you explain what that is? And maybe how you tracked it in your research?
Francesca Tripodi
Absolutely. So scriptural inference is what I refer to as a compare and contrast method of focusing on textual interpretations. And I first came to realize the value of this work when I was able to accompany those who I had met at these different groups to their Bible studies. And while we were in a Bible study, we were reading through the Bible we had focused on, you know, one or two lines of biblical text, and we're really digging into how that might apply to our everyday life. And then the person leading this group kind of pivoted and shifted focus, and then pulled out this copy of the tax reform bill, and then encouraged us to do this exact same deep reading of this piece of legislative jargon, quite frankly, right? I mean, I have no idea what happens in most of these legislative bills. They're not really written with public interest in mind. And he basically called on us to say don't trust the media to interpret this document for us. Do you need to dig into this text yourself and identify how its provisions might hurt local farmers or small businesses, more or less than large corporations? And then that basically allowed this to click for me like, oh, one, there isn't this one form of media literacy. So it's easy to say, Oh, we can solve this through media literacy. But I think that implies illiteracy. And what I wanted to do with my book is to say, No, conservatives, voters in this country are highly literate, they are reading a lot. They know a lot. And they are not just coming to their conclusions haphazardly. They have a profound engagement with the media. And the second thing I wanted to show is that even though this really is connected to Protestant engagement with the Bible, that what constitutes sacred text is, is transferable. So we see this a lot in constitutional conservatism. For example, many of the well, you know, the recent overturning of Roe versus Wade, for example, explicitly was saying the concept of privacy wasn't inscribed in the Constitution. And because it's not part of the original Constitution, we can't uphold that. So this notion of transferring a biblical literacy to a constitutional literacy in my ethnographic observations, and then also through media immersion, I saw this same form of media literacy used during Trump's first impeachment hearing, when people were calling on the audiences to quote unquote, read the transcript, and that via this transcript, Trump was exonerated. So even though it's initially tied to a biblical setting, it definitely transcends the power of the pulpit, if you will.
John Bailer
I guess I've never really thought that a tax code or a proposal might be divinely inspired. You know, so that's the idea that it's somehow deemed in the same way that some might view what you thought what we could think of as holy text would be read in the same way. That's remarkable to me.
Francesca Tripodi
And it's not even so much that the text itself is holy. I think what I'm trying to argue in my book is that the problem, the process of inerrancy, is what's holy that the textual translation is true. Right? So well, so a biblical inerrancy assumes a truth to the Bible, not just because it's sacred, but because we trust that text to be true. And we trust that interpretation to be true. And so that that role of trust and inerrancy is what I argue is transferable to these other documents deemed worthy of such trust. It's interesting, that's also why many conservatives felt that a significant piece of legislation that was passed on December 24, and many, many conservative politicians were like, they can't know what's in this. There's no way they read it. There's no way they read this because it was passed on Christmas Eve. And so it was this idea of like, unless you read it in full, you can't know if it's true or not.
John Bailer
I'm really curious about the reaction for some of the groups that you were interacting with. This is part of your inner, your ethnographic study. Have they read your book? And have they responded to it? You know, what kind of feedback have you received?
Francesca Tripodi
Yeah, so I don't know if they've read my book. Unfortunately, I've since moved from those spaces. And part of informed consent, I just didn't feel like it was ethical to keep in touch with people long after my research was done. But when I was at data and society, I released my report searching for alternative facts, which is the kind of primer for my book. And I shared that document with my respondents. And I also shared a lot of the key concepts with my respondents before writing that report. So the idea of scriptural inference and how I came to create that concept was very much agreed upon by the people that I interviewed. They felt that the ideas of what constitutes conservatism, you know, being highly connected to faith and firearms, and a free market and family. These were words from them, you know, again, not from me. Now, whether or not they agree that people they watch are exploiting those values in order to spread false claims. I'm not sure if they would agree or not, but I tried as best as possible in my book to really honor the spaces that I was in. I was reading it hoping that people who disagreed with me would read it, and wanted to be very clear and careful that persons that might read it and identify as conservative would hopefully not take offense to what I was saying. Now, some of the pundits probably won't like it. But I didn't interview them. So that's okay.
Rosemary Pennington
Speaking of pundits, I know Francesca, that you have been doing work on sort of notable entries on Wikipedia and a pundit would be a pretty notable individual. So you have a new paper out. And before we leave, I just wanted to see if you wanted to talk us through sort of what this new Wikipedia article is?
Francesca Tripodi
Yeah, yeah, I can talk briefly about it. Actually, would it be okay, if I kind of connected how I think Wikipedia is connected to this? Well, thank you for asking me about that, Rosemary, because I love Wikipedia. I think a lot about Wikipedia. And I've done a lot of research, looking at how gender and racial inequality maps on to the way that Wikipedia documents these things, right? How do people establish themselves? Right? And what's fascinating to me, is the way search is changing. Okay, so we know this from chat GPT. We know this from knowledge graphs, the way that we look for information is becoming much less exploratory, where we click on the link and read it for ourselves, and must much more answer it for us. And these, these ways of creating knowledge graphs are like answering our question for us, are very connected to open systems like Wikipedia. So a big thing that I like to think about is, how do those absences on Wikipedia reverberate off of Wikipedia? And how might people who are deemed not notable by a pretty small number of people effectively limit engagement with those people on a much wider level? So so? No, I'm totally on a different tangent?
John Bailer
No, no, I, you know, I'm, I think we're both just thinking about the changes that we might expect in terms of how we end up discovering information, or how much of it is being processed for us, as part of the way we request it. So when you were looking at kind of Notability? And what were, you know, entries that are showing up? You know, what, can you give us kind of the– not necessarily Twitter length of a Tweet of the results. But give us a summary of what are sort of the main takeaways from this work?
Francesca Tripodi
Well, it's funny you bring up Twitter because prior to its restructuring, the only way to get a blue checkmark was to establish notability, that you were an authentic person, and nobody cared if you are the authentic person, if you didn't constitute Notability. And one of the ways to establish notability was to have a Wikipedia page. But what I was kind of showing is that these Wikipedia pages are constantly contested. They aren't static forms of information. A summary of my Wikipedia work says that the website has these guidelines in order to ensure equitable assessment of their articles. And what my research finds is that these mechanisms for establishing notability are applied idiosyncratically, and that women and people of color often fall short of what constitutes notability even if they are notable per the guidelines that were created.
Rosemary Pennington
Well, that's all the time we have for this episode of Stats and Stories. Francesca, thank you so much for being here today. Stats and Stories is a partnership between Miami University’s Departments of Statistics, and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple podcasts, or other places you can find podcasts. If you’d like to share your thoughts on the program send your email to statsandstories@miamioh.edu or check us out at statsandstories.net, and be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.