Arturas Rozenas is an Assistant Professor in the Department of Politics at New York University. He was a National Fellow at Hoover Institution, Stanford. His research focuses on building theoretical models of authoritarian politics and testing them using natural experiments, field experiments, and machine learning tools. At NYU, he teaches courses on comparative politics and advanced statistical methods.
+ Full Transcript
(Background music plays)
Rosemary Pennington: Autocratic or authoritarian leaders work to control just about everything that affects the lives of those they rule, particularly information. Restricting the news and information individuals can access makes them reliant on the state, as they make sense of the world. It can also make them easier to rule. Authoritarianism and information are the focus of this episode of stats and stories, where we explore the statistics behind the stories and the stories behind the statistics. I'm Rosemary Pennington. Stats and Stories is a production of Miami University's departments of statistics and media journalism and film, as well as the American Statistical Association. Joining me in the studio today, our regular panelist John Bailer, Chair of Miami statistics department and Richard Campbell of media journalism and film. Our guest today is Arturas Rozenas, Rozenas is an assistant professor at New York University's Department of Politics. His research focuses on authoritarian states, electoral competition and statistical methodology. He joins us in the studio after traveling to Miami on a visit sponsored by the Havighurst Center for Russian and post-Soviet studies as part of the colloquium series on Russian media strategies at home and abroad. Arturas, thank you so much for being here today!
Arturas Rozenas: Thank you for having me!
Rosemary Pennington: You gave a talk earlier about the way autocrats manipulate economic news. So I'm just going to ask you to start the conversation. Why should anyone care about that?
Arturas Rozenas: Well, I think it's very important question these days because one of the things that political scientists have found when they studied the development of authoritarian regimes, is that modern autocrats, that want to rule today, in the age of mass media and mass communication, are actually very different from what we would imagine standard old school dictators, like Hitler and Stalin. So older guys, they usually tend to rely on things like very heavy ideology, very hard headed propaganda and repression, especially mass repression. And when people start looking into how these political parties have developed recently, what they noticed is that we have a bit of a new breed of autocrats, that are really relying on this idea that look, what we're doing, we are good economic managers. What do people want? They want economic growth and wealth and as long as we're able to deliver it, we are good. People aren’t going to protest on the streets, people are not going to oppose us, right? So then the question becomes, how do you exactly arrive, how do you exactly establish this reputation of being a good economic manager, right? And since a lot of news about the reputational information, about the reputation, comes from the news, right? Then the question is, if something bad happens, if the economy is not doing as well as it should be, when you are a good economic manager, what do you do? How do you control that information? So I think that is a fundamentally important question, because precisely of the way present autocrats position themselves politically.
Richard Campbell: Could you talk about that study of Channel One where you looked at over 300,000 news reports over I think a 17 year period and talk a little bit about what you think your key finding was and what may have surprised you?
Arturas Rozenas: Right, so there are two sets of findings that we have in that particular research deeper, and when we started the project we really sort of approached it very intuitively and sort of naively with OK, what would be the easiest thing to do, when things are bad on the ground, right? When the economy is not doing well and when you expect that the newspapers, right, the Internet and the television is going to report that the things are not going well, right? And we thought the easiest thing to do would be just to censor the news. That's the definition of being an autocratic government. You have ability to do that, especially on the sources of media that are controlled by the State, right? And so we set out and we devised a method to measure the amount of censorship of economic news on Russian state owned media. A statistical model with the help of which we were, we think we were, able at least to measure certain aspects of censorship and without going into details, now, what we found is that we found basically no tractable evidence of censorship of bad economic news. So then we thought further and we thought okay, when bad news are reported, we started looking into how those things are done. How are the bad news reported, and we started noticing certain patterns and that happened sort of inductively, right? We started seeing that whenever, many times when bad news are reported, usually they are reported in conjunction with certain attributional bad news. So we started seeing that look, bad things happen, right, something bad happens in the stock market or something bad happens with oil prices or things like that, they start attributing those news mostly to external factors like global economic forces, like the actions of foreign governments especially the United States. And so then, after this sort of inductivity, we decided OK, let's actually take the stock and just try to see if that is a part of the more systematic pattern. And again, without going into details, we devised a way to measure the degree of what we call selective attribution. Bad news attributed to external actors, good news attributed to internal actors like Putin and we found that there is a very big symmetry in terms of how good and bad news was attributed. So that would be the two main takeaway points.
John Bailer: We've spent some time meeting with people that are in the official statistics world and there have been many stories, there's fundamental principles of how official statistics should be done. And the potential for government, you know, kind of mismanagement or manipulation is pretty high. One of our guests, Andreas Georgiou was in the Greek system for many years and the result of his work was in conflict with what the government wanted to say and I just find that this idea of kind of the control of the message of what's reported by official statistics seems like a really powerful tool and something that really is a major goal for an organization…for a state to control. So how do you, you know, how do we support kind of the people that are working in this, trying to produce, you know, quality, unbiased you know, independent assessments of economic health of countries?
Arturas Rozenas: The government statistics, you mean specifically government statistics. So I think that is actually one of the most fascinating areas, this is not exactly my area of research of how governments statistics are manipulated, but I have a colleague of mine, Peter Rosendorff who actually has written an entire book about this, of how, when do governments present, first of all statistics transparently, because that's another way of manipulating information, is just not giving information.
John Bailer: And that's one of the rules in the fundamental principles is transparency, so…
Arturas Rozenas: Right, exactly. And then the second thing is, once you provide information, whether you distort that information or not. And so what I think, the way to make this process right, to induce incentives right, to make this process more truthful, is by really trying to calibrate the official statistics against the things that we can observe, right and that cannot be directly manipulated by the government. For example, right, China. Let's take China and there has been some studies done on that. It's not my study, but there have been a few studies done on that. So people at some point started really doubting the official growth G.D.P. growth statistics in China, right? And then they started looking, OK, well how can we actually see, if these numbers are distorted or not? And then they collected data, for example on things that you can measure without the help of Chinese government. Like for example nightlight density which is measured by satellites right and you look at it and you try to see OK, how does nightlight density correlate with economic growth? And you see one pattern in developed democracies, right, and you see a different type of correlation in places like China, right? And using that particular measure, people are able to show, look, there is some political aspect to that. There is more distortion of these official statistics at the time, for example, when there are important political developments inside China. Again, things like you can calibrate against observable data, like the density of roads, or the railways or any other economic, observable economic activity. So if everyone knows that this is going to be done the incentives are going to be there for government, to actually release more correct statistics, because you don't want to be found out that you are lying. That would be my take on it.
John Bailer: Very interesting, thank you.
Richard Campbell: You talked in your study of Channel One, you just spoke a little bit about…you didn't find evidence for direct censorship but you found evidence of what I think you called economic information manipulated through selective attribution and you talked a little bit about how bad news is attributed to external forces. But talk about the good news, economic news. Who claims credit for that?
Arturas Rozenas: Yeah. So numerically speaking, we find that basically 25 percent of all good news, good economic news that have been reported on the Russian T.V. One Channel in the 17 years were actually attributed to Putin. 25 percent. So 1 of 4 events that can be called as a good event, was attributed to Putin. And then you ask the question OK, how many, what percentage of bad economic events that have been reported, have been attributed to Putin? And the number is not exactly 0 but in statistical language, we cannot reject the null hypothesis that the number is actually 0 throughout 17 years.
(Collective laughter)
Arturas Rozenas: And then we also find an interesting thing that there are a lot of negative and bad news that are attributed to Russian officials, but again, fewer bad news attributed to Russian officials than good news. So there is a discrepancy on that dimension as well.
John Bailer: So your focus has been a lot on these economic indicators and outcomes. Are there other types of information that are commonly manipulated in these new regimes?
Arturas Rozenas: Yes. Of course yes. So one of the puzzles for us was precisely that, because when we were going into the study, we knew many anecdotal facts that Russian state controlled media actually censors information. It censors information for example, about the incidents of political protest, and censors all information about corruption of officials, corruption of government members. None of that information is anywhere to be found on the Russian official state controlled media. So we know that they're doing that. In fact censorship on the internet is even more interesting. There are thousands of websites that are officially banned by Russian government and the way you know that there are banned is, as you try to enter those websites from Russia, it is going to give you a message, this website isn't accessible according to the following set of laws that are in operation of government. We think this website is promoting extremists ideas. Usually every dictator says that the ideas which disagree with him are extremist ideas. Putin is not new in regards to that. So when we were going in, we expected to see also, a lot of manipulation or censorship of economic news and we didn't find any, right? Relative to political news, and I think the reason is precisely that people have a lot of information, individual, private information on economic matters, which is quite different from the information they might have about political matters. So I might not know if I live in Province right, somewhere in Russia, I might not know whether there was a protest in St Petersburg or not because I wasn't there. But I know that the prices have increased, right? If I go to grocery store. I know that my neighbors cannot get jobs, right? So it's very difficult for the media source to come out and start manipulating information on the things that people can observe. So again, what I'm emphasizing here is, there's always the ability for citizens to observe some outside information that constrains government's ability to censor it, sort of going back to the earlier conversation with government statistics.
(Background music plays)
Arturas Rozenas: So if we provide more information about potential economic performance in that country, again, it is going to restrict the government’s ability to censor or be believed.
Rosemary Pennington: You're listening to stats and stories and today we're with Arturas Rozenas, who is at Miami University on a visit sponsored by the Havighurst Center for Russian and post-Soviet studies as part of the colloquium series on Russian media strategies at home and abroad. Arturas, so every government manipulates information, right? They want to make sure that…or at least spins information, right? Obama spun information, Jacques Chirac spun information. How, if you are someone who is reading this and reading these news stories and maybe you don't know something like R.T. is a state government news agency, how would you suggest, people sort of navigate stories in a way to help them understand how the information is being manipulated, whether it is a manipulation that is…I don't want to say like, valid, but...there's going to be levels of manipulation. How do you sort through that as a consumer to figure out whether a news outlet coming from Hungary or Russia or Romania is trustworthy?
Arturas Rozenas: That’s an extremely difficult…
(Collective laughter)
Arturas Rozenas: So let’s see…two things. So one thing that you can do here, with ease, that Russian citizens cannot do with the similar ease, is that you can access information from multiple sources with the same amount of ease, right? When it comes to television, that's pretty much monopolized in Russia. There is an opposition T.V., but it is on the Internet, right, and it has a lot of difficulties to reach Russian citizens. So because now, here if even if you receive biased and spinned information, as I do believe you do, right, you can actually sort of check it. You can at least see if the point made in one media source is universally agreed, right? That's one thing. And the second thing, and this is one of the ongoing research projects that we are working on, is sort of idea, trying to understand how…what would be effective ways to raise the literacy, the media literacy. And so one sort of project that we have in mind right now, is actually to use some of our results, of our previous study, right, and try to see, how for example, if we provide people with information about the degree of selective attribution on Russian media, would that actually affect how they evaluate the credibility of the source? With the idea being here is that, well if I tell you that, in the last 17 years Russian state owned T.V. channel has never criticized Putin, if I just tell you that information, you must, we believe, reasonably think that there's something going on. But that is a statement that citizens can calibrate against their own experiences. That's very important. It is very different from saying hey, this is a biased media source or hey, here's an NGO from the United States that says that this is a biased media source. You cannot really verify that statement, right? But if I tell you something that can be intrinsically verified from the consumption of the source itself, the conjecture here, the expectation here is that it could be effective. Now of course I'm telling about this experiment that we're going to do in the future and if people learn about it, it is going to really mess up with the results.
John Bailer: Well we won't tell anyone, none of your study subjects! So I really think that's an interesting idea, the idea of calibration against your own experience, the idea of the intrinsically verifiable outcomes that are measured, particularly with the economic outcomes or you can think about your own life experience or your neighbor’s life experience. But the idea of measuring some more ephemeral components, I mean even the idea of the degree of autocratisis, I don’t know if that's the right word, autocratisis, as you think about kind of measuring the intensity of some of these less tangible outcomes, what are some of the ways that you approach problems like that?
Arturas Rozenas: So in one of our research projects, we were studying the coverage of Ukrainian affairs on the Russian news, after the Crimean annexation and when the two countries essentially entered into a proxy war. And we tried to identify whether in fact, you know if you look at it prima facie, if you observed Russian T.V., you would sort of see, OK, it's biased towards Ukraine for obvious reasons, that there are two countries in war. But how do you establish it systematically that that was actually the case? So one of the things we were doing, we measured, we took every article or every news report on the Russian state owned television, that concerned Ukraine and we evaluated the sentiment of that news report. And what we did, we tried to see if there was a structural break, if there was a very significant break in the average sentiment on how Russian news reports in Ukraine during the fallout between the two countries. And what we actually find is that well there was a decent amount of coverage of Ukrainian affairs on Russian T.V. but it spiked massively during the Euromaidan protests in Ukraine, in Kiev in 2013 and especially after the Crimean annexation. And so we show that there was a big spike, a big break and we also show that the same spike is associated with massively reversed sentiment. So if before that, the coverage of Ukraine was fairly neutral, we found that afterwards it became significantly negative.
Richard Campbell: You were talking today about conspicuously biased media can be persuasive. And you were also talking about, which often reminded me of the discussions we've had in here with statisticians about uncertainty, you were talking about what you could prove and what you can’t prove. So can you talk a little bit about the problem of whether you know, you study whether media is biased or not. How do you know if it's persuasive or not? How do you know how their audiences are responding to bias in media?
Arturas Rozenas: Right. So I think we can…evaluate empirically, certain aspects of the problem but I think certain other aspects of the problem are almost beyond the reach in some sense.
Richard Campbell: You called it unprovable.
Arturas Rozenas: Unprovable, right. So let me make the following distinction. We can ask the question…let's see, the question that we ask in one of our research projects is well, did Ukrainians who watch Russian T.V., did they vote differently than those who did not vote, right? And to answer this question, on surface, seems like simple, we could just compare people who watch T.V. and who don't watch T.V., but that is of course problematic because people decide whether they will watch Russian T.V. based on a preexisting sentiment. So what we do in that project, we explore this natural experiment that happened in 2014 when you bring in government banned Russian cable television inside Ukrainian territory, but some people who lived in the vicinity of Russia were still able to get the analog T.V. channel that was spilling over from the Russian territory. And the way that the broadcasting…the quality of the broadcasting signal varied, depended a lot of times on sort of natural topographic features, right? So some people were not able to get access to Russian T.V. even if they wanted. So what we did, we exploited this natural experiment and we found that yes, there was an effect, in the sense that people who had access and watched Russian T.V. were more likely to vote for pro-Russian parties. So we think that we're able to show that. What we are not able to show and what we think is fundamentally unprovable, is, whether actually the presence of this Russian T.V. in Ukrainian territory had an effect on the outcome of elections. Because that is a much more compounded quantity, a lot of things enter into winning election as such and so I think that fundamentally it is unprovable because we cannot turn back the history and ask, how would those elections look like had there been no access to Russian T.V. at all. So I think these are two different things, that there was a behavioral effect, effect on behavior of certain segments of population versus the effect on the aggregate political outcomes and therefore I think we should be much more careful when we talk about effects of informational campaigns on election outcomes.
John Bailer: Fascinating! Really interesting. So I'm curious, how did you get involved in this work? I mean what led you to this pursuit, this research?
Arturas Rozenas: So I think the story is actually very simple. I live in New York City and in 2014, I was walking on the Broadway and I saw this big billboard that was advertising Russia Today channel. Russia Today channel just started this big operation, right, and wanted to spread in the United States and they were making this big informational campaign and I thought well this is really, really something new, like you would not see, you know, advertisement of the Soviet news channel during the Cold War. So we're playing a new game, we're entering some new territory and at the same time of course because I was very interested in the fears in the Eastern European region, fears that have to do with Ukraine, with Russia, right? I was following what exactly Russian media is saying, and so I knew that a lot of information is twisted and spinned in a very interesting, and sometimes conspicuous, sometimes in a very subtle way, and I was just interested to see well, how does it work? So the anatomy of the thing, how exactly do they spin information and if they do, whether it is effective or not. So it is really, very accidental in some sense. It just hit me…one picture hit me and I thought OK there's probably a story here.
John Bailer: So let me follow up with what's been the most surprising thing that you've learned in this recent project?
Arturas Rozenas: I think the most surprising thing was actually that we were not able to find evidence, hard evidence, detectable evidence of censorship of bad economic news. That was really something that I did not expect to see. In fact the initial idea behind the stories and stats, the initial idea behind the story in this project was that we decided, look, we have this interesting new way to measure certain aspects of censorship or news media and we're going to write a paper, we're going to write a story about how to do this thing and we're going to illustrate it by showing hey, this is how you can apply to the case of Russia and by the way this is the amount of censorship that we find. And when we didn't find any, we decided to look deeper and understand what is happening. So that was an interesting surprise which I think led us to even more interesting finding in some ways, but yes.
Richard Campbell: I'm going to ask a question I know you can answer which is, do you think that the Russian government knows this? That they don't need to censor economic news or…but they do censor political news. I mean that that finding is there, but they don't need to do, you know, are they thinking about it that hard?
Arturas Rozenas: Oh I think that they would censor the thing they could do it effectively, because I think that the thing is still fundamentally easier on the part of people who actually produce the news. The way I understand how this thing works in Russia is that I don't think the government tells them exactly how to spin things, whether to say something or not to say and how to say things when they say them. What I think is that they create an expectation of how the news should be reported in order for them to be perceived favorably. And that is the work of editors and journalists to figure out how to do that. So my thinking here is from the perspective of the journalists, from the perspective of the editor, if bad news really arrives, the stock market crashes, the price of the national currency starts falling, things that actually did happen in Russia a lot of times, a much easier way would be just not to say anything about it, than to try to think creatively, well, how do I make this now into a good thing, right? So I think in that sense it would be easier for them to do that but I think precisely because they also…they are not only expected not to report anything bad, but they are expected to report in a way that is actually effective, right, that there is this natural constraint on their ability, to under-report bad news.
Rosemary Pennington: Well Arturas, thank you so much. That’s all the time we have for this episode of Stats and stories.
Arturas Rozenas: Thank you!
Rosemary Pennington: Stats and stories is a partnership between Miami University’s departments of statistics and media journalism and film and the American Statistical Association. You can follow us on Twitter, Apple podcast or other places where you can find podcasts. If you'd like to share your thoughts on the program send your e-mail to statsandstories@miamioh.edu. Or check us out at statsandstories.net and be sure to listen for future editions of Stats and Stories where we discuss the statistics behind the stories and the stories behind the statistics.