Riley Acton is an Assistant Professor of Economics at Miami University, as well as a Research Affiliate at the Institute of Labor Economics (IZA) and the College Crisis Initiative (C2i). She is an applied microeconomist who specializes in labor economics and the economics of education. Her current work examines the causes and consequences of decisions made by higher education institutions, the effect of local labor market shocks on K-12 and college students, and the impact of school finance policies on educational outcomes.
Episode Description
Last academic year colleges and universities across the US struggled with whether to mandate COVID vaccinations for their students. While colleges often require vaccines, the political controversy surrounding the COVID shots made adopting a vaccine policy a complicated undertaking. But according to one study, it had a profound impact on the national COVID death toll last fall. That's the focus of this episode of Stats and Stories with guest Riley Acton.
+Timestamps
How did you get this figure for deaths? (1:35) How did you get into Economics of education? (2:53) How did you come up with the framework of the study (6:48) How do you control in studies like this? (8:36) How has your story been covered? (10:35) Putting out a working paper (15:02) How do calculate a year of life lost? (17:14) What are the costs to mandates? (20:11) Any new questions related to this work? (22:00)
+Full Transcript
Rosemary Pennington
Last academic year colleges and universities across the US struggled with whether to mandate COVID vaccinations for their students. While colleges often require vaccines, the political controversy surrounding the COVID shots made adopting a vaccine policy a complicated undertaking. But according to one study, it had a profound impact on the national COVID death toll last fall. That's the focus of this episode of stats and stories where we explore the statistics behind the stories and the stories behind the statistics. I'm Rosemary Pennington. Stats and stories is a production of Miami University's Department of Statistics and media, journalism and film, as well as the American Statistical Association. Joining me is regular panelist John Bailer, emeritus professor of statistics at Miami University. Our guest today is Dr. Riley actin. Actin is an assistant professor of economics at Miami University, as well as a research affiliate at the Institute of labor economics, and the college crisis initiative, acting as an applied micro economist who specializes in labor and the economics of education. She's also a co author of a National Bureau of Economic Research working paper, examining the impacts of university COVID vaccine mandates. The study found the mandates lowered COVID deaths in the US by an estimated 5%. Last fall. Riley, thank you so much for joining us today.
Riley Acton
Yeah, great to be here with you all.
Rosemary Pennington
So I'm just going to ask the question, How did you and your co authors arrive at that 5% figure? Because it seems I mean, when I read the paper, I was sort of shocked that that's the number.
Riley Acton
Yeah, so to get to that number, we got to go back a few steps. So what we're looking at in this paper is county level outcomes. So we're not just thinking about colleges and their students or their employees, but we're thinking about the communities that colleges are located within. And what we're doing is we're tracking how county level COVID-19 outcomes, including cases and deaths, we look at hospitalizations a bit as well, how those outcomes changed before and after the start of the fall 2021 semester, and how that change varied based on whether colleges in that county had a vaccine mandate on their campus or not. So we come up with the total number of 5% by looking at how total deaths over essentially the fall semester about a 13 week period, varied on average in counties with COVID-19 mandates throughout that time period. And then we essentially add that up across all of the different counties that had the vaccine mandates to come up with a total number. And we estimate that that total number is about 5% of total COVID, 19 deaths that we saw during last fall.
John Bailer
You know, I really liked the idea of of thinking about the impact of policies like this at universities, because, you know, a lot of times the university may be one of the larger employers in a community. I think, in fact, where we are in our county, the university is the largest employer in the county. So it's it seems like it makes a lot of sense to think about how that might might drive kind of what's happening, and particularly when you have this an influx of a large number of students into into these schools at the start of the start of semesters. I'm just curious, how did how did you get interested in and kind of the economics of education? I mean, we're gonna get back to your paper. I mean, don't trust me, we're not going to let go of it. But just just as a little bit of background and context, how did you first get interested in this? Yeah. So
Riley Acton
in terms of how my interest in economics of education overall evolved, if I think way, way back, I was fascinated by classmates decisions of college and where they went to college, and are they going to college? And what were they going to study and I always just found all of that really interesting, I was that kid who was like making spreadsheets about where to go to college, and all of that sort of stuff. And then at the same time is that I was always very good at math in elementary and high school. And so I knew I wanted to do something in college that that related to math, but pure math just on its own for math sake didn't super appealed to me, because I was also really interested in in government and social science and politics and policy. And so in college, I found economics where those two things can kind of really come together. But I always kind of kept this interested in education, both higher education and K 12. So when I did decide to go do an economics PhD, sought out departments and programs where they had folks working on the economics of education, ended up at Michigan State, which has a group of faculty studying the economics of Education has a close relationship with the College of Education at MSU, which is a very well renowned College of Education and kind of went from there down this path of studying education in in all sorts of forms about how people make decisions about education, how institutions make decision. Jennsen what those effects are on all of us.
Rosemary Pennington
So you're already set up to do this kind of work. And I kind of wonder how you and your co authors decided to look at specifically the impact on the death toll because I under like, I could imagine thinking about prevalence in a county in a community. But you know, this this focus on the death toll and trying to figure out how these policies impacted that it's not something that I would have considered. So I wonder how you guys decided that was what you wanted to look at further? Yeah. So
Riley Acton
I would say we wanted to look at everything, right. We always started research projects and things like big ideas of like, how do vaccine mandates impact people? Right, there's this really big question. And then, of course, in the work I do, we're always sort of limited or directed by what sort of data is there available on different outcomes. And so the CDC has collected really good, weekly, even daily, at times, information at small geographic levels, like the county on cases and on deaths, those are actually the two outcomes that are probably the best covered by the CDC data. hospitalizations are also there, though, they can get a little wonky, depending on how many hospitals are actually in a county or people from that county being treated at their local hospital or hospital in another county. So there's a little more noise in that data there. So we knew we want to look at cases. And then we also had the data on deaths. So we decided we should also also go ahead and look at that. Initially, I don't think we necessarily thought we would see an effect on deaths since we knew that the population that was getting vaccinated was at a low likelihood of dying from COVID. But since we did see those effects, we were able to think about how do college students interact with the general population? And what sort of spillover effects are there between these groups?
John Bailer
This kind of study is really, it's really hard to think about setting up I mean, because you know, it's not a lot of times when we think about comparing what happens as a result of a policy or a change. It's often in the context of a design experiment, where you can randomly assign conditions, for example, is sort of the gold standard when when you're doing clinical trials. So you have an observational study. So how do you think about it? How is this how did you frame the idea of a comparison of institutions with these policies to institutions without and doing this comparison? Yeah,
Riley Acton
that's a great question. And I think this is actually an area where economists have developed a pretty robust toolbox and framework for thinking about these questions, because we often can't randomize what we would like to study, a classic example of economics of education. You can't just randomly assign some kids to go to school and some kids not to go to school and you know, see how they turn out as adults. Right. There's lots of experiments we can do. And COVID vaccine policies ended up being one right there was no oversight, saying you colleges are mandating vaccines, you other colleges are not at random. So the way that we go about thinking about this study is really trying to come up with a good what we call counterfactual. So what do we think would have happened at colleges with vaccine mandates, had those mandates not been in place? And so what we're really looking at is, before the start of semester, after the start of the semester, how were these COVID outcomes changing in places with and without vaccine mandates? And in the period before students came back? Does it look like they were changing in a similar way? Because if they were changing in a similar way, before students came back and before the mandates, we think would have an effect that gives us more confidence that had the mandates not been in place, they also would have continued to change in a similar way.
John Bailer
Yeah, I thought that was pretty neat in the models that you were looking at this kind of plus or minus 11 weeks. Yeah, sort of the start of that term being kind of this origin. And then you did the look back to see if what's happening prior to that in the county, and then the look ahead to see after they return what what has occurred. So when you're doing this counterfactual, you're sort of setting up this comparison. It's, it's kind of comparing to that county as if they had not had these kinds of policy and play. But there's so many ways that counties differ. You know, they have different different public health policies, they have different other characteristics. They have different population densities, they have different, you know, you name it. So what are what are some of the ways that you could try to you think about kind of the differences in counties and controlling for such differences?
Riley Acton
Yeah, so that's been kind of a thorn in our side throughout this entire paper, as we know that places are different. And in particular, we can just see in the data that places where the college has had vaccine mandates, places where colleges did not have vaccine mandates, are different counties, right. The counties where there are vaccine mandates tend to be a bit larger, a bit denser, important for the vaccine update, they tend to lean a little more liberal as opposed to conservative so we know that these are just different counties. So one thing we do is we're really relying on changes within a county over time, right? So it's not kind of apples to orange because we're comparing apples over time, you might say. And the other thing we do is we try to take really seriously these differences and control for them. So we allow for really flexible trends over time that we control for. And we allow those trends to vary based on the region of the country, a county is in based on political leanings, proxied, by their their vote share in the 2016 election, as well as their baseline vaccination rates. So in July, how many people in this county were getting vaccinated on their own? And so we let these trends soak up some of that existing variation in our models?
Rosemary Pennington
Rarely, so I were talking to you, because I saw people on Twitter, sharing the paper when it came out and talking about it. And I know it got a lot of coverage. And I wonder what it's been like to have this covered in the media and sort of maybe what that experience has been like for you. And if there are, there are maybe moments where you think the journalist got the story wrong, or, you know, could just talk us through that kind of experience. Yeah,
Riley Acton
so so it's been exciting, a little nerve wracking, given the controversial nature of this policy, but but exciting at the same time. So to back up a little bit, we released this paper through the National Bureau of Economic Research, where two of my co authors on this paper are Research Affiliates. What the NBR does, are part of what they do is they have research affiliates who are really well known established researchers in the field, who then can essentially release preprints of papers before they've gone through the entire peer review process and get them out to the research community as well as to ideally journalists and policymakers on a quicker timeline. And so that's where this paper is, this paper is sitting at a journal right now, hopefully, getting an r&r Sometime sometime soon, if our fingers are crossed. So we have it out. And that makes it a little scary to write because we haven't gone through the entire peer review process. That said, it's been really exciting to see journalists and university leaders and people like that interested in this paper. For the most part, the journalists, I've spoken to have done a great job understanding the paper kind of understanding the punch line of our results, and thankfully have not had any wild misinterpretation out there that
Rosemary Pennington
you're listening to stats and stories. And today, we're talking about the impact of COVID vaccine mandates, with Miami University's Riley Acton.
John Bailer
So just to follow up on on that, that how it's being reported and how it's being covered. I mean, one of the things that we've talked a lot about on this podcast, as is this idea of kind of the uncertainty that are associated with some of the results. And a lot of times, what we'll see is this focus and emphasis on the point estimate that comes out of this. So you know, when looking at at the report, you were talking about the the the reducing new COVID deaths by, say 5.4 per 100,000 population. But then as you know, within your paper, there's also the standard error. And so you're looking at a standard error of about 2.6. So, you know, I'm curious as you think about framing the story here, you know, there's the point estimate of this, but if you do the plus or minus some multiple of 2.6, you'd say I, you know, it's reducing deaths by anywhere between point two and 10.8 per 100,000. residents. So can you talk a little bit about about how, how do you think about conveying in fact that there is some uncertainty associated with these types of numbers? Yeah,
Riley Acton
I think that's a great point, because of this, the five and then the 5%, that we kind of do the back to the envelope calculation on that's really been what's picked up by headlines. And we wavered, actually a little bit on do we put this 5% of total deaths in there, because we do know that this is a noisy estimate. It's not extremely precise. One of the things we do in the paper that I think is actually pretty interesting, and we're very happy with how it came about, is thinking about this competence interval, in terms of there being different effects in different types of communities, right. So this is an average effect that has some noise to it. But in the paper, we then break that effect down into kind of different types of counties, and can get a little more precise in subgroups of counties. And one of the things we see when we do that is that the effects are largest and slightly more precise, in places where as we mentioned at the beginning, colleges are really large relative to their county. So when you have a college population that is really coming in as a large group of students with a less large county, we see larger effects of vaccine policy there and can have a little more precision. We also see that when we tried to look at counties where we would have would have expected students to be more likely or less likely to be vaccinated had there not been a mandate, and we see that places where we might expect lower vaccination rates without a mandate ended up with having higher, more pronounced effects on the out right in those counties.
Rosemary Pennington
Hearing you talk about all of the things that you guys thought about, as you were putting this research into place, I do want to go back to the idea that this is a this is a working paper that was out and was was, you know, being circulated. And now it's under review somewhere. I wonder what that feels like, if you could talk through again, sort of the decision to sort of, I know, you have authors who are affiliated with, with the Bureau. So I wonder, you know, is that a process that you there's, there's push right, in research communities to do more of this, like pre release of stuff? And I kind of wonder now that you've been through it kind of what your thoughts are about kind of having this work out there before it's gone through sort of the more formal channels that we sort of expect?
Riley Acton
Yeah, that's a great question. So I will say in economics, there's really a culture of putting out papers before they've gone through the full publication process, in part because the publication process and lags and economics, for whatever reason, tend to be quite long. So it can be years and years between when a paper is maybe first draft and a circulating draft among colleagues. And when it ultimately ends up in a journal in somebody's office. So economics has developed this culture, in part through the NBR, of releasing Working Papers. Now, with this paper in particular, we knew we wanted to get it out there both because we thought it was really policy relevant, particularly before semesters were starting and colleges were thinking about, do we keep a vaccine mandate? Do we not so we wanted to get it out during the summer, we also wanted to get it out to more or less stake our claim that, hey, we're researchers that are working on this paper so that we could be aware if anyone else was trying to answer the same question and in similar or different ways than us. That said, we did kind of push off getting it out until we had flushed out a lot of these different heterogeneity, analysis, robustness checks, these different things I've been talking about, because we did want to make sure that we were pretty confident in the numbers we had in our tables, before we put it out, because we did have an inkling and a hope that it would get picked up by journalists, and we knew that they might latch on to particular numbers there.
John Bailer
So you also included a couple of other things, endpoints like years of potential life loss, and also value of statistical life. And, and part of my interest in this, I have done some collaboration with people in Occupational Safety and Health over the years. And in particular, when you look at things like, you know, occupational, fatal injuries, it tends to be relatively young workers. You know, the median age, if I'm remembering right was about 35. So So you know, when you think about the impact of these diseases, it's not like a cancer that's occurring a year or two earlier, it's like someone who's dying much, you know, very early in life. So So you you reported, like the average years of life loss from COVID, to be between nine and 14 and a half years, which was from other your reporting other researchers. But but the value of statistical life at 62 years, was $350,000, which I thought was a, you know, that's, that's actually this is this is hitting close to home.
Rosemary Pennington
This is what I was waiting for.
John Bailer
But but you know, as I think about this, it's an interesting question about how these kinds of valuations, you know, whether it's, it's the monetary component, or you're, I mean, essentially, what you're doing is waiting a year of life loss with some economic component, could could you just give a little bit of backfill to talk about, you know, how is something like a year of life loss calculated? And also, then how is that a value kind of attached to that? What are some of the thoughts that economic economists Yeah,
Riley Acton
so I will say, This is not my area of value is to just go live estimates. I mean, at a very simple level, you can think about labor market earnings, right are going to be part of that, depending on the years of life lost, you can think about consumption, right? So how many dollars would this person have been putting in to an economy over time from age 62, or 65? Till whenever we think they may have passed away naturally. I think that is maybe a helpful way of thinking about this in this context, when we're thinking about counties, right and thinking about how is this impact going to be felt by counties, not just right now, but in the years and years to come? If you have lost people in your community, of course, there's a huge psychic and social toll that that takes, but it also is removing people from your community who otherwise would have been paying taxes, going to local establishments, contributing in so many different ways to the life and the livelihoods of other folks in that community.
Rosemary Pennington
And it really makes you think about the impact of this on particularly Smalley not less densely populated counties where, you know, a county in Ohio like Cuyahoga or Franklin or even Hamilton, which is our neighboring county, can absorb that loss in a way that Scioto County where I'm from, which is you know, Appalachia and middle of the state pretty economically challenged It would just it would be a different sort of cost to them to have lost these individuals, in addition to sort of the, the psychic and the social. So I thought that was a really interesting way of thinking about that.
John Bailer
Yeah, and I liked the idea that, you know, as I was, I was thinking about this coming out as sort of a preprint, you know, an archive kind of preprint that's coming out there, you're just adding to the conversation, there's a decision that needs to be made, you know, so if you're gonna make this decision about what this is potential benefits of this, you know, whether it's known with certainty is not, is not maybe critical in the story, as much as saying that there is some, here are ways to think about the ways this might benefit. And, and there's sort of this idea of these other areas that you've reported, including, you know, the years of life lost, or the value of statistical life, that's part of this calculation is additional kind of benefits. So then you have to say, Well, what, what are the costs of having this in place? mean? So you have this, you know, you've gone through and you've talked about kind of the benefits of having such mandates in play? Or there? Are there costs?
Riley Acton
Yeah, that's a great question. And that's something we've remained a bit agnostic on in the paper, since we're not trying to directly measure the costs. But the argument that we try to make is, here are the benefits. Now, you institutional decision makers need to think about what are the costs to your community to mandate this vaccine? And those tend to be hard to measure? Right? What is the psychic cost of making someone who doesn't want to get a vaccine, get that vaccine? Right? I don't know what the dollar amount to put on that is, what are the logistical costs, right, there's probably some sort of administrative costs of making everyone report their vaccine card uploaded into a system, maybe there's it maybe their staff, maybe there are various logistics involved in doing that, that's probably going to vary campus to campus, situation to situation. So we're hoping that by giving some information on the benefits, different institutions can then think about the costs to their particular situation, compare these things, and just be aware that this does have benefits. At least it has seemed to have benefits on average, last fall.
Rosemary Pennington
I wonder what kind of follow ups you and or your co authors are thinking about? I mean, I know this is under review right now, this paper, but are there other displays and other questions for you that you're going to explore now?
Riley Acton
Yeah, so there, there are so many questions related to COVID-19 and higher ed. So this whole team of researchers were all people who think about economics of education and higher education, and I think all of us have experienced during the pandemic, some changes in research questions and research agendas, because this has obviously been just a huge disruption to education systems as we know them. One thing I am interested in this work that I don't know if I will necessarily be the one to write the paper, but hopefully someone will is how these sorts of vaccine mandates changed, what colleges different students were interested in, right? So did some students decide not to go to a particular institution? Because it did or did not have a mandate there? Does that change something about the demographic composition of the student body and kind of how did those effects perpetuate for years and years to come? Because we know that the social networks you're exposed to in college can be really important, you know, for many years down down the road.
John Bailer
So I'm gonna, I'm gonna channel my inner rosemary. Now I got married Rosemary really scared. You know, what you were just describing, really suggested that kind of to complement the quantitative work that you've done. Are there qualitative components to doing this? I you know, I can Well, imagine that you that some kind of qualitative investigation of kind of student decision processes might feed into that, then I do. Okay, Rose.
Riley Acton
Yeah, I mean, I would love again, I, I'm on the quantitative side, that is kind of my bread and butter that I worked on. But I think there's so much room for qualitative work within the COVID 19 pandemic of understanding all of these things. We're talking about right thinking about how did students perceive these mandates? What sort of cost did it feel like to them to have to report this information to institutions? Did it affect whether and where they went to college? How did communities feel about having these mandates in place? How have communities felt about the loss of people in their communities? All of this is really important, I think, for qualitative research and for journalism.
Rosemary Pennington
That's a lot of time we have for this episode of stats and stories. Riley, thank you so much for being here today.
Riley Acton
Yeah, thank you so much for having me.
Rosemary Pennington
Stats and Stories is a partnership between Miami University’s Departments of Statistics, and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple podcasts, or other places you can find podcasts. If you’d like to share your thoughts on the program send your email to statsandstories@miamioh.edu or check us out at statsandstories.net, and be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.