Robert Santos is vice president & chief methodologist at the Urban Institute as well as President-Elect of American Statistical Association. He has over 40 years of experience designing research and evaluation studies as well as sample surveys. His expertise includes quantitative and qualitative research design, sampling, survey operations, and statistical analysis; specialty areas include Hispanics, blacks, undocumented immigrants, and other disadvantaged populations.
Episode Description
The US Census Bureau is conducting its annual count of the American population this year. Concerns have emerged about this particular census and these have included potential impact of a citizenship question. The shortening of the window for the count, the deadline for reports for reapportionment. All of these concerns might translate into miscounts that impact allocation of federal funds of representation in our legislative branch. The current status of the 2020 Census is the focus of this episode of Stats and Stories with guest Rob Santos.
+Full Transcript
John Bailer: The US Census Bureau is conducting its annual count of the American population this year. Concerns have emerged about this particular census and these have included potential impact of a citizenship question. The shortening of the window for the count, the deadline for reports for reapportionment. All of these concerns might translate into miscounts that impact allocation of federal funds of representation in our legislative branch. The current status of the 2020 Census is the focus of this episode of Stats and Stories where we explore the statistics behind the stories and the stories behind the statistics. I’m John Bailer. Stats and Stories is a production of Miami University’s Departments of Statistics and Media, Journalism and Film, as well as the American Statistical Association. Joining me is regular panelist Richard Campbell, former Chair of Miami’s Media, Journalism and Film Department. Rosemary Pennington is away. Our guest today is Rob Santos. Santos is president elect of the American Statistical Association and the Vice President and Chief methodologist at the Urban Institute, a nonprofit research organization focused on issues of public policy. Thank you so much for being here Rob.
Rob Santos: Well, thank you for the invitation, it’s an honor.
Bailer: Rob I’d like you to expand a little bit on a quote you recently had which was a nation is best served by securely and effectively using additional time for US Census Bureau career staff to implement the decennial Census as accurately, completely and fairly as possible. And particularly I’m interested in the idea of when you say accurate, complete and fair, can you help unpack that for us?
Santos: Yeah certainly. The notion is that you can actually have a census that counts everyone but is not fair. The fairness comes when you think about the folks in two different buckets, households in two different buckets have participated in the Census. The households that are typically those that immediately and willingly respond to a request from the Census Bureau to its decennial census time, we need to do our civic duty and etcetera, and then the historically hard to count folks in households where they either have some hesitation they have philosophical issues or they are in situations so dire that they don’t have time to think about completing the form, they may not have internet access and so forth and it turns out that things like the citizenship issue that came up and the commerce of state at the white house it creates citizenship counts, whether or not there was a citizenship question on the Census as well as other types of aspects like there’s a pretty toxic immigrant policy in the federal nation there are things going on that make it so that immigrant households are feeling pretty unwelcome whether they are documented or not, and those types of things instill fear and so if you take the combination of historically hard to count to households in addition to folks who are fearful and folks who are in dire circumstances, either at risk of being thrown out of their home because they can’t pay rent or because of COVID they have sick individuals that they are caring for it’s a family without a job and they don’t know where they’re going to get their next meal those combined actually make the historically hard to count households even that much more harder to count , and so the fairness comes in when you take a look at for example at the current self-response rates of the Census. The Census Bureau right now knows this there is a 66.7% self-response rate as of October 7th and actually if you just think about in 2010 that final rate, at the end of the day was 66.5, so they’re doing better in the sense it will last a decennial census, however that masks the fairness story with regard to the populations in the United States. If you take a state like New Jersey. New Jersey has a 69% self-response rate right now but look at Newark. Newark has a 50% self-response rate. That’s the self-response rates and the rates that households responded by themselves, whether by internet telephone or mail. And why does that matter? because the census is now mostly 99% completion rate, it matters because there’s a very clear connection between the Census Bureau’s own research between the rate of self-response and the risk of being undercounted as a population. And yeah sure the Census Bureau at the end of the day, whether it was September 30th which has now passed, or October 31st will declare victory with a 99.9% completion of all states. What all that means is that they brought to final disposition and their counting activities and those counting activities will rely heavily on going to neighbors and asking neighbors who lives at the house next door or across the street or landlords who lives in these apartments that never responded and you’ll end up getting guesses and sometimes the guesses are only someone’s perception of who live in the household and guess what with COVID, COVID, if you recall correctly who caused a really significant change in household compositions where people lived because when the initial lock down occurred it was a scramble. I man people from New York, New York City got out of there faster than anybody’s business right? And then they stayed away. I mean I, myself, have a nephew that left almost immediately and is still in the state of Texas.
Campbell: Rob, you wrote about postponing, you wrote on your blog back in March the idea of postponing the Census when COVID hit. Is that-
Santos: I was early about postponing by the way.
Campbell: So um I guess I just want to hear more about the effect of COVID, I’m just imagining going to households and trying to get people to answer the door even with masked people and trying to get a response.
Santos: That is definitely an issue and I actually am a member of one of the Census Bureau’s Unity Facebook pages that features that features individual film members that are out there and it’s pretty clear that the wearing of the mask had a negative impact on the willingness of answering the door. There were countless stories of people saying stay away, or that the field person would knock and knock and knock with the lights on in the house and see people not answer the door. And you know you don’t blame them I mean we have, people’s lives are at stake, and we are only learning now what the most effective ways to social distance are, as well as the efficacy of wearing masks, and that information is not uniformly held by the public. So you have individuals with their preconceived notions and I don’t care who comes I’m not answering the door and that’s especially true with households among the hard to count that include people in communities of color, people who are renters and who are at risk of being thrown out because they haven’t paid rent. You know it could be somebody with a mask and a badge on but they’re not going to look to see what the badge says it could be the sheriff with an eviction notice. There are a lot of things going on and a lot of things to unpack in that. but I did want to circle back around and close out the thought with regard to the fairness issue within this issue. The completeness issue is that because the Census Bureau goes to extraordinary lengths with their field workers to at least get the number of people in a house that they think is occupied, there’s a chance that they could come close, although I have my doubts, they could come close to say an overall state population however because of the self-response rate issues that communities of color and impoverished neighborhoods tend to have a much much lower response-rate than the bigger communities that have response rates in the 70s versus communities of color they could be in the 40s. you have a really big risk of the basically suburban neighborhoods the middle class neighborhoods the higher income neighborhoods instead of being counted accurately they are overcounted, whereas the communities of color, the communities at most risk of COVID, eviction, hunger being undercounted and when that situation arises even when you have a ballpark good estimate of a state within the state you have an unfair accuracy in the count. And what that means because we have a zero-sum gain in the 1.5 trillion dollars that are dispersed, a good chunk of that comes in forms of block grants to states. States then decide okay we’re going to release that money to communities on the basis of their population. Kind of makes sense right? However if you have a communities who are accurately or overcounted versus communities that are undercounted because of the zero sum gain some communities will get way more than they deserve while other communities will get way less and that’s the nature of the unfairness and something that really needs to be highlighted and of concern with this particular census because I believe that those inequities will manifest themselves significantly this round in 2020 relative to any of the other censuses and unfortunately that situation will be baked in for the next ten years because the Census Bureau and the way we operate uses the decennial census as the benchmark or starting point for the next ten years of population projections and things like the American Communities Surveys, the current population survey that estimates employment or the consumer expenditure surveys that’s used for the CDI, all those rely on good population estimates to accurately calibrate where population is within states and across states and within subgroups and population subgroup like African Americans, Native Americans, Asian Americans, those are hard to count communities and you can solve and compound to continue to lead to an inequitable distribution of resources as well as political representation for the next few years.
Bailer: You know Rob this seems like something that was really unexpected associated with a pandemic, you know you don’t think about the cascading of impacts, and as you were describing this, thinking about people going out the follow-up of the census workers and just how much this changes the game from the last ten year exercise. Wow, I mean this pandemic impact it’s not just us currently doing podcast interviews from home, this is changing the lives and the impact on this country for the next ten years in a dramatic way.
Santos: Unfortunately, yes and also unfortunately there is a policy component to the quality issue in the census counts by having repeated uncertainty on the length of the field period and the submission dates and counts that basically threw a wrench into census operations. So initially there was a plan to continue the fieldwork through October then have the due diligence of the full set of processing and quality checks that the census bureau collected and then the submission of counts in April. But by the White House requiring the submission at the end of December that threw the initial wrench into the operation. Suddenly the Census Bureau had to scramble and decide where they are going to cut quality. Where they can cut the due diligence that they normally do. Then they condensed field period with a field staff that was already lower than they expected to have because when COVID hit the typical types of employees which were retirees were at high risk and they didn’t want to have anything to do with field work, going out in such desperate times. So, there was less field work in the operations and start planning for the December submission which for them. however then came the injunction by the judge and that threw another wrench into the operation. So imagine you trying to do your work and just even doing a statistical analysis, this is Stats and Stories, you know you plan out your analysis time and you’re doing it, what if someone walked in the door and said no you need to be done tomorrow not in three weeks you’d just go haywire, you start doing stuff and then they walk in the door and say no no it’s okay you can go back to next week and then you breathe a sigh of relief, you start replanning what you’d originally done and then there’s another injunction okay now it’s back to October 5th instead of October 30th. And so, the career staff were just, I would imagine and fully expect that they were just pulling out their hair trying to decide okay what’s our due date what do we need to plan for? And the real unfortunate thing is that people don’t realize the importance of the post processing that has to be done to identify duplicates, to do reality checks. I mean they’re barely able to do a laugh test to look at the counts for certain cities and communities because a lot of that has to go out the door in order to get a countdown. And so there are a lot of sacrifices being made that will impact directly the quality of the census which is why we need some good quality metrics posted as quickly and expeditiously as possible by the Census Bureau and we’re hoping and praying that they do that in order.
Bailer: So, you’re listening to Stats and Stories and today we’re talking with Rob Santos, president-elect of the American Statistical Association and Vice President of the Urban Institute.
Richard Campbell: Rob I’d like to switch gears a little bit here and talk about the election, we’re about a month away as we do this podcast, and you wrote an op-ed piece in 2016 about some of the bad data or the bad statistics that came out of some of the states where the election was really close and I think the overall election was within the margin of error. Hillary Clinton won by I think two percentage points that was within the margin of error but some of these states like Michigan, Pennsylvania, Wisconsin I think surprised a lot of people and could we sort of return to that topic and are the data that we are seeing now as Biden seems to be widening his lead in these states, is this something we can trust or is there going to be another surprise?
Santos: There very well could be another surprise. The story it’s not over until it’s over. I will say on reflection based on the assessments that were done by a task force sponsored by the Association for Public Research of which many American Stat members are also members of that association and participated in that task force that the general assessment was that last time around the political polls weren’t all that bad. There were a couple of really bad ones in the states I forget what you call those states, the states – the highly contested states like Michigan and Wisconsin.
Campbell: Toss-up states. Do you call them that? I couldn’t find that word either.
Santos: So, the toss-up states had a couple of pretty poorly performing polls. Last time around the polls were pretty accurate in margin of error and that’s fine. This time around so much has changed for folks that swat at this issue. The only organizations are doubling down on their rigor and making sure they are doing things as best they can. I believe Ian Gallow had this ginormous very transparent assessment of its polls and so this time around people are really doing their due diligence. Having said that, life as we know it has changed and it’s changed in a couple of ways. The first of which is COVID and the whole nature of well what it is that constitutes a likely voter. You know a lot of times people like to say and it’s really true that polls are a snapshot of a point in time and you’ve got to be really careful about whether you’re talking about what do people think, what do adults think or what do the people that are going to present at the polls think. So there’s sort of general population polling there’s registered voter polling and then there are the folks that like to sort of play roulette, the folks that like to predict who are the likely voters and COVID has changed society in such a way that we have to be really careful about the likelihood voter models because there are different types of individuals. There may be folks who would routinely vote but they are not going to do so if they are required to go out to a polling location, they don’t want to risk catching COVID. There are folks that will totally embrace mail in voting if they are allowed to have that option. And so, there’s this whole mixture of confounding factors that go into the likely voter models that I think that that is the source of me saying it could go any way. So, I’d day what I said back then, sit back and enjoy the ride. If you did due diligence, understand the data for what they are, snapshots and predictions that are subject to error and know that there are a lot of assumptions that go into deciding who is a likely voter, many of which we cannot measure. We don’t know what the impact of COVID is going to be because there hasn’t been a lot of elections between the time COVID hit and now, and the nature of COVID and how it’s effected society is changing on a day to day basis as we learn how to deal with the epidemic.
Bailer: You know that sort of takes me back to some of the comments you were making about the census too, just the census had no idea they were going to be trying to execute this in a pandemic. You know all those years of planning in some ways went out the window. You know one of the things I wonder about with elections just to follow up on Richard’s question. You know even if it says that there’s a 70% chance of some outcome, you know a 30% chance is still pretty big and I think that it’s kind of like you know if you told me that this bridge that I’m going to drive on has a 30% chance of falling I think I’d pick a different bridge. It’s just this idea that we seem to always want to round anything above a half to one. How do we tell this story Rob, in a way that people can appreciate that this uncertainty and the fact that these probabilities, these proportions that are being reported, they’re not zero or one?
Santos: And in fact, if you come out with a 70% versus 30% you have to understand that in many ways it’s subjective. It relies on somebody’s statistical model of who is going to vote and that model may or may not be true or they may be true today but in two weeks something is going to happen, like I would hate for it to happen but a resurgence you know hot spots all over the country, that could fundamentally change the outcome of the election in the performance, the predictive performance of the polling. So, we just have to keep all that uncertainty in mind. You know people live with uncertainty all the time and it’s curious why we would want to apply a different measure of uncertainty to political polling and say oh it’s 70% so this time around the polls are bad, when in fact we live with uncertainty every day. There are the poor individuals who have to go to the doc and they get MRIs or biopsies or whatever and basically the doc will tell them the prognosis is x,y,z; it could be three months or six months or we have a treatment and we believe it’s going to be effective, but there’s a 10% chance that it isn’t. People live with those uncertainties and they have life and death meanings. Elections, well, some would argue that there’s a life and death issue here but in general elections don’t have those types of consequences, yet people want a level of uncertainty. Same thing with hurricanes, you look at all those hurricane models, they go allover the place. And there’s typically a couple that tend to be more predictive than others but anything can happen with a hurricane, it can strike anywhere, and we’ve seen that, even this season that it’s predicted to go one way and it goes another. We live with that uncertainty, uncertainty is all around us, we should accept it, understand it and be prepared to deal with whatever the outcome is.
Campbell: I heard Nate Silver I think at the end of 2016, he used a football metaphor of kickers have a 70% chance of making a field goal. Well, they often will miss, they have a 30% chance of missing and that’s how we should think about it. There’s a chance that someone would win but there’s also a pretty good chance that they’ll miss, and that’s certainly what happened in 2016.
Bailer: Well, you know, or something happened that wasn’t the larger probability.
Santos: Well, I’ll tell you another way to look at it. To the extent that chances these predictive probabilities aren’t 99 versus one, but instead are 70-30 or 50-50 or whatever, that should instill the importance and urgency of doing your civic duty and so for no other reason we say we need to act, we need to vote, we need to do what we can to make a difference.
Bailer: Well, I’m afraid that’s all the time we have for this episode of Stats and Stories. Rob, thank you so much for being here.
Santos: It was a delight, let’s do it again.
Bailer: Stats and Stories is a partnership between Miami University’s Departments of Statistics and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple Podcasts, or other places where you can find podcasts. If you’d like to share your thoughts on the program send your emails to statsandstories@miamioh.edu or check us out at statsandstories.net and be sure to listen for future editions of Stats and Stories, where we explore the statistics behind the stories and the stories behind the statistics.