Sharon Lohr researches and writes about statistics: where they come from, how to interpret them, and how to tell the good statistics from the bad. After receiving her Ph.D. in statistics from the University of Wisconsin-Madison, Sharon taught for 25 years at the University of Minnesota and Arizona State University, where she was Dean’s Distinguished Professor of Statistics. As a Vice President at Westat, she developed survey designs and statistical analysis methods for use in transportation, public health, crime measurement, and education. She now does freelance statistical consulting and writing. See the feature article about Sharon in the September 2018 issue of Amstat News.
Episode Description
If you’ve been following the news much then you may have noticed reporters beginning to explore how COVID is impacting crime rates around the country. Police commissioners are even appearing on newscasts trying to explain how various COVID measures may have changed the kinds of crimes they’re seeing in their cities. One of the problems becomes tying those changes directly to COVID and of course, a long-standing issue when it comes to crime rates is understanding how we measure crime in the first place. Measuring crime is the focus of this episode of Stats and Stories with guest Sharon Lohr.
+Full Transcript
Rosemary Pennington: If you’ve been following the news much then you may have noticed reporters beginning to explore how COVID is impacting crime rates around the country. Police commissioners are even appearing on newscasts trying to explain how various COVID measures may have changed the kinds of crimes they’re seeing in their cities. One of the problems becomes tying those changes directly to COVID and of course, a long-standing issue when it comes to crime rates is understanding how we measure crime in the first place. Measuring crime is the focus of this episode of Stats and Stories, where we explore the statistics behind the stories and the stories behind the statistics. I’m Rosemary Pennington. Stats and Stories is a production of Miami University’s Departments of Statistics and Media, Journalism and Film, as well as the American Statistical Association. Joining me are regular panelists John Bailer, Chair of Miami’s Statistics Department and Richard Campbell, former Chair of Media, Journalism and Film. Our guest today is Sharon Lohr. Lohr is an Emeritus professor in the school of mathematical and statistical sciences and the dean’s distinguished professor of statistics at Arizona State University. She is also an independent statistical consultant. Her research interests include survey sampling, design of experiments, and applications of statistics in education, criminology, and law. She is the author of the book published in 2019 Measuring Crime Behind the Statistics. Sharon, thank you so much for being here.
Sharon Lohr: Nice to be here, thank you for having me.
Pennington: How did your interest in how we measure crime develop?
Lohr: It actually started when I was a [sound cuts out] adjusted statistics, held a workshop for people on using national crime victimization survey data and so before that I’d been working in [inaudible] analysis, which is a completely different area of statistics, and after going to this workshop, it was actually a two-week workshop I just fell in love with the whole subject of survey sampling and how do you measure these things.
John Bailer: So, did you ever get involved in studies in designing survey tools or instruments for studying crime before you got involved in writing this book?
Lohr: I have been. You know I’ve worked on the national crime victimization survey data for a long time and while I was at ASU I participated in a project on developing a mail survey that different localities could use to measure crime. So, for example, if a community wanted to implement a new community policing program and they wanted to have a mail survey to evaluate the effects of that, they could do a survey before and then after having the program in place.
Bailer: So how do we learn about crime in our communities? What are some of the sources of data that’s used to inform our understanding about how safe is the place where we live?
Lohr: Well, there are two major sources of data. So, the first major source of data is the police statistics. These are collected by the FBI they’re published in the uniform crime report and they have been collecting nationally since 930 so that’s when they started. In the 1960s through the government, Lyndon Johnson had a crime commission and its recommendation was that there should be a survey- a household survey of crimes. Can we start this one over?
Bailer: Sure.
Lohr: Yeah I’m sorry.
Bailer: No worries. Take two.
Lohr: Okay, so there are several major sources of crime. So, one of them that started around 1930 is the uniform crime reports and this is a compilation of statistics that are collected by law enforcement agencies across the country. And if you go to the FBI website you’ll find them and they have a nice crime data explorer that you can explore all of the trends over time, you can download the data and these are crimes that are known to and recorded by police agencies. The other major national source is the national crime victimization survey, and this is a survey of households and people within the households, people aged 12 and over within the households. And it asks them about their experiences with crime. And not surprisingly the national crime victimization survey generally has higher crime rates than the uniform crime reports because it’s asking people about their experiences and it’s also capturing crimes that are not reported to the police. And so that’s basically the only source of information we have nationally on crimes that are not reported to the police. And it changed with the type of crime, but overall, for violent crimes about half or so tend to be reported to the police, and we know that from the survey.
Bailer: And which types of crimes is there the most disagreement between these two data sources?
Lohr: It’s generally the more serious the crime the more agreement there is.
Bailer: That makes sense.
Lohr: And also, for crimes that you really need a police report to get recovery, such as motor vehicle theft, motor vehicle theft has the highest percentage reported to the police because your insurance company wants the police report.
Richard Campbell: Sharon, what’s one thing that you would want the general public to understand about crime statistics that they probably don’t understand very well right now? And I say this because you seem to have this commitment to being a statistician who wants to reach the general public in a way that some scientists and statisticians aren’t trained to do that; they don’t do it. So, what doesn’t the general public understand about crime statistics?
Lohr: I think it’s that all crime statistics are estimates. They’re not exact numbers. And I think that a lot of people, they see the homicide statistics from year to year and they think oh, there were exactly this number of homicides this year nationwide, but even homicide statistics are an estimate.
Bailer: I liked how- I’m sorry, please. I really liked how you described in your book these issues of you know, the idea that there’s missing data, that there’s measurement challenges. Could you talk a little bit about these other sources of uncertainty that are part of the data that’s being reported as kind of a follow up to Richards question?
Lohr: Right well, all statistics that we look at have uncertainty associated with them and when you look at a survey like the national crime victimization survey they’ll give you a margin of error about the statistics. And it’s just like when you see election polls right now, we’re seeing a lot of those being publicized at the moment. And they all have a margin of error associated with them but that’s just one type of error and that’s the error that results because the poll has a sample of, say, 1,000 people instead of looking at everybody and the same thing for the national crime victimization survey. It has a much much much larger sample, but it doesn’t look at every household in the country. So, the sampling error, that’s what people learn about in their intro stats course, right? When you’re doing a confidence interval that’s the kind of error you’re looking at. But all of these other data sources have errors too. So, the police reports, they have missing data from all of the crimes that are not reported to the police. They also have measurement error because sometimes a crime is put into the wrong classification. So, something might actually be an aggravated assault, but it’s recorded as a simple assault. Or a burglary, they might not know, and it might be recorded as a vandalism. So, there’s that kind of error as well. And these errors- there’s no measure of these that is reported to people.
Campbell: So, you- one of the things that you’re interested in which I’m also interested in is that most people find out about crime statistics through journalism. Through the reports that reporters make of your work and other statisticians’. What can journalists do better? I think you even comment that they’re getting better at it, but what aggravates you when you see a news report on a crime statistic?
Lohr: I want to start by acknowledging what you just said and that is that journalism about statistics has just improved tremendously over the last few years and so you’ll see a lot of reporters going in-depth into the sources and into what could possibly you know be affecting the statistics other than the things that are reported in the source. But I think one thing journalists want to be careful of is looking at very very short-term trends and reporting them as real events. And I think we’ve seen a lot of that this Spring. So a recent example I saw was a news story that said the number of people shot in New York City was up I think it was like 414%, I don’t know if you saw that story, but it was based on one week of data from the New York Police Department, and it happened to be the week wherein 2020 I think there were 74 people were shot and I think they compared it to a week in 2019 where 14 people were shot and that was the biggest discrepancy that was reported for the year.
Bailer: You know, it seems like that’s one of the themes you encounter in your book is this idea of understanding context, and I really like, particularly the table where you talk about the eight questions to ask about a statistic. So, you know in the example you just gave you know it sounds like the question that you’re hitting on there is what were the methods that were used to get this information. That was part of it and also what is the source of the statistic? So, what are some of the other examples within your eight questions that help you understand the quality of a statistic?
Lohr: I mean I think the primary one is what is the source and I never want to trust a statistic where somebody says oh well I saw this on Facebook, right?
[Laughter]
Bailer: Really?
Lohr: Yeah, how about that. And I mean actually that chapter about how to judge the quality of statistics, that’s the reason I wrote the book. Because I wanted to write something for you know the general citizen who sees statistics all around and thinks well how do I know which ones I should trust because this source says this statistic and this source says this statistic and what I wanted to get at there is that there are statistical principles that you judge statistics by.
Pennington: You’re listening to Stats and Stories and today we are talking with statistician Sharon Lohr, author of the 2019 book Measuring Crime Behind the Statistics.
Bailer: You know Sharon, in your book Measuring Crime, I really enjoyed the chapter that was touching on crime statistics 1950 and beyond and one of the people you introduced in there was Edith Abbott and I was just thinking wow, given this description and all the work she did it’s really remarkable to not hear her and describe and discuss in some of our intro classes. I mean and by the way, I think that’s a challenge I’m going to ask some of the introductory professors in our department as they talk about historical figures within statistics. Can you talk a little bit about what lead to her work in crime statistics and in fact she’d published in 1950 a report related to statistics crime in Chicago? What- why was Edith Abbott so engaged in this topic and what put her in a place to have this impact?
Lohr: I mean this is- I had actually never heard of Edith Abbott before either and it’s partly because she just didn’t appear in the histories they’d written about statistics, or about crime statistics and this is even though she was elected to be a fellow of the American Statistical Association later and she was well known at the time and you know President Hoover later picked her to be on a commission writing the report on crime. You know she was the first woman to be the Dean of a US graduate school, so she was extremely well known, just not in conjunction with statistics. And so, what I think happened is that she was just in the right place right when there was this need for a report because in the Spring of 1914 what happened is there was this crime wave in Chicago. Or I should say people thought there was a crime wave because nobody had any statistics to back it up. And so, at the time Charles Merriam was an alderman on the Chicago City Council and so he wanted to have an investigation into what are the statistics behind crime, you know what actually is going on? Is crime really you know increasing like the newspapers are saying they are at the moment. And he didn’t know very many statistics himself, and there’s been documentation about some of his limitations in that regard, but he knew Edith Abbott. And so, she was teaching in Chicago School of Philanthropy and she was at Hull House which was the settlement house founded by Jane Addams and this was the place to be. If I could go to any dinner party in history I would want to go to one of those dinners at Hull House, where you just had all these incredible people gathered around the table discussing the issues of the day. And so she had a background in statistics, maybe the best background you could have had at the time, and so he asked her to take charge of gathering the data and what’s amazing to me is that she did all of the data gatherings and she wrote her report in less than six months without any computational, you know, assistance from electronic devices.
Bailer: You know it was funny when I was reading your description of this I felt like I could be reading some of the headlines from the present and some of the work that motivated what she was doing. So, what were some of the assertions that she either [inaudible] or refuted with some of the data she collected?
Lohr: Well, one of the big things that she looked at was the issue of immigration in crime and that was a big debate back in the day. A lot of politicians took one side or the other but without any data. And what she did is she gathered all of the data sources that she could find in Chicago from the police department, from the courts, from the house of corrections, and she looked at the quality of those data sources to be able to assess whether they’d be good for her purposes. So, she found that some of them were better quality than others, but she had suggestions for [inaudible] the quality of all of them. And so with respect to the issue of immigration and crime, she compared the percentage of people from each nativity group in Chicago with those in the 1910 census and her conclusion was that actually there were fewer foreign-born people in Chicago being arrested than their proportion in the 1910 census.
Campbell: And the mythology at the time was that crime was coming more from foreign-born right? And that’s what she uncovered was that wasn’t actually true. That reminds me of today actually in some of the reporting that’s gone on.
Lohr: Right and so she, I mean she did acknowledge that her data were not of very high quality and that was another thing that I think set her ahead of her time, that she was so concerned about the data quality. But her conclusion from her investigation is she reported that the foreign-born accounted for 36% of arrests, 35% of convictions but 54% of the Chicago population of men aged 21 and over. So, a big discrepancy.
Bailer: That’s such a nice simple reasoning- a proportional reasoning to try to make that comparison to make that case. That’s an effective communication, I mean- did it have an impact? Did it change the way things were being reported, do you know?
Lohr: I haven’t been able to find that it had that much impact and it was partly because of the context when the report was released in 1915 because it was released right after one of Chicago’s more colorful mayors was elected, Big Bill Thompson, and he was strongly anti-immigrant.
Pennington: Sharon- Oh sorry. One thing that I have been thinking of a lot in relation to some of the conversations we have had particularly around Florence Nightingale, and now this conversation we’re having about Edith Abbott is that there seems to have been a number of women who were working early in the field of statistics that certainly popularly we don’t recognize. Like Florence Nightingale, we recognize as this sort of you know Saint of nursing but I think the general public has no idea that she was a figure in the field of statistics as well and why is it that so many of these female people who were sort of helping shape the field so early have been overlooked and why do you think it’s now that we’re starting to learn more about their contributions? Or at least understand them better?
Lohr: I think that’s an excellent question. Because Edith Abbott, she is well known in the field of social work, but in Chicago in just Chicago at that time, there was a publication by the University of Chicago that listed the people teaching courses and their publications and they listed what, I think around 60 faculty in the Sociology department. And four of those were women. And when I looked at the publications women were out publishing the men at the time and they were publishing in the same journals, but then what happened is the women were then split off into social work, and the men were retained in sociology, and so I think that had a big impact on how the work was viewed academically. And it’s also for Edith Abbott’s report you know I don’t think you gave to [inaudible] any kind of conspiracy to keep women from being recognized, her report was a chapter in a larger report that was authored by Charles Merriam. And so, a lot of people referred to this report, but they referred to it as the Merriam Report. Not as the Edith Abbott report.
Campbell: We can probably guess here that men were writing the early history of statics also, which is-
Lohr: Right well, the other thing is that Abbott was not on the committee that started the uniform crime reports. I think there were men on that committee that were familiar with her work and that conveyed some of her ideas but it’s hard for me to believe that some of the early uniform crime reports would not have been better if they had gotten Edith Abbott on there, herself.
Bailer: Yes. So, you wrote that her work helped set the stage for the UCR. So, can you talk about a couple of those specific contributions that were embraced and then incorporated in the UCR?
Lohr: Well, one thing that she emphasized was that there needed to be uniform definitions for how crimes are classified and recorded. And you needed to have procedure for how this is done. So she talked about how in Chicago in 1915 that you know there weren’t any good data on crime complaints and so she argued that the closer you actually get to the crime the better the data are likely to be. So, courts are further away than the arrest data and the arrest data are further away than the complaints to the police, and so she argued that having data on the complaints to the police would actually be a really valuable source of information about crime.
Bailer: I wonder as I think about this I was thinking 1915 when was the first department of statistics in the US? Wasn’t it in the 1940s?
Lohr: Actually, there were state bureaus of labor statistics.
Bailer: Okay, I was thinking of academics. But in terms of service to the state we probably see offices earlier than that but yeah.
Lohr: Yeah but certainly there were a lot of government offices in the 19th century. I think Carol Wright became commissioner of labor 1885- I may be wrong about that.
Bailer: Yeah but I was thinking just going back to the academic side of it, it just as a discipline I think statistics emerged pretty late in- it was 1940s or so before one of the first in the US.
Lohr: I think Iowa state was before that.
Bailer: Was it? Okay. I think they fight about state there. [inaudible]
Lohr: Right but Gertrude Cox came to NCC from Iowa State Department of Statistics there, so.
Bailer: So, Richard, I want you to notice that there’s been a lot of history to talk about now.
Campbell: That’s right I also wanted to compliment Sharon on her writing and I did want to know about your interest in writing and on your interest as a statistician reaching the general public, where did that come from? And where did you learn to write? Because some statisticians, no offense John…
[Laughter]
Bailer: Hey wait a minute.
Campbell: No, he actually does write well. So where did that come from?
Lohr: I think it’s just that I read everything that I kind of come across and so but the interest in writing this book came from just conversations with friends and neighbors who aren’t statisticians. And I think what we’re seeing right now is there are so many sources of information that people have and there are so many statistics being thrown at you. And so, a natural question is well how do I know what’s a good statistic and what’s not a good statistic? How do I know what I can trust? And I think there are a lot of really excellent books about statistics. But I didn’t see one that addressed this issue about how somebody can who is not a professional statistician look at what’s out there and judge whether they should trust it or not.
Pennington: Well Sharon, that’s all the time we have for this episode of Stats and Stories. Thank you so much for being here.
Lohr: Okay, thank you for having me.
Pennington: Stats and Stories is a partnership between Miami University’s Departments of Statistics and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple Podcasts, or other places where you can find podcasts. If you’d like to share your thoughts on the program send your emails to statsandstories@miamioh.edu or check us out at statsandstories.net and be sure to listen for future editions of Stats and Stories, where we explore the statistics behind the stories and the stories behind the statistics.