Claire Kelling is an assistant professor of statistics at Carleton College. She’s an expert on data and statistics in relation to police use of force and says her work sits "at the intersection of criminology and spatial statistics". Kelling organized the 2023 Ingram Olkin Forum on Statistical Challenges in the Analysis of Police Use of Force. Five articles from that forum appeared in a special themed section of December’s issue of Chance including several authored or co-authored by Kelling.
Episode Description
The use of force by police has been in the news a lot lately, in connection to everything from protests on college campuses to the death of individuals during arrests. There’s no singular, shared definition of what use of force is according to the National Institute of Justice. A local police department will set a standard, but that threshold for when an office should use force varies from place to place. Having no standard set of rules or definitions makes it difficult for researchers to study the issue. That’s the focus of this episode of Stats and Stories with guest Claire Kelling.
+Full Transcript
Rosemary Pennington
The use of force by police has been in the news a lot lately in connection to everything from protests on college campuses to the death of individuals during arrests. There's no singular, shared definition of what use of force is. According to the National Institute of Justice, a local police department will set a standard, but that threshold for when an officer should use force varies from close to plays, having no standard set of rules or definitions makes it difficult for researchers to study the issue, and that's a focus of this episode of stats and stories, where we explore the statistics behind the stories and the stories behind the statistics Secretary Pennington, stats and stories is a production of the American Statistical Association in partnership with Miami University's departments of statistics and media, journalism and film. Joining me, as always, is regular panelist John Bailer, emeritus professor of statistics at Miami University. Our guest today is Claire Kenley, an assistant professor of statistics at Carleton College. She's an expert on data and statistics in relation to police use of force, and says her work sits, quote, at the intersection of criminology and spatial statistics. End quote, Kelling organized the 2023 Ingram Olkin forum on statistical challenges in the analysis of police use of force. Five articles from that forum appeared in a special themed section of December's issue of chance, including several authored or co authored by killing she's here today to talk about those articles that forum and the issue of statistics and police use of force more broadly. Claire, thank you so much for joining us today. Thanks for having me. You know, I was struck by reading the articles, and again, by looking at the National Institute of Justice and a couple of other entities when I was researching this that there is not a unified definition for why, what use of force is, or even when, when a place has a definition what various levels are. How does that impact the work of researchers like you and these people who participated in this forum when you're trying to study this issue?
Claire Kelling
Yeah, it's a great question. One of the things it makes me think of is a recent project where we're comparing cities across the United States to figure out what information are they collecting. How are they recording it on various use of force incidents in their urban area, and it really varies how they classify levels of force. There are some cities that have dozens and dozens of categories of force that they record, and there are other cities that have basically just four levels that they use to categorize the type of force incident. And I think one of the major challenges that researchers have is trying to make some sort of conclusion about patterns between different cities. For example, if you wanted to compare two cities, but they collect data in completely different ways, they organize data in completely different ways. Can be really challenging to make conclusions about patterns in one city compared to another city, or if there's an intervention in one city, whether it would apply to another city, and especially if you think about urban versus rural, that's a whole other set of challenges, where the coding and the data and even the technology available in Different cities can really be quite varied.
John Bailer
So if you were able to take, take the some magic wand, and you said, okay, here is what it should look like. What, what would, what would you have recorded? And you know, what are some of the factors you and one of the papers that, that we reviewed, you talked about situational, then officer demographics and subject demographics. Can Can you expand on this a little bit?
Claire Kelling
Yeah, so one of the things that I think is most important, and I'm biased, I'm a spatial statistician, so this is something I'm really interested in. But one thing that is quite uncommon with across use of force incident data sets, is location information, and specifically precise location information. Every once in a while you'll see use of force data sets that contain information about the neighborhood or the police beat, which is a it's not a huge geographic area. It's not like a county, but it's a relatively large area. And every once in a while, you'll see anonymized addresses, so addresses that have a little bit of information about where that location is redacted, but yeah, location information would be huge. Location information would allow us to say a little bit more information about where use of force incidents are happening, and specifically to characterize maybe. What kind of neighborhoods those use of force incidents are happening? What's their relationship with who is living in those neighborhoods? For example, this is really playing back into my research, which is in large part in in conversation and in collaboration with community members, where they really want to know where are things happening, where in our community are use of force incidents happening compared to our lived experiences and our knowledge of where we're being policed. So that's one thing. I'm really I'm really interested in spatial information. But on top of that, it would also be really helpful to have information about who is involved in the use of force incidents. So you mentioned citizen and Officer information, characteristics. One big part of that is demographics. It's becoming more and more common to have things like the age, the gender, the race, the ethnicity of people that are involved in the use of force incidents, but still, many, many cities don't release anything of that kind. But it would also be informative to have a little bit of information about who, exactly, beyond those demographics, is involved in use of force incidents. So a couple of cities across the United States release officer and citizens IDs, so you can track if an officer is involved, for example, in more than one use of force incident. Where are they involved, geographically, and what kind of incidents are they involved? Usually higher levels of force, lower levels of force, for example. And that can be really helpful to track, again, maybe problematic patterns across the city by a person rather than by the whole police force? Yeah, maybe just one more thing I'll say about that there's a sort of conflicting theories about use of force, whether it's whether in some departments, it's kind of like systematic problem, like in terms of the training that's being offered, for example, in other departments, or in other theories, you see things like bad apples, like it's individual police officers that are causing problems and having access to information, like the officer ID or badge number, can help us study that in more detail.
John Bailer
Yeah, that's, I was just curious. It sounds like that's, that's, that's a lot of detail on this, and it's, it's increasing the load. How would that be implemented? It seems like you have all of these autonomous organizations that are collecting this information. Is there some how would this be implemented? If you if there was kind of a consensus form, a standard form for the types of information that should be collected, who would be the one that put that in place?
Claire Kelling
Yeah, it's good question. And I know there's a new national use of force database that exists, I believe it's collected by the FBI. There is definitely incentive there, in some way, to collect information that has these standardized fields that's then going to be reported to the federal government. That information is not publicly available right now, we have information about what percentage of agencies are reporting, but very little information beyond that is publicly available now. I'm hoping that will change in the future, but I believe, yeah, in my opinion, federal data collection would be huge in incentivizing and actually giving structure to what information, what variables would be helpful to collect for each use of force incident. Beyond that, there is already guidance by the IACP. I think it's the International Association of Chiefs of Police that gives information about what information should be collected for each use of force incident, ideally, and this is a recommendation. It's not always followed, obviously, in these different cities, some of them give the bare minimum of information. Some cities don't even release data at all publicly on use of force incidents, but some federal guidance, either that is compiling the information like the FBI through the National use of force data collection, or through IACP on giving guidance on what use of force data collectors should be looking at. I think that'd be really helpful to move us forward here
Rosemary Pennington
in in one of the articles you talked a lot about about spatial statistics and sort of the struggles of figuring out how to study this in space, because you have a problem of definition. Are you going to define space by the census tract? Are you going to define it by the precincts? Could you talk through a little bit of some of the challenges of trying to study this spatially.
Claire Kelling
Yeah. So one of the challenges of saying police use of force in a spatial way is because a lot of police use of force data sets don't actually contain any information at all about space. So there's information usually on when. There's usually some information on what. Happened, like the level of force that's used, if you're lucky, there's information about who, but where is usually sort of a last component, if we're very, very lucky, to get some information on where. So what that means is, when we do have information on where, the level of aggregation is usually pretty high. So we have information, yeah, maybe on the precinct level, maybe on the beat level, but very rarely do we have addresses. So when you analyze information on the precinct or the beat level, one of the issues is that doesn't always line up with demographic data, such as census data, so you have to deal with this misalignment problem, where census tracks can be different sizes, but they can also police beat, for example, can cover two census tracks, and then you have to make a decision, okay, what? How do we associate these two things? And there's quite a bit of literature out there with some guidance on how to do that, but it's definitely a challenge when you're dealing with these misaligned spatial problems. Another spatial challenge comes with privacy. Many use of force incidents and crime data, public health data in general, is privatized in some way, and the usual way that that's done, if we're lucky, to get some sort of address information beyond place beats or precincts, is just by redacting the last two digits of an address. And that basically puts every use of force incident on a grid, which is definitely an unrealistic picture of where use of force incidents are actually happening. So as reach researchers, we have to think about, well, how do we analyze this data that has essentially been discretized in some way, in a way that is actually maybe representative of the original data? So part of my research is figuring out, yeah, how do we analyze that data that's been privatized in some way? But also, how do we maybe generate data that's been privatized in a better way? So those are both sort of angles that my research is going in, you know, before,
John Bailer
before we go much further with this, I think it would be helpful to just talk about kind of, that gradient of use of force. Because, you know, there's a I was interested as I was reading through some of your your articles, you're talking about everything from it this, this gradient from empty hand control to hard controls to less lethal to lethal. So can you, can you give some examples of kind of, this range of possible uses of force, and then, then you also have the all the control conditions where there wasn't use of force that that have to be considered. Oh, yeah.
Claire Kelling
Okay, so one of the most fundamental problems in use of force research is we don't have a sense, usually, of all of the interactions between police and civilians, so we have data sometimes on, yeah, interactions that didn't involve a weapon, like if a police officer is talking to or yelling at even a civilian, sometimes that is represented in the data. But yeah, we certainly don't, in most cases, have information on all of the non interactions, the spaces where police officers maybe saw civilians or had a non confrontational conversation, for example, none of that is represented in the vast majority of data sets. I will say there is quite a bit of what I find to be very interesting research going on right now, looking at body worn camera footage, so that you can actually try to get a sense of more of those interactions that didn't result in force, so you can get a sense of when force actually happens, and hopefully in the future, a comparison to when force doesn't happen. But yeah, so when we think about levels of force that is recorded. Yeah, it's really a huge spectrum. And a lot of initiatives focus on fatal force. But really, I think the most, the vast majority, of force that's being used is definitely not fatal. So when we think about my town that I'm in right now, I'm very close to Minneapolis, and that's the focus of a lot of my work. Lately. There's use of force incidents that range from, yeah, empty hand control, all the way to using weapons of various kinds, like tasers, firearms, things like this in various data sets, not just in Minneapolis, but in other ones that it is reported if there's some sort of verbal command, if there's a canine involved, things like this. So really, a wide array of things
Rosemary Pennington
are being recorded. You're listening to stats and stories, and we're talking about statistics and police use of force with Carleton College's Claire Kelly. Claire. We're talking to you because of this sort of special section in chance that was spurred by a forum you organized. I guess my question for you is, what made you feel like you needed to organize this forum?
Claire Kelling
Yeah, so I started out in graduate school, really interested in criminal justice data, and I wanted to combine, really what was at the time, and extracurricular interests in gender based violence, in community organizing, with my academic interests in statistics. And I really had a hard time combining, finding a way to academically combine these two interests of mine, and I didn't really see a lot of spaces where statisticians were really involved heavily in this research. And I also think in policing research, it's really hard to get started. The data is a mess. The methods are quite challenging at times. And I wanted to provide a space that researchers that all stages of their career, but really especially young people, students, undergraduate students, graduate students, postdocs and early career, faculty and professionals. I wanted to develop a space where they could start, where they could get engaged, or they could get a sense of some of the preliminary challenges in this area, so that they felt like they had ground to stand on and could potentially engage in a deeper way in this research. So this forum, yeah, was what I thought would be a great opportunity for these interested folks, but maybe intimidated folks, to get more involved in this area, with the goal of making an impact, of trying to better understand policing data, try to develop better statistical methods to analyze this data, and to try to engage all sorts of folks in the conversation. So this conference that we had had statisticians, but it also had economists, sociologists, political scientists, and we also engaged folks outside of academia. So we had people from government agencies, we had folks from community organizations, both in Minneapolis and elsewhere, come to the forum from nonprofits. So it was a really great way to have a varied conversation that really Yeah, brought people into this area and hopefully along Long standing way. You
John Bailer
know, one of the things I believe I read in one of your papers, it said that you were also hoping to help people that were wanting to start doing the analysis of policing data to avoid common mistakes when analyzing such data. What are some of the common mistakes that people would make if they're naively approaching such data? Yeah, so
Claire Kelling
one common mistake is not looking at what the data represents. And I'll give you one example that's right at the top of my mind, because yesterday, I was looking at the Minneapolis use of force data, which was recently updated. And the idea of this data set is that each row represents a use of force incident. But if you actually look at the data more critically, the way they've coded certain incidents now is actually not a use of force incident. It contains information that is relevant to a use of force incident, but it's not actually representing a use of force incidents, about, yeah, the length of an incident or the highest level of force that's being used, but it's information that should be a variable, not actually representing a use of force incident. So those rows should actually be either deleted or Re Coded, not counted as an additional incident. So if you weren't looking into that data with a critical eye, you would have had an over count of the number of use of force incidents in a specific area. So that's one reason, one way that you can have common mistakes and analysis is not thinking critically and looking into the data to figure out what a row represents. Is it in an entire interaction? Is it just one part of an interaction, for example, or is maybe there information that you don't even want contained in that row in your data set, another common mistake that comes up very frequently, especially when you have maybe undergraduate students or early career people looking at use of forest data is making causal conclusions when that's not appropriate, and that's not my area of expertise, but You do see it happen quite a bit these, yeah, causal statements and causal conclusions, especially about demographic information and its relation to higher or lower levels of force being used.
John Bailer
You know, I was interested in your comment about the body cam data that might be available. And I thought. Myself thinking, I wonder if body cams have in it GPS recorders, if that's something, if that's if that's metadata that's being recorded as part of those body cams. And sort of as a related point I'm thinking about, if you have this video record of an interaction, is there at some point, are methods going to be available to help with classifying degree of force in an interaction through some some kind of classification tools,
Claire Kelling
I certainly hope so. And I think eventually AI tools will probably be relevant for that kind of analysis and coding. I think we have to be very careful about how and if they are ever used in that case. But right now, I think a lot of the work that's being done on body worn cameras is using student coders, so students that are watching the videos and have this data collection or data coding mechanism of labeling in some way what's going on in that video, but I certainly hope that eventually we'll be able to get very, very detailed information about, yeah, what's going on, both in terms of when use of force is happening in a video, but also what's happening in a video that's not use of force in terms of this bigger scope of Officer civilian interactions,
Rosemary Pennington
use of force incidents, especially if they are they lead to a death or there's lots of witnesses, end up becoming news stories you know all the time. What advice would you have for journalists, given the work that you do, on how to best cover these incidents?
Claire Kelling
Oh, gosh, yeah, that's a really good question. One of the things that I spent a lot of time thinking about is how to engage the community in my research, and specifically how I can add to that conversation. As a statistician, we know that the community is that are dealing with police violence on a day to day basis, and I can't understand, I can't possibly understand the full nature of police violence in a community just from the data. So interacting with community members has been a really important part of my research. Incorporating the community through community partners and organizations into my research plan has really been important. And yeah, my, I guess my advice to journalists would be, yeah, to think about one how, how is the community sort of grappling with this? How is the community understanding how this, maybe one incidence, fits into maybe a bigger context of police violence in their area. So drawing that bigger picture of lived experiences, I think, would be really helpful, but also secondarily, thinking about what's the bigger picture from the data as well? So yeah, lived experiences, but also actual data analysis, I think could potentially draw a bigger picture than one isolated use of force incident.
John Bailer
So I'm, we're here. I'm curious about, well, what's next for you? You know, as you, as you think about this, what? What are you excited about working on now that might help with with the analyzes that of police use of force data?
Claire Kelling
Yeah. So this forum, the chance, articles and other initiatives have gotten me really excited about building community in this area, so bringing in young people. I have a statistics and Public Policy Research Group at Carleton College where I involve really young students, really experienced students, all across their time at Carleton in this area, to try to show them how statistics can make an impact, but also to show the importance of engaging our community in this research as well and in these conversations. And I think, yeah, in statistics, sometimes the focus can be so much on data analysis that we forget about the actual people that are involved in the phenomena that we're studying. So yeah, I think that's been really exciting for me to kick off my research group. I've been working with students since I arrived at Carleton, but officially, my research group started this fall. So that's been really exciting. I'm also really excited to dig more into this area where I'm living now. I moved to Minnesota three years ago, and I've been forming relationships here, but to yeah, get to know the community more. Get to be more engaged in the sort of city council aspect of things has been really exciting. So yeah, I'm looking forward to getting to know Minneapolis even more, getting to know the community members that I'm working with even more too. So,
John Bailer
so one, one last little question, the you know, how, how do the when you're in, when you you're interacting with police departments, what do they think about these types of analyzes? What's kind of the feedback that you've received?
Claire Kelling
Oh, yeah, that's a that's a challenge. Challenge. So I will say my experiences is not universal, but I'll just share two anecdotes. One anecdote is on an analysis in Baton Rouge. We're working on an analysis of intimate partner violence in Baton Rouge, and intimate partner violence is really not very well geographically coded, for good reason. Not a lot of precision is given when you're saying where intimate partner violence events happen. And that's well, well taken that there shouldn't be a lot of precision there. But I'm working with the police department in Baton Rouge to try to understand some sort of prevention awareness analysis of that kind of data, and they've just been so helpful. They've been so responsive. They are willing to help us analyze and collect data that isn't publicly available but can be understood at some sort of higher level that can be made publicly available safely. So that's just been really helpful in terms of having open communication with the police department staff on what can be released, what can't be released and why. On the flip side, in Minneapolis, it's been really difficult to learn about how policing data is collected, why data practices have changed, the documentation of use of force data in Minneapolis is pretty sparse in terms of what's contained in those data sets, and it's been quite challenging to Get more information about that especially, yeah, if you email the data folks at the Minneapolis Police Department, I've had varying degrees of responsiveness from them. So yeah, that's definitely been a challenge that I'm hoping will change over time, as in building relationships here, rather than, you know, emailing folks that I'm sure are very busy, but yeah, that's something that it's been challenging in some jurisdictions and quite wonderful in others.
Rosemary Pennington
Stats and Stories is a partnership between Miami University's Department of Statistics and media journalism and film and the American Statistical Association. You can follow us on social media outlet formerly known as Twitter or Apple podcast, or other places where you find podcasts. If you'd like, share your thoughts on the program. Send your email to statsandstories@miami.oh.edu, or check us out at statsandstories.net, and be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.