Mike Lopez serves as Senior Director of data and analytics at the National Football League, and he's held this position since August of 2018, after previously serving as director in the same department. Before this role, Mike worked at Skidmore College as a lecturer, research associate and as an assistant professor.
Adriana Gonzalez Sanchez is pursuing a PhD in business analytics from the University of Cincinnati, with a research focus on discrete data analysis and statistical methodology.
Ryan Elmore is an associate professor in the Department of Business Information and Analytics at the University of Denver's Daniel College of Business. His research interests include statistics and sports, non-parametric statistical methods, and energy-efficient high-performance computing. He currently serves as an associate editor for the Journal of Quantitative Analysis in Sports.
Download “Beyond the Box Score: Does Icing the Field Goal Kicker Work in the NFL?”
Episode Description
American Football is a game played with 11 players on offense competing against 11 players on defense, with a sequence of plays executed as one team attempts to outscore the other, but each discrete play reflects actions on the field by players and off the field by coaches. For example, what is the best spot on the field to complete a pass and not get it intercepted? How do quarterbacks differ with respect to their release points? When should you ice the kicker? These are the questions that we are trying to answer on this special football-themed episode of Stats+Stories.
Timestamps
Background in Sports Analytics 2:00
Evolution of Football Data 5:23
2024 Kickoff Rule Changes 8:36
Next Generation Stats and Data Collection 15:28
In-Game Analytics and Coaching Decisions 20:35
Fourth Down Strategy Revolution 24:23
Future of Player Evaluation 27:20
Replay Process Efficiency 28:30
Ryan Elmore and Adriana Gonzalez Sanchez Interview Starts 31:50
Inspiration for the Research 33:23
Play-by-Play Data 37:53
Defining Iced Kicks & Matching Approach 40:33
Causal Inference & “Parallel Universes” Idea 45:05
Strategic Use of Timeouts & Other Situations 51:03
Future Research Directions 57:25
Full Transcript
John Bailer
Hey, everyone, I’m John Bailer, here with another double-feature episode of Stats + Stories. This time, the show is doing a deep dive on the world of football analysis. Our first guest is the current Senior Director of Data and Analytics at the National Football League, Mike Lopez. Mike shares terrific insight about some of the ways the NFL and its teams use data to learn more about league, team, and player performance and play.
American football is a game played with 11 players on offense competing against 11 players on defense, with a sequence of plays executed as one team attempts to outscore the other. But each discrete play reflects actions on the field by players and off the field by coaches. For example, what is the best spot on the field to complete a pass and not get it intercepted? How do quarterbacks differ with respect to their release points? How does kicking accuracy vary across placekickers? When are teams more aggressive on fourth down? How do coaches use their timeouts and challenges? What have been the biggest National Football League (NFL) comebacks in the last decade, and does luck play a role in the success of NFL teams?
On our show today, Mike Lopez will help us understand analytics at work in football today. I'm John Bailer. Stats + Stories is a production of the American Statistical Association, as well as Miami University's Departments of Statistics and Media, Journalism, and Film.
I'm joined in the studio by my colleague Rosemary Pennington, Chair of the Department of Media, Journalism, and Film. Our guest today is Mike Lopez. Mike currently serves as Senior Director of Data and Analytics at the National Football League, a role he has held since August of 2018 after previously holding the position of Director in the same department. Prior to this role, Mike worked at Skidmore College as a lecturer, research associate, and assistant professor. Mike, it's a delight to welcome you to the podcast.
Mike Lopez
John and Rosemary, thank you for having me. Very excited to talk about statistics and football.
John Bailer
There you go. You're starting off on the right foot. So, I'm curious: How did you get involved in sports analytics and in your current position?
Mike Lopez
I grew up wanting to major in math in college. I was a football player at Bates College as an undergrad, and I majored in math. At that point, I liked sports, I played football, and I was a math major, but the only data that existed in sports at the time was baseball. So, there I am, writing a thesis comparing Nomar Garciaparra and Derek Jeter using pretty naïve baseball stats, but it was really the best that was available at the time, and that jump-started my interest in this area.
But at that point, there really wasn't data in football. And I've always had a little bit of a grudge—now knowing that I could have been doing other things with football data. At the time, I was learning the right techniques, right? You're learning multiple regression, methods for visualization, and things like that. But that data just didn't exist. In retrospect, I probably could have collected it myself, but at that time, all you really had was baseball data.
Once football data became a little bit more public—and there are a lot of folks who enabled that—and I started teaching statistics in college, it was a really good blend of what I was interested in. I would do it for fun: you think of questions, and then you say, “Let's go answer them.” Nobody's asking you to answer them. You don't have a homework assignment. You just want to figure out the answer. That's what I would do with football data. And it turned out the league office was interested in some of those very same questions. They had an open role in 2018, and it was a good fit for where I am.
Rosemary Pennington
Now, I know we're talking about football today, but I'm a Guardians fan, so I have to ask you about this research on Nomar and Derek Jeter. I love Nomar. What were you looking at, and what did you find?
Mike Lopez
Oh gosh, I can barely remember what I did yesterday. But that thesis was honestly a lesson in bad data visualization. At the time, you don't know that, right? You're trying to find out what, in sports data, is actually predictable.
In other words, you measure something, and you want to know how likely it is that the same observation will carry forward—whether it's batting average in baseball, runs saved on defense, or passer rating in the NFL. At the time, it was mostly about splitting your data into two parts: an early sample and a later sample, or first half of the season and second half of the season, and figuring out what best predicts future success.
At the time, Nomar graded out a little bit better on a lot of the advanced data. He played one or two more years, and Jeter put together a much longer and probably better career. Back then, I was going to say Nomar, but in retrospect, I wouldn’t trust my graphs, and I wouldn't trust much about that analysis.
John Bailer
As you were talking about the data and getting started, why do you think football data lagged behind baseball data? What made football data more available, and then made this sort of analysis more common?
Mike Lopez
That's a great question. I think part of it is that all that existed in football was box-score data, and you had a really hard time parsing that box-score data into discrete events. Whereas you can take a baseball box score and know exactly what the discrete events are—who the pitcher was, who they faced, what the outcome was, who fielded it. Football just didn't have that data.
Additionally, the amount of context in football is generally a lot larger than it is in baseball. That's not to say that other things don't impact baseball—and the more you learn about baseball, the more you realize how much you wish you knew. But in football, you don't even know which players are on the field unless you have data you're getting outside the NFL. And that's kind of nuts—that we didn't even know which 22 players were on the field at a given time.
So, because of that missing context, all you really had was who got the ball and how many yards they gained, and that's it. And there's often a lot more you want to know. For a while, that information just wasn't out there publicly.
John Bailer
The questions I was asking in the introduction were all questions that emerged when I was looking through analyses you had done. I thought it was really fun to see that there were analyses at different levels: analysis at the league level—what’s happening there—analysis at the team level, and then analysis at the player level. Am I missing other categories? What are the collection of types of questions that you're being asked to address with NFL data?
Mike Lopez
When you're at the league office, in some respects, you're working for the 32 teams. We are principally concerned with the quality of the overall game. We often don't care who the two quarterbacks are or who the specific defensive players are. We certainly care who they are, but in terms of our job, it's about equity. We want to make sure we have a level playing field. We want to make sure we have an exciting game. We want to make sure it's well officiated, and we want to make sure it's as safe as it can be.
Those are the confines we're operating in. Our job is often to ask the question, “What makes for the best game?” There are a lot of things you can change—different levers you can pull—but they come with trade-offs. Something might improve officiating but come at the expense of pace of play, or it might improve competitiveness but come at the expense of health and safety. Those are just hypotheticals, but a lot of what we do is say, “If we do this, what will be the impact on the game?”
We can use replay and get a lot of our penalty calls correct, but that comes at the expense of a really long, drawn-out game that I think would ruin the entertainment for fans. So those are the types of things we're balancing and trying to answer to help the league. A lot of that uses simple box-score-type data. A lot of it uses more advanced player-tracking data. It all depends on the question we're asking.
Rosemary Pennington
I want to ask about what was a fairly significant change the league made in 2024 when it comes to kickoffs and how those are handled. Could you talk us through what that change was? And I know you tracked some data to get a sense of how that might have impacted the way special teams were approaching the kickoff.
Mike Lopez
The kickoff is roughly 6% of NFL plays—maybe 3,000 plays a year, a little bit less than that. One of the things that people noticed—this was maybe even before my time, but a lot of folks in the league took notice—was the injury rate on that play. It has a high element of space and speed, and that’s the type of behavior that leads to a higher rate of injuries.
So, trying to reduce the space and speed of that play, but also doing it in a way that kept the play competitive, was on the to-do list. It became a task that lots of folks in and around the league—and in other football leagues—worked on. The XFL was really the first league to try this form of a kickoff.
The question was: How do you make the play safer but also keep it competitive? There is no one “right” kickoff. I'm not even sure the one we have now is the perfect kickoff. But you're trying to come up with an entertaining play. For a while, if you were a fan, you saw somewhere around 70% of kickoffs go for touchbacks in 2023. Why are you going to watch a play if 70% of the time the ball is going into the end zone, and when it does come out, it's usually going to the 25 and that's it?
We wanted a play that would keep fans watching the game and have them excited. The expression Rich McKay—the co-chair of the competition committee—uses is “put foot back in the game.” Have this be a real play. For a long time in football, we had a lot of kickoff returns—that used to be common. So, it was about reducing the space and speed of the play, getting it more in line with the run and pass in terms of injury rate, but also offering competitive play.
John Bailer
When these changes are being proposed, you're doing all this analysis—rates of injury, touchbacks, and other information you can accrue over the course of observing this. But ultimately, when the proposal goes forward, what arguments—data-based arguments—are you making to suggest why changes should be made?
Mike Lopez
Our job is to be objective and to supply the decision makers—who, in this case, are the clubs, the 32 teams. There are 32 votes. Our job is to be objective and say, “If we do this, this is what we anticipate. If we do that, this is what we anticipate.”
Ultimately, the way the NFL works is that there are no rules changes in season. It’s an end-of-year conversation: assess where the year was and say, “Here are the things we want to tweak or modify.” Back in the offseason—it would have been two years ago now—was the first time we started thinking about the dynamic kickoff.
It required at least two-thirds of the votes—24 yes votes out of 32. I spend a lot of time working with numbers, and I still joke that I can't figure out what 24 out of 32 is, but you have to get 24 votes.
Again, a lot of our job isn't to say, “Go do this” or “Go do that.” It's to say, “If you do this, this is what we anticipate. If you do that, that’s what we anticipate.” Now, we could reduce all injuries on kickoffs if we just put the ball at the 25 and got rid of the kickoff altogether. So it's about balancing what you want in terms of an entertaining play and what you want in terms of health and safety.
Another reality is, if we look at officiating data and we add 1,000 returns in a season, we're going to add a lot of penalties on those returns. So, are you okay with calling an extra half a penalty per game or something like that—which is about where we are now?
That's how we're framing it. “If we want this as a result, this is what it would be.” And then ultimately, we go back and cross-check and say, “Yep, we were heading in the right direction,” or “If we were wrong, why were we wrong, and what can we do to get better?”
Rosemary Pennington
I do want to close the loop on this kickoff thing. So, you did change kickoffs for last season. At the end of the year, I know you looked at some data. What did that show? What did that change mean on the field? Were there more returns that actually were coming out of the end zone and into the field of play?
Mike Lopez
The first year we implemented it was 2024, and that was almost a bit of a trial year. We anticipated maybe around 40 to 50% of kickoffs being returned. We didn't get that high. I think we were somewhere around 33–34%. It was higher than before, but the main thing was to make sure the health and safety data came back and said, “Yep, this is what we anticipated.”
Once that happened, it gave us the go-ahead to say, “All right, how do we change the levers of the rules of this kickoff so that we can get more returns?” One of the things we did last year was move the touchback from the 30 to the 35. That meant if you kicked it into the end zone, that was a big disincentive—the ball was coming out to the 35. Not an actual penalty, but a strategic penalty, so to speak, for kicking it deep.
So now, more teams are kicking into the landing zone. Last year, we only had about one in five kicks land in the landing zone. This year, we're at 73%, so almost every team is trying to land it in the landing zone. They understand that's a better strategy for the kickoff team, and as a result, that's encouraging teams to return the ball.
John Bailer
I'm going to continue closing the loop now, because it seems there are consequences of making these types of changes in the NFL game. How does this move then influence what’s done at the collegiate level or even at high school and other levels of play? Do you start seeing this type of influence?
Mike Lopez
It's certainly the case that the NFL has a big influence on what college or high school might think about, but there are differences between the levels. I wouldn't want to jump and assume that college could go implement it and have the same exact type of play.
College kickers—a lot of them can't reach the end zone. As a result, there are some college kickers who probably couldn't even reach the landing zone, depending on the level. So I'm not sure it would be apples-to-apples to think they'd get there.
We’ve certainly had good success with our kickoff. I'm not sure our president totally agrees—he's made some comments about how he hates our kickoff—so maybe that's what's holding college or high school back. But we can only control what we can control. Ultimately, we'll want to make our kickoff better, too, if there are any levers we can continue to pull.
John Bailer
You're listening to Stats and Stories, and we're talking with Mike—I'm sorry, I'm going to do that again.
Charles, you're listening to Stats and Stories, and we're talking with Mike Lopez about analytics in the NFL. So, Mike, how has football analytics changed with technology—these Next Gen Stats? What data do you collect in the course of a game?
Mike Lopez
For a long time, we were restricted to what I would call box-score, play-by-play data. If there are 150 plays in an NFL game, your dataset had 150 rows, and you could track who was the ball carrier, who made the tackle, the quarterback, receiver, distance of the pass, etc. But that was about it.
In 2017, all the players and teams got what we call Next Gen Stats data. There is an RFID chip in each player's shoulder pads that tracks wherever they go on the field at a rate of 10 frames per second. That gives you their speed, their acceleration, their orientation. Clubs got that on all teams and all games in 2017, and that is what we popularly refer to now as Next Gen Stats.
That is a massive dataset. You go from 150–160 rows to maybe 200,000–300,000 rows per game. Before, your analyst was opening up Excel and looking at play-by-play data. You can't open up 200,000 or 300,000 rows of data in Excel and manipulate, clean, and analyze it easily.
So now, you need folks who can analyze data in R or Python, which are the two most popular coding languages for statistics in football at this point.
In 2018, we opened that data up to all teams and all games. The prior year, you only got it on your own team. In 2018, you really started seeing football data take off. Around that time, a couple of companies—primarily PFF—started putting a lot of their work out there. PFF was hand-collecting scouting data that would tell you, for example, whether it was man coverage or zone coverage, if the player was in a two-point stance or a three-point stance, and other very detailed information that could be tagged to that play.
You then had the play-by-play data you’d always been used to, plus the scouting data—which is what coaches have always been using in their brains and tracking internally—and the tracking data. Suddenly, you had a lot. It was almost an open book: “What do you do with this data?” That’s really when I think football data started taking off.
John Bailer
I'm curious: With all the data that's now available, and with teams building deeper analytics benches to help analyze the data that's collected, what type of data can be used during the course of a game? Can an analytics coach influence decisions in real time?
Mike Lopez
At this point, my hunch is that almost all but a few head coaches have an analytics specialist on their headset. That person—a guy or a woman—is tasked with some of the questions you mentioned earlier: when to call a timeout, when to throw a challenge flag, when to go for it on fourth down, and other strategy-type decisions that are tough for a head coach to think through in real time. Most of those folks are using data to drive their decision-making.
But there is no live technology that they are using. They are not allowed to have an internet connection. They're not allowed to have a phone. There's no dynamic Excel sheet that can be updated in real time. There’s just not much you can do analytically during the game.
What you do have is a notebook or binder that you can bring in. You can flip to the right page in that notebook and say, “All right, it's fourth-and-four at midfield in a tie game at the start of the fourth quarter. Here's what we’re going to do on fourth down.” I can do that as a team analyst, or I could just use my gut and tell the coach that. But most team staffers at this point are bringing some type of binder or cheat sheet, and that’s what they’re using.
My hunch is that we’re going to grow in this space and that there will inevitably be slightly more technology in-game. This year, the NFL is trial-running basically a static Excel-like spreadsheet that opens up and can be used to track participation in the game.
For years, every single team would have a low-level staffer tracking, “Hey, was Rosemary in on this play?” or “Hey, was John in on this play?” They were tallying it up literally old-school. Knowing that, the league said, “Why don't we just track this for them? We have this data. Let's free them up to do actually interesting and helpful things.”
So we’re giving them some participation data in a spreadsheet now, but that’s not something you can use to fit real models in-game. There's no AI in-game. Maybe football will eventually come around and become a little bit more data-savvy in-game with strategy, but I do think there’s a component of, once the game clock starts, teams have to use their preparation as a guideline, as opposed to relying on machine-learning recommendations.
Rosemary Pennington
The idea of going for it on fourth down has come up a couple of times, so I want to ask about this. We've had someone on to talk about it before. I grew up watching the Browns and watching football—I love it; it's my favorite sport.
In the ’80s and ’90s, it felt like teams were only going for it on fourth down when it was on the goal line and you knew that if you didn't score, you were going to trap your opponent deep, maybe get a safety, and get the ball back. It has felt like coaches are increasingly becoming more aggressive about going for it on fourth down.
I remember watching a couple of games this season where it felt like it wasn't just a coach going for it when they had a yard to go. I've seen coaches who seem to be weighing it when it's two or three yards out, which seems like a big difference.
I don't know if that's just me noticing something at a particular moment, or if, in the data you're collecting in the NFL, you're really seeing a notable change in how coaches are approaching that decision on fourth down.
Mike Lopez
It's been the most transformative change in football over the last eight or ten years. I wouldn't say it's entirely driven by data, but a big chunk of it is folks with backgrounds similar to mine simply pushing and pushing coaches to be more aggressive.
There’s historical research dating to the 1960s and ’70s that suggested teams could be more aggressive, and here they are. Almost every single year, in our end-of-year “Season in Review” for Data and Analytics, we show the competition committee that we’ve set a new record for fourth-down aggressiveness.
Then the next year: “We have set a new record for fourth-down aggressiveness.”
Then the next year: “A new record.”
Every single year, we tick up, and it's really transformed the game. It's not just the fourth-down attempts themselves. It does a couple of things.
First, it keeps your punters off the field. We have somewhere around 1,000 fewer punts in a season now than we did a decade ago, and that's just a massive change. You're taking a play that brought a specialist onto the field, who booted the ball into the air while players sprinted down and fair-caught it near the goal line, or whatever. Some would call that entertainment; some might say it's just a different kind of play.
Now you're keeping your starting quarterback on the field. You’re having a major, high-leverage play with big implications for each team's chance of winning the game. As a league, we like that. That's a good thing for the game. We have more game-deciding plays with our best players on the field.
Second, it’s changed what third-down strategy is. Before, on a third-and-eight, you were probably throwing an eight-yard pass trying to pick up a first down. Now we see teams on third-and-eight run the ball, because they know that if they can pick up four or five yards, they're going to go for it on fourth down.
Ironically, it's led to fewer incomplete passes, because teams don't pass the ball as deep as often. They're okay with running the ball. We now have fewer plays per game in part because we have fewer incompletions.
So it's been a massive shift that had the intended consequences the league expected: more fourth downs—very exciting—fewer punts, and a massive number of fewer punts, and also fewer plays per game in part because teams are going for it on fourth down. It's been pretty exciting to watch. At this point, if you're not going for it, you're pretty far behind in terms of where you are analytically. That edge you could have gotten four or five years ago is a little bit smaller now.
John Bailer
That's an interesting question. I read a book many years ago called Scorecasting, and one chapter was devoted to a coach—this was at the high school level—who always went for it on fourth down, always went for two points, and always did an onside kick. For this coach, it was a philosophy of possession, which I thought was really interesting.
Mike Lopez
That book’s over my right shoulder.
John Bailer
It's a great book, isn't it? I really enjoyed it.
Mike Lopez
It's one of the reasons I'm in this field. Those types of questions, I think, are really, really fascinating.
John Bailer
In some of the data-fest competitions you’ve sponsored, I saw ideas that seemed to influence player evaluation—for example, a new “figure-eight” drill that was introduced for player evaluation. How has analytics in football influenced player evaluation and then, maybe ultimately, player preparation for competition?
Mike Lopez
I think football is a lot like every other sport, where for a long time drills were done, plays were called, and decisions were made largely because that’s how they'd always been done.
Think about where players lined up in baseball for a long time—your left fielder, your center fielder, your right fielder. They stayed in the same spots. Then you saw data come out and say, “Hey, listen, we really need to shift those players a lot.” You had second basemen lining up where the shortstop used to be, and vice versa.
In football, we largely ran the same drills at the scouting combine—which is where college players go to prepare for the draft and show how athletic they are—because those were the drills that were initially done. Players ran the 40-yard dash, even though offensive linemen won't ever really run much more than 10 yards at a time in an NFL game.
So at the league office, one of the things we did was take a step back and ask: If we were to design drills for the Combine from scratch, what are the player movements they do a lot in games that we are not measuring at the Combine?
One of those movements, for defensive linemen in particular, is that they move in almost circular patterns. If you think about a defensive end trying to rush the quarterback, they're not usually moving in a straight line. They're moving in a curved path. Similarly, teams run a lot of stunts, where players move around other players.
Yet all the movement drills defensive players were doing at the Combine were straight—straight runs, then a 90-degree turn, then another 90-degree turn, or a 180-degree turn. But they don't really do those movements in games.
So one of the things we've helped with at the league is trying to think about what newer Combine drills should look like. The hard part is, it's tough to take away some of those old drills. You have a lot of coaches who are used to having a 40 time and a short-shuttle time.
But we can add some of these newer, more position-specific drills to help evaluate movements that players are more likely to perform in a game.
Rosemary Pennington
So you are dealing with, I'm assuming, massive amounts of data. What do you think is a data story the sports media has not gotten wind of yet that they really should be paying attention to when it comes to the NFL?
Mike Lopez
Oh man. The funny thing is, we deal with massive amounts of data to answer some questions, and then we deal with the most old-school form of data to answer others.
For example, we track every quarterback sneak to identify if somebody behind the quarterback pushed the quarterback. Is that the fanciest form of data collection? No. Literally watching a play and putting a 1 or 0 in your Excel sheet is the most old-school form of data collection. But ultimately, it's not in our play-by-play data. If you look at that data, it does not say “Mike pushed Tom on the quarterback sneak,” or “X pushed Jalen,” right? You have to watch and figure that out.
We've thought about using player-tracking data to figure it out, but ultimately that doesn't tell you where the limbs are. You could have players who are right next to each other but not actually pushing.
So, to answer your question, some of the stuff we're doing is really old-school but answers really interesting questions. One other area we spend a lot of time on is thinking through our replay process and what the timing of replay looks like on every single review.
When the coach throws a challenge flag, or when the AMGC—our Art McNally Game Day Center in New York—sees something on the field that needs a decision on a certain play type and they can stop the game, the second that happens, we start a stopwatch.
We track from that moment through the end of the decision all the different parts:
How long did it take the league to make the decision?
How long did it take the officials on the field to make the change?
How long did it take us to come out of commercial break, if there was a commercial break?
How long did it take for the next play clock to start—which is basically the ball being back in play?
How long does it take the coach to talk with the referee?
Each element of the replay process we’re timing, and then we’re figuring out, “Can we be more efficient?”
If you looked at NFL replay a decade ago, our average replay was two and a half to three minutes long. Even a couple of years ago, our average replay review was somewhere around two minutes and twenty seconds to two thirty.
This past year—certainly currently—I believe we're averaging about a minute and twenty-two seconds. So we've shaved off about a minute per review.
There are a lot of reasons we've gotten better—technology has improved, the process has improved—but a big part of having a more efficient replay process is just having targets: knowing where you were, knowing that you can be better. If there's a tough decision, of course the room is going to take the amount of time needed to make the correct decision.
But once you have what we call the “kill shot”—that means we see the key angle, we know what the decision should be: did the player complete the pass, did the ball break the plane, did the player reach the line to gain—once we have that, we're done. Then the question is: how do we get this announced and keep the game moving?
Ultimately, we’re an entertainment product. If we're spending three or four minutes to make a decision—especially on plays where, if you look at film from 10–15 years ago, the fan at home already knew the decision—why are we continuing to wait?
I don't want to pick on college football, but every time I watch it, I feel like I can go to a commercial break and cook a soufflé during the time they take to review something. So we're trying to be more efficient.
You mentioned all this data that we could be using—here, we're using a stopwatch. So it varies.
John Bailer
Wow. Thank you, Mike, for all of that wonderful insight into how the league and teams are using data to track what happens in the NFL.
Next up, we have Professor Ryan Elmore and grad student Adriana Gonzalez Sanchez to talk about their paper in Significance magazine about when to ice a kicker and, more importantly, whether it works.
It's late in the game, and your team is leading by less than three points, and your opponent is driving down the field. A field goal will win the game. An attempt to kick this field goal likely will be the last play of the game. Do you call timeout in hopes this will unsettle your opponent's placekicker?
The folklore in American football is that calling a timeout as the kicker is about to attempt the field goal will increase the chances that the kicker will miss. Is this true, or is it a fiction that might be debunked?
A 2024 article in the “Beyond the Box Score” column of Chance magazine titled “Does Icing the Field Goal Kicker Work in the NFL?” by Adriana Gonzalez Sanchez, Sierra Martinez, Ron Yurko, and Ryan Elmore attempts to answer this question, and we're happy to have a couple of the authors on the show today.
I'm John Bailer. Stats and Stories is a production of the American Statistical Association, as well as Miami University's Departments of Statistics and Media, Journalism, and Film. I'm joined in the studio by my colleague Rosemary Pennington, Chair of the Department of Media, Journalism, and Film.
Our guests today are Adriana Gonzalez Sanchez and Dr. Ryan Elmore. Gonzalez Sanchez is pursuing a PhD in business analytics from the University of Cincinnati, with a research focus on discrete data analysis and statistical methodology. Elmore is an Associate Professor in the Department of Business Information and Analytics at the University of Denver's Daniels College of Business. His research interests include statistics and sports, nonparametric statistical methods, and energy-efficient high-performance computing. He currently serves as an associate editor for the Journal of Quantitative Analysis in Sports.
Also, I've known Ryan since the late ’90s, when he was working on a master's degree in statistics at Miami. Adriana and Ryan, it is a delight to welcome you to the podcast.
Adriana Gonzalez Sanchez
Thank you. It's a pleasure to be here.
John Bailer
So, just to start, what inspired you to begin this investigation of whether icing the kicker works?
Ryan Elmore
Adriana, do you want to take this, or do you want me to start?
Adriana Gonzalez Sanchez
You can take it because it's connected to your basketball research.
Ryan Elmore
Right. Going back a few years, I wrote a paper related to calling timeouts in the NBA. We were looking at whether calling a timeout when the other team is on a run actually affects the game.
After that paper was published, I was talking to Ron Yurko, who runs the Carnegie Mellon Sports Analytics Camp. I told him I thought I had a problem that would be really good for some students who were going into that camp. Ron thought it would be great as well, so he asked me to present it to the students.
Adriana was excited about it. Adriana, you can come in and talk about where you picked it up.
Adriana Gonzalez Sanchez
Sure. I was in the Carnegie Mellon summer camp, and we had to pick a topic for our final project. We had different proposals from different professors, and I was potentially thinking about going into a PhD program. I was really interested in research and getting to know how it was done.
After talking to Ron, he told me this project was a really good start because it was an interesting topic and also something that could be published in the near future. Those were my two main reasons for picking the project.
Rosemary Pennington
I grew up as a Cleveland Browns fan, which comes with its own set of ups and downs—and one of them includes kickers. Phil Dawson was this incredible kicker for the team at a time when the team was not so hot. But in my childhood, there was a kicker, Matt Bahr, who very memorably missed a couple of really important field goals.
As I was reading through this, it brought back all these flashbacks. I wonder: What is it about icing the kicker that you found so interesting, and how did you actually go about figuring out whether it’s worth using that timeout to ice a kicker?
Ryan Elmore
First, apologies for being a Browns fan. I grew up just across the river from Cincinnati in Kentucky, so I've always been a Bengals fan. They're basically the same—Bengals or Browns—they've both been pretty terrible historically.
As for your question: When I worked on the NBA timeout paper, I wanted to look at problems where you’re evaluating a coaching decision—not just team performance or player performance. In other words, coaches have a decision to make. Can we evaluate whether that decision actually affects the outcome of a game?
Icing the kicker fit that mold. I'm going to be totally honest: I'm not a huge football fan, but I knew this action in a game was completely dependent on the coach calling a timeout. Then there is a very clear outcome that happens right after that decision. We can measure that outcome and ask, “Is this affecting the result?”—in this case, whether the field goal is made or missed.
I just liked the idea of evaluating a coaching decision that directly.
John Bailer
Can you talk a little bit about the data you used to start investigating this question?
Adriana Gonzalez Sanchez
We used the NFLFast package. It's available in R—you just have to download and install the package. It’s basically a package that cleans the data posted on the NFL API and makes it readily usable for analysis. It contains play-by-play data from NFL games.
Rosemary Pennington
As you were going through this data, I saw that icing tends to happen on longer field goals. If it's a chip shot, a coach isn't going to waste a timeout to ice a kicker on what's, for the most part, an obvious make.
If we're talking about a 60-yard field goal, how do you puzzle out whether it's the distance that led to the missed field goal or the choice to ice the kicker? That’s the big question for me: How do you disentangle distance from that choice?
Ryan Elmore
First, icing the kicker really didn't have much of an impact on the outcome. It’s mainly a function of distance and the conditions present in the game—turf versus grass, dome versus outdoor, and so on.
From a statistical perspective, it's a relatively straightforward problem. You're including a variable for “Did the coach ice the kicker or not?” and then you have all these other control variables you're looking at as well.
I’d say you could go back to one of the classes I had with John long ago and handle this in a straightforward way. The reason we didn't do it then but are doing it now is, as Adriana mentioned, we now have the data available. We have data on every kick, conditions associated with each kick, and attributes associated with those kicks—field conditions, as I was saying a moment ago.
Really, the NFL making these data available is what enabled this problem to be solved.
John Bailer
So can you help us understand how you extracted the data for the kicks that were iced or potentially iced? You had this play-by-play dataset that you could extract from, and at some point you had to select from it all field-goal attempts. And then from that, you had to do something to identify which might have been iced. Can you talk a little bit about that?
Adriana Gonzalez Sanchez
Yes. I think we had a column indicating field-goal attempts, and then we looked for timeouts called just before the kick. Whenever there was a timeout called within a few seconds prior to the kick—if I remember correctly, we used a window of about five seconds—that’s what we considered an iced kick.
I did this a couple years ago, so it might not be exactly five seconds, but that’s how I extracted the kicks that were iced versus the ones that were not.
Ryan Elmore
To your other question—what are the potential kicks that could have been iced? We essentially used a matching tool in R. You look at whichever covariates you want to match on. It could be time in the game, relative strength of the opponents, and so on.
You look at instances in games where a kick happened, and then you try to see all the instances where kicks were not iced. Those are your potential controls. Within that potential control batch, you’re trying to match conditions that are similar to the actual iced kicks.
Adriana Gonzalez Sanchez
Yes, I remember that the main variable we matched on was win probability. We tried matching on others, but the results were pretty much the same in the end.
John Bailer
One variable I found myself thinking about as I was reading your paper was the experience of the kicker. Is it someone who just joined the league versus someone who is a little more seasoned?
And by the way, Rosemary, I'm sorry that this brought up these difficult, almost PTSD-type reactions related to kicks in football.
But the experience of the kicker seems like one factor. Another is: Did the kicker miss the last kick they took? There are so many ways you might imagine trying to get into the kicker’s head.
Did you consider some of these other factors, or was it just not feasible to implement them?
Adriana Gonzalez Sanchez
We looked at a bunch of different aspects: player experience, the weather, the type of turf, whether the stadium had a roof or not, because all of that can affect whether the kicker makes the field goal.
But keep in mind that the data we used runs from 1999 until 2021. The quality of the data back then is not the same as it is now. It was difficult to get good estimates of some of those aspects from the earlier years.
Maybe we had really useful data starting around 2008 or 2010. It would have been hard to base the analysis only on half of the years in our dataset.
Ryan Elmore
Yes, and to your point about kicker experience: Adriana and I were thinking about potential questions offline, and one of them was whether we could do something similar in college football.
I think kicker experience would probably have a bigger effect in the college game, just because they're less experienced. When you get to the pros, you're talking about the 30 or so best kickers in the world. They're really good at what they do.
So whether you're in your first year or you're a seasoned veteran, you're still a very, very good kicker. The coach calling a timeout is probably not going to rattle you much. It’s more like, “All right, here we go again.”
John Bailer
You're listening to Stats and Stories, and we're talking with Adriana Gonzalez Sanchez and Ryan Elmore about icing the kicker.
One of the things you mention in your paper is that this isn't just a random decision that a coach makes. There are parallel universes that branch off from this point in time: one where the timeout is called, and another where it isn’t.
Can you talk a little bit about what that means in terms of the analysis you would do, or some of the challenges you faced?
Ryan Elmore
Sure. In a perfect world, it would be great if they had you or me or Adriana sitting behind the bench flipping a coin to see if the coach should call the timeout or not. But coaches aren’t going to allow us to do that.
So, in this situation, we try to do the best we can. I don't want to go off on a monologue here, but I was listening to a recent Stats and Stories podcast where you had Greg Matthews on—he and I are good friends. I had just listened to his talk at JSM, and he said one of the things he tries to do when he takes on a new problem is imagine, “If I had the perfect set of data to answer this question, what would it look like?”
Unfortunately, we don't have that randomized controlled trial here where we randomize kicks into “we're going to ice” or “we're not going to ice.” So what do we do?
We try to find kicks that were similar to the iced kicks. Then, as I was saying before, we match on those attributes and try to see if there's a difference using causal inference methodology. We try to create a pooled sample where we have like kicks—some iced, some not—and analyze it from that point.
So we're trying to recreate that randomized controlled trial that we don't have access to.
Rosemary Pennington
Adriana said that you also used win probability to assist with the matching. Could you talk through how you conceptualized win probability and how it factored into this?
Adriana Gonzalez Sanchez
We think of win probability as a single number that estimates how likely the possession team is to win the game at the start of a play, accounting for situational context such as field position, time remaining, score differential, and more.
We found that iced kicks tended to involve plays with a higher win probability for the kicking team.
Ryan Elmore
One thing to add about win probability: that’s not something we computed ourselves. It comes from the dataset Adriana mentioned earlier, the NFLFast dataset.
Adriana Gonzalez Sanchez
Yes. The NFLFast package calculates it for us because the NFL API doesn't have win probability directly. The R package extracts the information and calculates it, so we don't have to do it manually.
John Bailer
As part of the analysis you did, you modeled the log-odds of making the field goal as a function of the distance of the kick and whether or not the kicker was iced. As a side comment, did you include any interaction between those? You treated them as additive components. Did you also play with other model structures?
Adriana Gonzalez Sanchez
I think we did try an interaction, if I'm not wrong, but the results were pretty much the same. We tried incorporating different aspects of the game as well, but the results were very similar to what we reported.
Ryan Elmore
Yes, in the end we kept things pretty simple, particularly for the general Chance audience. I think they appreciated something direct and to the point.
Rosemary Pennington
Timeouts in any sport are precious, and you only get three of them in football. I wonder, now that you've done this research, if you were advising an NFL coach about whether it's worth holding onto a timeout to ice somebody, is it worth it? Or should they use it in a different way?
Ryan Elmore
I would say: Don't hold onto it just to ice them. Icing isn't really going to have much of an effect. You should call timeouts when you need them throughout the game.
There are two answers here. First, no, I wouldn't hold onto a timeout in the hopes I can ice a kicker later on. I would use them as needed throughout the game.
However, at the end of the game, if you still have a timeout, there's no reason not to use it. It doesn't carry over into the next game, so there's no reason to hang onto it. Just use it—even though our research suggests it's not going to affect the outcome. What's it going to hurt, really?
John Bailer
So you agree, Adriana?
Adriana Gonzalez Sanchez
Yes, I agree with what Ryan said. If you have a spare one, use it. But don't save it just because you think it's really going to work—it might not.
John Bailer
Ryan, you were talking about the idea of imagining the perfect dataset if you could do this as a controlled clinical trial.
I was also thinking about when is the best time to use timeouts more generally. It seems the best time to use a timeout would be when it leads to the greatest increase in your probability of winning the game. That would be the strategic, “best” use.
In the context of football or other sports you’ve investigated, do you have thoughts about when those pressure points might be?
For example, icing a kicker at the end of the first half might not matter much. But at the end of the game, it might matter more.
Ryan Elmore
That's exactly the problem we worked on prior to working on this one. We looked at runs in the NBA. If a team is on a run and the coach calls a timeout, does that actually affect what happens afterward?
For that problem, we were looking at the scoring margin—the differential in scoring margin in the two minutes after the event. It's a similar type of question.
As far as other sports or other times in an NFL game, going for it on fourth down is a well-studied area, but there could be interesting work combining timeouts and fourth-down decisions.
For example, suppose the offensive coach really wants to set his team up. Maybe they're driving down the field, but they get to a fourth down. If they were to score a touchdown, how would that change win probability? Is it better to call a timeout there to maximize that chance?
I think that could be interesting to look at. But as I said before, I don't watch football enough to be deeply into when coaches should make certain decisions. I'm more of an NBA or basketball fan, and this project was an easier transition to football, which is why we studied it.
Adriana Gonzalez Sanchez
Same here. I'm from Spain, so I had no idea about the rules of American football before starting this project. So don't ask me about plays—I’m not going to be good at answering that question.
Ryan Elmore
It's American football. Adriana knows a lot about football—just a different kind of football.
John Bailer
I was thinking about that in terms of icing the kicker. In world football—soccer—penalty kicks are a classic example where a goalie will do all they can to try to disrupt a player's flow. They’ll walk out, pick up the ball, set it down, wave to their parents in the stands—a lot of ways to break rhythm.
Can you think of other examples? I think it's a really interesting topic—the idea of breaking flow in a game to try to change the trajectory of win probability.
Adriana Gonzalez Sanchez
I can give an example from tennis. I played tennis in college, so I'm familiar with the game.
If I lost a set really easily—6–1 or 6–2—I might go on a restroom break between sets just to reset my mind and maybe break my opponent’s rhythm a little bit. In more professional tennis, when you're tight in a third set, you might get into a small argument with the umpire to throw off your opponent and break their rhythm a bit.
In college tennis, this happens all the time. Maybe after the first set, a ball lands on your court and it doesn't really bother you, but you walk and move it as slowly as you can to give your opponent extra time to think, feel more pressure, and you hope they'll miss the next serve. It might not work, but we still do it.
John Bailer
Ryan, please.
Ryan Elmore
I was just going to say that any of those instances, in whatever sport you're interested in, where there's some element of gamesmanship—you could study that.
I was thinking about baseball, for example, when a pitching coach might go out to the mound just to reset things and see what's going on with the pitcher. You could study how that affects the next number of pitches, or the rest of the inning, or whatever might be going on.
Gamesmanship, going back to your question about flow and interrupting flow, is sort of the whole point.
John Bailer
So what's next for you? You've done this project. You concluded there really wasn't much effect at all in this case. What's the next problem you're interested in investigating?
Adriana Gonzalez Sanchez
In sports? Right now, I'm doing my PhD, so I don't really have all the time in the world to focus on sports problems. I would love to, hopefully once I get a job, but I can't do that right now. The clock is ticking, and I really have to focus on my dissertation. So there's not really a future sports problem for me right now, but I would be excited to have one in the future.
Ryan Elmore
Once Adriana finishes, we’ll think of more problems for her to work on.
For me, there's a soccer problem I want to look at—basically, player positions. There are some companies making positional data available for players on the pitch. I want to look at how interactions of players affect outcomes.
That's a tough problem because, when you think of “good outcomes” in soccer, everyone thinks of goals—and there are very few goals. So we want to think about other outcomes in a soccer match that we could look at to evaluate whether players are doing what they should be doing.
Maybe it's about increasing win probability or something similar—I'm not sure yet. But that's in the back of my mind at this point.
John Bailer
Well, I'm afraid that's all the time we have for this episode of Stats and Stories. Adriana and Ryan, thank you so much for joining us today.
Adriana Gonzalez Sanchez
Thank you.
Ryan Elmore
Thank you.
John Bailer
Stats and Stories is a partnership between the American Statistical Association and Miami University's Departments of Statistics and Media, Journalism, and Film. You can listen to us on Spotify, Apple Podcasts, or other places you can find podcasts.
If you'd like to share your thoughts on our program, send your email to statsandstories@amstat.org, or check us out at statsandstories.net. Be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.
Thank you, Ryan and Adriana, for joining the show. Well, we are long past the two-minute warning, and it’s time to leave the stadium and head home. Thank you all for joining us for this special double-feature football episode of Stats and Stories.
Stick around—we will be back on Wednesday, December 24th, with a feel-good episode for the holidays.