Sam Waters is the Managing Editor of the Harvard Sports Analysis Collective. He is a senior economics major with a minor in psychology. Sam has spent the past eight months as an analytics intern for an NFL team. When he is not busy sounding cryptic, he is daydreaming about how awesome geospatial NFL data would be. He used to be a Jets fan, but everyone has their limits.
I see two legitimate rationales for immediately benching a running back who fumbles, one long-term and one short-term. In the long-term, a coach might argue, benching a fumbler will teach that player a lesson, inspiring him to improve his ball security for the rest of his career and help the team in future seasons. Meanwhile, in the short-term, the same coach might say, a running back who just fumbled is more likely to fumble on his next opportunity, and needs to sit on the bench so he can’t hurt the team right now. The long-term issue is harder to test, so here we’ll focus on the short-term concerns. If we want to test this reasoning, we need to find out if a running back with past fumbling problems is actually more likely to fumble in the future.
First, we can look at the consistency of fumble rates across seasons to see whether fumbling looks more like a repeatable skill or random noise. The autocorrelation for fumble rates, which measures the association between a player’s fumble rate in one season and his fumble rate in the next season, should give us a sense of this. Below we can see the relationship between running back fumble rates in “Year N” and “Year N+1” over the last ten seasons:
The autocorrelation of fumble rates for the running backs who had at least 100 touches in consecutive seasons over the last decade is 0.19. This indicates a weak, positive association between year-to-year fumble rates, so guys who fumble frequently one year somewhat tend to fumble frequently the next year. We can tweak our analysis by changing the touches cutoff or using more than one past season to predict Year N+1 fumble rate, but doing so yields similar results. It’s important to get a sense of fumble rate consistency on a seasonal basis, but this initial look at the data doesn’t give us a full picture of the short-term rationale for benching fumblers or its validity.
The short-term rationale posits that a running back who just fumbled is more likely to fumble again soon. Given two players who were similar in every respect up until the moment that one of them fumbled and one of them didn’t, the short-term rationale coach would opt for the one that didn’t fumble. But is there really any difference between these two otherwise identical players?
To answer this question, I used the Rubin Causal Framework to compare single-game fumble rates between players who fumbled the week before and players who didn’t fumble the week before, but had similar histories leading up to the game in question. (If you’re interested, you can read more about the Rubin Causal Framework here and at other Google-accessible places.) I measured player similarity using a player’s likelihood of fumbling in a particular game given their age, games started, touches, rushing yards, and fumble rate both over the previous three years and in-season leading up to the fumble in question. After controlling for these factors, I was able to find a point estimate and variance estimate for the effect of fumbling last week on fumble rate this week.
This method estimates that the effect of fumbling last week on this week’s fumble rate is a decrease of 0.16 percentage points, the opposite of what those in the short-term rationale camp might expect. The 95% confidence interval of this point estimate ranges from -0.43 percentage points to 0.11 percentage points. In other words, we don’t have enough evidence to say that the effect of fumbling last week on fumble rate this week is meaningfully different from zero. This doesn’t mean that momentum doesn’t exist from game to game when it comes to fumbling; we just couldn’t find sufficient evidence to support it here.
It might be better to look at this question on a more granular level by using play-by-play data. The play-by-play version of this study might pose a question like “What are the chances of a running back fumbling sometime in his next ten carries after a fumble?” If we use games, a coarser unit of time than plays, we lose information, so answering this question using play-by-play data seems like the logical extension of this project.
Another study could also improve on these methods by using better covariates to identify similar players. We used basic stats like yards and touches to quantify running back similarity, but more sophisticated stats would probably provide a more accurate proxy for a player’s past quality and characteristics. This more advanced set of covariates could help us avoid confounding factors, enabling us to get closer to the true causal effect of recent fumbles on current and future fumbles.
While this study design leaves room for improvement in the future, it provides some evidence against employing the short-term rationale when benching a running back for fumbling. If coaches are really using this rationale, they are actively hurting their teams, because they are sacrificing overall running back quality for a potential gain in turnover margin that we cannot find evidence for. Unless evidence for the existence of fumbling momentum emerges, coaches might want to leave the immediate-fumble-benching strategy behind.
Except Mike Tomlin. Someone needs to provide intro paragraph material for all the football writers out there.