Sam Waters is the Managing Editor of the Harvard Sports Analysis Collective. He is a senior economics major with a minor in psychology. Sam has spent the past eight months as an analytics intern for an NFL team. He used to be a Jets fan, but everyone has their limits.
The NFL season might only be nine weeks old, but the league’s playoff picture is already starting to gain clarity. Having a playoff picture that is too clear, too early, might make games a little less exciting, so I thought it would be interesting to see if this year’s playoff qualifiers really are more certain at this point than in past years.
We can start to attack this question using the projections at nfl-forecast.com, which use the team efficiency ratings here at Advanced NFL Stats to estimate each team’s chances of making the playoffs. According to these projections, eight teams currently have a probability of playoff qualification higher than ninety percent. While these teams (like Denver, Kansas City, and Seattle) are virtual locks, we have thirteen teams with a ten percent chance or less of qualifying, leaving us with a pretty polarized distribution of playoff odds after nine weeks of play:
After Week 17, when the field of playoff qualifiers is completely certain, we will have twelve teams with a 100% chance of qualifying and twenty teams with a 0% chance of qualifying. As you can see in the histogram above, we are already moving in that direction. With the fate of 21 teams almost sealed, we are left with a pack of only eleven teams in the middle vying for the four playoff spots that are still up for grabs. If this degree of clarity is normal, we would expect to see approximately the same number of teams gravitating toward either end of the distribution at this point in previous seasons.
If we want to go about comparing the degree of “playoff picture clarity” across years, we have to find a way to measure it. As we approach a perfectly clear playoff picture, every team gets closer to either a zero percent or one hundred percent chance of qualifying, so we can gauge the clarity of the playoff picture in any given week using the spread of the playoff probabilities for all thirty two teams. I’ll use standard deviation as my measure of spread, so in weeks with a higher standard deviation of playoff probabilities, the field of teams that will qualify for the playoffs is clearer. For example, after Week 17, the standard deviation must be 49.1 (12 teams as 100%, 20 teams at 0%). In contrast, if we assumed every team had an equal chance of making the playoffs, the standard deviation would be zero. If we compare the standard deviation of playoff probabilities in Week 9 this year to the standard deviation in other years, we can find out if and when the playoff picture was this clear in previous seasons:
The standard deviation for Week 9 in 2013 is higher than the Week 9 standard deviation for every year except for 2009. That year we reached the same level of clarity that we have now back in Week 7, when Indianapolis, Denver, and New Orleans had playoff probabilities over 99% and five teams already had a 0% chance at the playoffs (rough seasons for Cleveland, Kansas City, Oakland, Detroit, and Tampa Bay). Meanwhile, in 2010, fans had to wait all the way until Week 15 to get a playoff picture as clear as our current one, with the Chargers and Chiefs battling for the AFC West title while the Seahawks and 49ers fought over the NFC West and the Giants and Packers vied for the sixth seed in the NFC.
If we compare the standard deviation for every week of 2013 thus far to the standard deviations for those same weeks over the last five years, we can see that the playoff picture has been clearer than average since Week 5, when NFL forecast started making these projections.
It looks like the likely field of playoff qualifiers has been consistently more clear than usual over the last month, but judging by the 2009 results, this level of clarity is not unprecedented. As an aside, the dip in week six in the graph comes from missing data – in years that tended to have higher standard deviations, nfl-forecast did not publish week six playoff probabilities. I expect that a larger dataset would smooth this dive significantly.
Our original question was whether this year’s playoff picture was more certain at this point than it has been in past seasons. After looking over these data, it appears that the answer is yes - the likely playoff qualifiers are more certain than they have been over the last five years, on average. But this season is not an outlier by any means, and there have certainly been seasons with greater or comparable certainty at this point over the last five years. The situation looked more extreme before this weekend’s games, with major changes to Green Bay (-19.7 percentage points), Dallas (-11.2), Chicago (+23.2), Philadelphia (+17.5), Miami (+14.0), and Tennessee (+13.7) all pushing the standard deviation back down. So with another week in the books, it has actually become less obvious which teams will be playing in January. Not that there’s anything wrong with uncertainty – not knowing who will be in the playoffs means the remaining games will have even higher leverage, making them more exciting for everyone.