The QB passer rating system is roundly criticized for various reasons, yet it continues to be used in the media and by analysts around the NFL. The formula used to create the passer rating is a bizarre, complicated, redundant, incomplete, and arbitrary equation. And it should be replaced.
CURRENT NFL PASSER RATING CRITICISMS
It is bizarre because the result, the actual rating, signifies nothing. The units are in terms of...well nothing--not yards, or points, or completions, or their equivalents.
It is complicated because it combines four separate components that each practically require a slide ruler to compute. For example, the interception component is 2.375-(Int/Att x 25).
It is incomplete because it does not consider sacks.
It is redundant because it includes both completion percentage and yards per attempt. The yards per attempt stat is highly dependent on on completion percentage. Yards per attempt is actually just (yards per catch) x (completion %). Therefore, completion percentage is double counted in the formula.
It is arbitrary because each of the four components are not weighted in any meaningful way. The components of the formula are based on multipliers and constants selected to give the rating a nice scale rather than based on their importance to scoring or winning. They also use arbitrary maximums--each component is capped at 2.375 for an unknown reason. Look at the interception component formula above. Why is it multiplied by 25? Why not 30 or 20?
My personal criticism of the passer rating is that it includes touchdown passes. They should not be included in a passer rating because they are the result of many other factors beyond QB passing proficiency. For example, a QB could be on a team with a terrific defense that frequently produces turnovers deep in an opponent's territory. Or he could benefit from a great running offense that sustains drives. Further, TD passes are the culmination of all the other attributes of the passer. Accuracy, avoiding interceptions, and avoiding sacks all lead to TD passes.
I contend that passer attributes that enable the TD passes are far truer measures of passing performance than whether a WR broke a 7 yard slant pass for 50 yard TD run. Why should the QB get credit for that? What about a perfectly thrown 40 yard pass to a WR knocked out of bounds at the 1 yard line, followed by a 1-yd TD run? We don't need a fantasy football scoring formula, we need a reliable passer rating system that tells us how much a quarterback contributes to his team's success.
BUILDING A BETTER RATING SYSTEM
My goal is to create a meaningful passer rating based on factors controlled primarily by the quarterback alone. The weights of each component are based on their statistically predictive power of team wins. The units of the rating are in "wins added." In other words, the result of the new rating reveals, holding all other team performance measures equal, how many wins above (or below) average, a QB's passing performance would add to his team over the course of 16 games.
The formula is based on a multivariate regression model of team wins. Using data from the past five NFL regular seasons, the regression model estimates team wins based on the efficiency stats of each team including passing, running, turnovers, and penalties. Regression models can hold all other factors equal. By only adjusting the factors of interest (passing performance) we can see the effect on the estimate of season wins. The regression model automatically weights each factor corresponding to its actual importance in terms of team wins. Arbitrary weighting is not necessary.
The "wins added" model of rating athletic performance is based on a rich foundation of previous work in other sports. In baseball, Bill James's runs created statistic weights its various components in a way that approximates a batter's share of runs scored by his team. The recent book Wages of Wins proposed a similar stat for NBA basketball called Wins per 48 Minutes (WP48). The author, David Berri, weighted stats such as points scored, steals, turnovers, and assists based on a regression model of team wins. WP48 estimates each player's share of his team's wins. [I should also note Berri also attempted something similar with QB ratings, called QB Score, with far less success. It weights the components based on a regression and includes sack data. But it contains the same comlexities, redundancies, and other problems with traditional passer rating. In fact, it produces nearly identical results with the traditional rating, correlating at 0.95.]
The new rating system is also a derivative of a baseball stat called the Defense Independent Pitching Statistic (DIPS) developed by amateur sabermetrician Voros McCracken, now employed by the Red Sox. DIPS is a revolutionary concept because it intentionally ignores a large part of conventional pitching statistics. It is based on the idea that once a batter puts a ball into play, the pitcher has no control over whether it is a hit or an out. It is mostly due to luck, defensive positioning, and fielding skills. DIPS considers only those stats that a pitcher has total control over such as strike outs, walks, and home runs. Although DIPS only includes a subset of pitching performance, it is actually more predictive of a pitcher's year-to-year ERA than his actual ERA. It's better because it's a truer measure of pitching ability that ignores the noise of luck and other factors beyond the pitcher's control. The stats that remain are good proxy variables for the ones not included.
This new passer rating system is based on the same concepts as runs created, WP48, and DIPS. Like runs created and WP48, the new passer rating system is computed in terms of a concrete outcome. In this case, it is in terms of team wins instead of team scoring. And similar to DIPS, the new system includes performance measures primarily controlled by the quarterback. Although it is much harder in football than baseball to extricate one player's contribution from the rest of his team, the new system ignores spurious stats such as TD passes.
In summary, the new passer rating:
1. Is not arbitrary. Each component is weighted exactly as much as their relative importance to winning games. These weights are derived from a regression model using data from all teams since the 2002 expansion.
2. Is computed in terms of team wins. The regression model allows the passer rating model's component weights to translate directly into how many additional wins a QB's performance would yield, on average, over 16 full games.
3. Is not redundant. The components do not double count passing stats.
4. Includes only the passing stats primarily controlled by the QB. Factors such as passing yards after catch are not included. Admittedly, this is an imperfect feature of this rating system, but that is due to the team nature of the sport of football. No method could perfectly separate a single player's contribution from his team's.
The three components of the new system are completion percentage, interception rate, and sack yardage rate. In three prior posts, I discuss why I included each one, how they are calculated, and list how the QBs of 2006 rank for each component. The resulting formula of the new passer rating is:
QB Wins Added = (Comp% * 0.18) - (Int/Att * 50.5) - (Sack Yds/Att * 1.57) - 8
Why subtract 8? Because that is the average number of wins achieved by the average QB, and we're interested in 'wins added' above the average performance.
2006 PASSER RATINGS
Now let's look at how the QBs of 2006 fared in the new passer rating system. The total 'wins added' for each quarterback is listed. The 'wins added' for each component of the rating are also listed, along with how each QB ranks among his peers. The traditional NFL passer rating (QBR) is also listed for comparison.
|Rank||Player||+Cmp% Wins||Cmp% Rank||+Int Wins||Int Rank||+Sk Wins||Sk Rank||+Total Wins||QBR|
We see that Manning, Brees, and Garcia top the list, but some surprises follow. QBs such as David Carr, Mark Brunell, and Tim Rattay actually fare pretty well. Each of those three QBs place in the top 10 for different reasons, as you can see by how they rank in each component. Those QBs performed fairly well despite being surrounded by below-average teams.
McNair ranks high at number 7. Although he wasn't starting on anyone's fantasy team last year (16 TDs all year), he led Baltimore to 13 wins by avoiding sacks and keeping the ball out of the hands of the opponent's defense.
One final note: I think this formula is incomplete. You might notice that QBs who tend to throw short passes tend to rank highly. These QBs tend to have good completion % and interception stats. The formula should include this consideration. I think the best way would be to use yards per completion minus yards after catch. The stat would be called "air yards" or something catchy. Unfortunately, YAC totals for quarterbacks are not publicly available--I've searched everywhere. I know it is kept by Stats, Inc., but I can't find it for any year after 2002. If anyone knows where to find QB YAC stats, please leave a comment.