Trying to measure individual performance in football is nearly impossible. Perhaps kickers are the only players whose performance we can isolate from the rest of his team. The position in a distant second might be the quarterback. But measuring QB performance with statistics is still very problematic. Consider this tale of two QBs.
One quarterback led his team to a 12-4 regular season record after starting slow at 3-3. He won the Associated Press's MVP award. Another quarterback was replaced due to ineffectiveness early in the season by an aging journeyman best known for headbutting a stadium wall. Which one would you want on your team? You wouldn't be able to tell from their official stats.
Although one didn't play the entire season, both passers had nearly identical "per attempt stats." The first QB threw for 7.2 yards per attempt and the second QB threw for 7.1. They both had a 95 NFL passer rating. Actually, Tavaris Jackson's was 95.4 and Peyton Manning's was 95. But then again, I might be able to approach a 95 rating if I were throwing dump-offs to Adrian Peterson.
If football were a brand new invention, and we had to decide how to credit the various amounts of yards gained to various players, how would we do it? If I said, "There's this kind of play called a pass, in which a thrower passes the ball to a another player who then runs with it as far as he can. I say we credit all the yards run by the receiver to the thrower," you'd say I was nuts.
I'd say, "Well, it takes a special kind of talent for a passer to get a lot of yardage after the catch (YAC). I won't be able to prove it, in fact, I won't have any evidence for that statement at all, but I still think our primary measure of a passer should include all those yards." I'd be laughed at.
Here are the QBs from 2007 who led the league in percent of their passing yardage as YAC: Croyle, Testaverde, Greise, Harrington, Favre, McCown, Losman, and Lemon. The 2006 list includes Brunell, Carr, Favre, (Rob) Johnson, and (Alex) Smith. There's isn't a single guy on that list who we can call a legitimate starter.
The 2008 season's list of leaders in %YAC include Cassel, O'Sullivan, Campbell, Favre (again), Losman, and Wallace. But Matt Cassel is good, right? Maybe not. Keep in mind how good the team around him was. He was handed the keys to a Ferrari. If a QB racks up his passing yards with YAC, he's either throwing lots of short check-downs and screens, or he has spectacular receivers--or both. Neither is necessarily an indication of a particularly skilled passer.
If we throw away all the YAC and look underneath, what do we have left? I call it Air Yards (AY). It's the distance forward of the line of scrimmage a pass travels. Although it's not a perfect measure of a passer, I think it makes a lot more sense than crediting Donovan McNabb with 71 yards and a touchdown for a 1-yard screen pass to Brian Westbrook.
To be clear, I'm not claiming that a QB has absolutely zero contribution to YAC. The QB has to complete the pass for there to be any YAC in the first place. It's just that the majority of credit assignable between the QB and receiver is due to the receiver. (Much of it can be attributed to the defense and to random variation). Plus, there are better ways of crediting the QB for a completion. Looking at Air Yards at least tells us a lot about a QB that we wouldn't otherwise know. I might be throwing a little of the signal out with the bathwater, but the remaining signal-to-noise ratio is hopefully much better.
Here is how the 2008 regular season Air Yard stats break out. Click on the table headers to sort.