Last season you might recall a dust-up between Harvard evolutionary psychologist Steven Pinker and popular science author Malcolm Gladwell over whether teams really have any ability to predict which college QBs will pan out into good pros. You might be wondering what the heck a psychologist and a pop-science author have to do with NFL football.
In his book What the Dog Saw, Gladwell wrote about how hard it is for school administrators to discriminate the better teacher candidates from the lesser candidates. Gladwell used the NFL draft to illustrate how difficult it is for anyone to predict human performance, even in a sport where there is ample performance metrics and every step, throw, and catch is videotaped from 12 different angles. Gladwell was referring to what was reported by economists Dave Berri and Rob Simmons as a "very weak" correlation between draft order and per-play performance by QBs.
In an exchange of letters following Pinker's critical review of What the Dog Saw, Pinker took issue with Gladwell's claim that there was "no connection" between when a QB is taken in the draft and his per-play performance. Pinker wrote that this is "simply not the case."
As has been pointed previously, the problem with the weak correlation cited by Gladwell is that it excludes players who are not judged good enough by coaches during their development to warrant much if any playing time. At its core, the NFL draft is a process of selection, and we should expect selection bias will taint most attempts at analysis. Gladwell looked at the draft process and (correctly) said:
"Coaches and GMs turn out to be good decision-makers when it comes to drafting quarterbacks when you consider the fact that the quarterbacks who never played aren’t any good. And how do we know that the quarterbacks who never play aren’t any good? Because coaches and GMs are good decision-makers!”
But Gladwell's argument cuts both ways. The only way to see that coaches and GMs aren't any good at drafting QBs is to assume they're no good at choosing which QB on their roster to play in games!
In this post I'll attempt to settle the question of whether NFL scouts really have any ability to identify the better QBs. Do the QBs picked higher in the draft turn out to be better performers on a per-play basis? Is Pinker correct that they do, or is Gladwell correct that they do not?