By Brian Burke
Massey has continued research into the draft. His presentation at the 2012 MIT Sloan Sports Analytics Conference outlines his recent findings. (I recommend using IE to view the presentation. Chrome didn't play nice with the video.) The slides from the brief can be viewed here.
If I understand things correctly, Massey has found that:
1. The draft requires skill, as demonstrated by virtue of the superior performance of earlier picks to later picks.
2. Teams are not significantly better than others at the draft, largely due to the fact that:
3. Randomness dominates the process. It is an exceptionally uncertain task to predict human performance years into the future.
4. Therefore, teams are (statistically) roughly equal in ability to draft talent.
This makes sense in a lot of ways. All teams have equal access to information. Personnel talent is diversified and highly mobile across the league. Draft techniques and strategies can be observed and copied readily. Yes, it's true that if a clown were made GM of a team the draft would not appear random. But because team personnel staffs are not clowns (perhaps except a couple), the draft is effectively randomized in terms of successfully selecting quality players.
Massey goes on to point out specific strategies that are good bets in any highly uncertain environment, including the draft.
1. In a random environment, the best thing you can do is increase your opportunities, i.e. number of picks. (Stop trying to milk a 1 in 100 chance into a 1 in 90 chance. Go get yourself two 1 in 100 chances instead.)
2. Minimize the costs of choosing. (Don't pay for the right to think you know more than everyone else.)
3. Don't trust any one opinion. Trust the consensus.
4. Success is largely luck. Failure is largely luck. Success and failure should be evaluated in terms of the soundness of the process rather than outcome. Don't allow outcomes to be attributed to individuals.
I generally agree with the results and their implications, but I have two comments about the Massey-Thaler methodology.
First, a lot of the performance value of each player is highly dependent on opportunity. That is, players drafted highly get more starts, more Pro-Bowl opportunities and other performance achievements merely due to the opportunities granted to them by virtue of when they were selected. It's a causation-direction problem. Are young players successful solely because they are good, or also because their teams have sunk varying levels of investment in them? I believe performance achievements in the Massey-Thaler methodology are mostly due to real player talent, but some degree of sunk investment is involved.
Second, what about the Gladiator effect? High premiums on the very top picks might be justified by the fundamental rules of the sport. An All-Pro QB can't be replaced on the field by two mediocre ones. There are limited numbers of roster spots available, so having multiple opportunities at the expense of top picks might be a mistake. I'd suggest that there's a happy medium, maximizing the number of picks near the peak surplus value at the end of the first round, or wherever the new peak surplus is based on the current CBA.