There's this guy in my fantasy league who finished 10th out of 10 teams in terms of total points scored--dead last. But somehow he finished with a 7-6-1 record and nabbed the 4th and final playoff spot. This past weekend was our league's semi-final round, and he won his game by...1 point. A meaningless sack of a 3rd string QB in the final seconds of a 20-point game on Monday night game put his team over the top. Despite managing the very worst team in our league, one more lucky win means he'll takes home all the marbles.

But that's just fantasy football. What about real football? Can a team get lucky like that, where every punt stays out of the end zone, every loose ball bounces into their hands, and Ed Hochuli rules their QB's obvious fumble was an incomplete pass?

Explanation

This is one of my most fun stats--"team luck"--and last year it got plenty of criticism. I estimate team luck by using my efficiency regression model to calculate each team's expected wins--how many wins a team can normally expect, on average, given their actual performance in offensive and defensive running, passing, turnovers, and penalties. The difference between the expected wins and actual wins is what I loosely call team luck.

There are plenty of things my model does not consider, special teams being the most prominent. But special teams plays are the most random events in the sport, save for the coin flip. Luck is a punt that lands on the 5 and skids into the end zone for touchback instead of bouncing into the air and being downed at the 1. A kick or punt return for a touchdown certainly requires skill, but *when* the kick return (or missed field goal or anything else) occurs means everything.

A kick return when a team is already ahead by 20 points doesn't mean much, but when a team is behind by 3 in the 4th quarter, it means the game. Teams and players can't control when those events occur, or else they'd save them up for when they matter most. So in a very substantial way, they are luck, at least when it comes to deciding game outcomes. (For a more thorough discussion, see this essay on NFL luck.)

Lucky and Unlucky Teams

So far this year, the luckiest teams are the Jets, Patriots, Titans, Bills, Broncos and Vikings. Basically, these teams have 2 or more wins more than their stats suggest they should have.

On the other side of the coin are the Packers, the Chargers, the Saints, and the Chiefs. Of particular interest are the Lions, who are on the verge of the first 0-16 season. Normally, a team with their stats would have won a game or two by now.

One interesting thing about this analysis is that Miami's turnaround isn't so miraculous as it seems. Despite their 1-15 season in 2007, their stats indicated they normally would have won about 5 or 6 games, but were unlucky in the extreme. I think their improvement this year is real and substantial, but not as drastic as their record indicates. According to their stats they should be a 9- or 10-win team this year, which would be a 4 or 5 game improvement rather than a 9 game improvement.

The Falcons showed the same pattern, but to a lesser degree. Although they notched only 4 wins, they "should" have won 6. That shouldn't take anything away from their comeback this year, though. In 2008, they continue to be one of the more unlucky teams.

The Charmed One

Looking back at 2007's luckiest/unluckiest teams, one thing stood out as most remarkable. Last year Green Bay was the #1 luckiest team while the Jets were near the bottom at #31. But this year the Packers are the least lucky team while the Jets are the luckiest. So what changed? According some minor media reports, I vaguely recall an obscure quarterback was traded from the Packers to the Jets...

"Ahah!" my brother-in-law and noted Favre lover is saying right now. Your model's residual doesn't measure luck. It measures Favre-ness. You know, that intangible mumbo-jumbo that John Madden blathers on about every other Sunday night.

But my model does include passing, interceptions, and sacks. What it doesn't include are things like J.P. Losman fumbling on a 2nd and 5 on his own 16-yard line with a 3-point lead and less than 2 minutes remaining...which was instantly returned for a game-winning touchdown. If you can make a case that Brett somehow intangibled his way to that win, then I'll gladly stand corrected.

The Full List

RANK | TEAM | GWP | Curr W | Exp W | Luck |

1 | NYJ | 0.42 | 9 | 6.0 | +3.0 |

2 | NE | 0.45 | 9 | 6.5 | +2.5 |

3 | TEN | 0.68 | 12 | 9.7 | +2.3 |

4 | BUF | 0.27 | 6 | 4.0 | +2.0 |

5 | DEN | 0.42 | 8 | 6.0 | +2.0 |

6 | MIN | 0.49 | 9 | 7.1 | +1.9 |

7 | NYG | 0.66 | 11 | 9.4 | +1.6 |

8 | DAL | 0.54 | 9 | 7.8 | +1.2 |

9 | SF | 0.26 | 5 | 3.8 | +1.2 |

10 | ARI | 0.48 | 8 | 6.9 | +1.1 |

11 | IND | 0.63 | 10 | 9.0 | +1.0 |

12 | CAR | 0.72 | 11 | 10.2 | +0.8 |

13 | MIA | 0.59 | 9 | 8.4 | +0.6 |

14 | PIT | 0.74 | 11 | 10.5 | +0.5 |

15 | HOU | 0.47 | 7 | 6.7 | +0.3 |

16 | STL | 0.12 | 2 | 1.8 | +0.2 |

17 | SEA | 0.24 | 3 | 3.5 | -0.5 |

18 | BAL | 0.68 | 9 | 9.6 | -0.6 |

19 | TB | 0.68 | 9 | 9.7 | -0.7 |

20 | JAX | 0.40 | 5 | 5.7 | -0.7 |

21 | OAK | 0.26 | 3 | 3.8 | -0.8 |

22 | CHI | 0.62 | 8 | 8.8 | -0.8 |

23 | CIN | 0.24 | 2.5 | 3.5 | -1.0 |

24 | ATL | 0.72 | 9 | 10.2 | -1.2 |

25 | CLE | 0.37 | 4 | 5.3 | -1.3 |

26 | WAS | 0.59 | 7 | 8.4 | -1.4 |

27 | PHI | 0.71 | 8.5 | 10.1 | -1.6 |

28 | DET | 0.11 | 0 | 1.7 | -1.7 |

29 | KC | 0.27 | 2 | 4.0 | -2.0 |

30 | NO | 0.65 | 7 | 9.3 | -2.3 |

31 | SD | 0.60 | 6 | 8.5 | -2.5 |

32 | GB | 0.55 | 5 | 7.9 | -2.9 |

Brian:

Excellent post. You've got one of the best football sites going. Nice work. I did a post on Bucks Diary that recommends and links to this post (for Packer and Favre fans).

Ty (Bucksdiary.blogspot.com)

Concur w/ TWC. Thoughts about the relationship of "luck" and regression to the mean?

I still disagree with you that the difference between estimated wins and actual wins is "luck". That's too general. Certainly the bounce of the ball is a part of that difference, but so are the intangibles that all players, Favre included, have in their arsenal. One intangible stands out in my mind as much more important than others: focus. Players who maintain good focus during games are more apt to being in the right place at the right time, or recognizing a particular defensive scheme, or keeping a better view of the ball while running a route. None of that is luck, it's all skill...it can be taught and practiced.

Listen to the podcasts I did with several prominent sports psychologists for more info.

“I'm a great believer in luck, and I find the harder I work, the more I have of it.” - Thomas Jefferson

“Luck is what happens when preparation meets opportunity.” - Seneca

"I believe in luck: how else can you explain the success of those you dislike?" - Jean Cocteau

"Shallow men believe in luck. Strong men believe in cause and effect." - Ralph Waldo Emerson

-2.9

+3

Thank you for "proving" favre-ness.

"If you can make a case that Brett somehow intangibled his way to that win, then I'll gladly stand corrected."

Going in to that game Jauron was 9-2 vs. Favre including 8 straight losses.

“Clearly the responsibility for the last call, the play-action pass, that was mine,” Jauron said. “That goes right on me. It backfired clearly and caused us to lose the game."

So, Jauron takes credit for a bad call and the loss. Jauron knew the power of Favre-ness and was doing everything he could to put up another TD to make sure Brett couldn't beat him a 9th straight time. His judgment was clouded by Favre-ness, he made a bad call and lost the game.

Ty-Thanks.

Doug-I would consider focus a tangible: why wouldn't superior focus show up in passing, rushing, and turnover stats?

David-Here's an older article on regression to the mean and team records.

I should note I used the "explanatory" version of the regression model rather than the "predictive" version. This accounts for defensive interceptions and fumbles and does not consider those luck.

I should also note that I don't think that the entire residual is luck, but a great deal of it is.

I've got another one for you, Other Brian:

"Luck is a tag given by the mediocre to account for the accomplishments of genius." - Robert Heinlein

I agree with the point that all too often losers call the winners just lucky. I'm also a strong believer in cause and effect, hard work, preparation, focus, and all that.

But once you account for those things, what do you call what's left over?

And if I'm Jauron, I shouldn't be too worried about the 9th best passer in the AFC!

I too won a fantasy playoff game because of that meaningless sack of a 3rd-string qb in a blowout on Monday night ... but I was the high-scorer in the league (by more points over 2nd place than I averaged on the season) and playing that very 2nd-highest scorer. Our game was in the 120s (high for our league), and the 3rd highest score that week was <100 (99.73).

That sack changed my season from wasted opportunity to possible best-in-this-league's history!

I disagree that the difference between computer forecasts and reality is luck.

The major reason for actual results to be different from model output is that there is something wrong with the model.

There is an old adage that you never swim in the same river twice, and a team is not the same team week to week. For example, the Giants are not the same team as when they had a healthy Buruss and Jacobs. The week to week differences in who dresses, who plays and the health of the players creates a problem for computer driven solutions, and I don't have an answer or a better way to look at the problem.

Another factor not implicit in most models is the "skill" of the coaching staff...how good the coaching staff can find and exploit the opponents weakness, how good is the play calling, how good is the game theory part of the game plan, etc.

And then there is a learning curve for both players and coaches.

There is "luck" component ... truly random occurrences that impact on who wins and on the final score, but they should tend to even out over a season if they are truly random.

The map is not the territory, and the model is not reality. The closer the model is to reality, the less important is luck.

Jarhead-I agree with you almost whole heartedly, except for what proportion of the model's residual is due to unaccounted-for skill, and what proportion is random.

Don't underestimate the importance of luck--in sports and in life. For our entire upbringing, our parents, coaches, teachers (or drill instructors) taught us that our own effort, ability, skill, and preparation determine our outcomes. As actors within the situation, we have no choice. If we shrug our shoulders and say luck can ultimately determine our lot in life, we've become nothing better than...Democrats I suppose.:)

But as observers outside the situation, we can be more objective about just how much random circumstance affects outcomes. Sports offers an excellent opportunity. In fact, in any sport--NFL, MLB, NBA or NHL, we can calculate exactly what proportion of outcomes are due to luck. We just need to compare the variance of the binomial distribution to that of the actual distribution of wins in a league.

The example below is from one of the links I included above, but I know not everyone has the time or desire to read through all of it. Consider this very simple example game:

Assume both PIT and CLE each get 12 1st downs in a game against each other. PIT's 1st downs come as 6 separate bunches of 2 consecutive 1st downs followed by a punt. CLE's 1st downs come as 2 bunches of 6 consecutive 1st downs resulting in 2 TDs. CLE's remaining drives are all 3-and-outs followed by a solid punt. Each team performed equally well, but the random "bunching" of successful events gave CLE a 14-0 shutout.

The bunching effect doesn't have to be that extreme to make the difference in a game, but it illustrates my point. Natural and normal phenomena can conspire to overcome the difference between skill, talent, ability, strategy, and everything else that makes one team "better" than another.

Also, if anyone gets the chance, I highly recommend the book 'Fooled By Randomness' by Nassim Taleb.

interesting article.

One further breakdown on "luck" would be to see if there is any pattern in what type of team is "lucky".

For instance, are top O teams luckier than poor offensive teams?

QB rating v. luck (the favreness is interesting)?

Top D vs bottom D?

Number of pro-bowlers on team vs luck rating?

Plus/minus turnovers vs. luck?

Margin of victory v luck?

etc.

-bob

Yes, this is the question when looking at a model..."what proportion of the model's residual is due to unaccounted-for skill, and what proportion is random." I thought my position was that the difference was not all luck, with no estimate of how much was due to unknown unknowns and how much to luck.

I don't understand you statement "...we can calculate exactly what proportion of outcomes are due to luck. We just need to compare the variance of the binomial distribution to that of the actual distribution of wins in a league." Perhaps I don't know enough stats, but I don't get that statement.

I do agree that luck or random events (and strings of random events) are very influential in real life. And I have been very lucky in many ways. LOL

By the way, the take away point for me in Taleb's book is that people often over rely on the certainty of models and statistics in decision making, and under estimate risk and randomness.

Right. In the real non-sports world, the 'r-squared's of our own personal models of things are really small, and the residuals are very large. Taleb's point (as I took it) is that randomness (or luck) is a very large part of that residual.

We also overestimate the r-squareds of our models of things. We think we can predict things well but we really can't. We fool ourselves into attributing success or failure to things we can control.

But unlike most of life, sports feature controlled settings with level playing fields and carefully measured causes and effects. Sports are like a controlled statistics laboratory where the r-squareds can be very high, and the residuals are relatively small.

I think we agree on Taleb's book.

Financial markets, as well as sports, are some what artificial and controlled settings with many known and measured relationships, and many high r^2 and many very smart people with very big computers working on forecasting. Many of the models have blown up due to under estimated risk and unknown unknowns, they did not blow up due to random events.

As for high r^2 in sports, it depends on what you are forecasting. I think you said you had about .80 in a regression where the independent variable was win or loss. I think that if the independent variable is margin of victory, then the r^2 is a lot lower, and if you try to forecast the score for each team, the r^2 is even lower.

The same principle goes for horse racing... a model predicting the winner will have a higher r^2 than one predicting the exact order of finish. It depends on how the problem is framed and the model set up.

I still don't understand your statement that "...we can calculate exactly what proportion of outcomes are due to luck. We just need to compare the variance of the binomial distribution to that of the actual distribution of wins in a league." Can you explain that in a way that a slow student can understand?

Sure. I should probably do a full post on this, but here is the quick and dirty. This is Tom Tango's method, not mine.

We can say the outcomes we observe in any sports league are driven by two components—the true skill and ability of each team, and the randomness of the sport. The variance of teams wins would therefore be the variance true ability plus the variance of luck.

Var(observed) = var(skill) + var(luck)

Let’s assume every NFL game is decided purely by luck--a 50/50 coin flip. The distribution of team wins, from 0 to 16, would be the binomial distribution, similar to a bell curve, with lots of 7, 8, and 9-win teams and very few 0, 1, 15, or 16-win teams.

The statistical variance of the binomial distribution is the square of its standard deviation:

Var(luck) = (0.5*0.5/16) = 0.0156, where 16 is the number of games.

We can compute the total variance of skill and luck from the actual observed variance in the real distribution of NFL games. From recent years this is:

Var(obs) = 0.0361

Therefore, Var(skill) = Var(obs) – Var(luck), which is:

Var(skill) = 0.0361 – 0.0156 = 0.0205

R-squared is basically the percent of variance explained by the independent variables. In this case, the r-squared of skill is:

r-squared(skill) = 0.0204/0.0361 = 56.7%

which means the r-squared(luck) = 43.3%.

But remember, half of the time that the effects of luck overcome the effects of skill (that is, luck is the decisive factor) the “better” team will still win. (A team can be both good and lucky). So, the better team should win: 56.7 + 43.3/2 = 78.4% of the time.

Throw in home field advantage, which complicates things a little but is obviously known, and we get something over 80%.

Tango’s post is here.

I did a round-about way of estimating this here.

Thanks for the explanation

Care to revisit your Belichick and cheating argument from last year? This makes the second year post-scandal that the Patriots are 2 wins over expectations. I would think it would be worth your time and a demonstration of your ethics to revisit something that inflamatory.

David-You're right. I noticed that too. Perhaps Belichick really is a true genius. The fact that his team has gone #1 and #2 in 2 consecutive years since the cheating revelations moves the needle further away from "cheated his way to greatness" and closer to "genuine genius."

But I don't agree that my analysis was all that inflammatory in light of what came out since I wrote that article. We now know that the rule-breaking (or cheating or however you want to describe it) was fairly pervasive and was conducted as far back as his arrival at New England. It was never honestly investigated and the league destroyed the evidence.

He did it. That much can't be debated. How much of an edge it gave his team is really the only question. But the fact that Belichick continues to give his team an advantage since the spotlight was shone on his methods lends credence to his defenders.

Also, I never claimed he wasn't a good coach. It's not an either/or question. He can be both very talented and a cheater. And I'd say he's been a bit lucky too. Usually when someone or something is such an astronomical outlier, there are several factors all contributing simultaneously.

Inflammatory was perhaps a poor choice of words, but generally anything around that scandal could be described that way. Passions run high.

As a new reader, I'd still like to see something on it.

Please ignore the idea of number of pro bowlers on a team vs. luck. How can any measure of Pro Bowlers be taken seriously when the fans don't pay attention to who deserves to go Vs. name recognition. Abraham of the Falcons being snubbed is a prime example of how bad the pro bowl picks can turn out.

Doug-I would consider focus a tangible: why wouldn't superior focus show up in passing, rushing, and turnover stats?Well surely it would but I think the claim is that "focus" would disproportionately affect the performance at critical points in the game. Any good player can play well when the game plan seems to be working. But in the final few moments or when unusual opportunities present themselves some players get thrown off their game by the pressure.

It sounds reasonable, but it has to be tested. You need some way to distinguish critical points of the game that players can be aware of versus critical points of the game they're utterly incapable of perceiving.

The only idea I have is if you find games that were decided by only a few points and then look at plays earlier in the game when they wouldn't know that would be the eventual outcome. Compare that to games that were blowouts looking at plays earlier in the game when the score was still close.

Alternatively -- what fluctuations in luck from season to season would the mathemetics predict? Are the fluctuations you see here, including the -3 to +3 fluctuation within the expected range? Or is it a sign you're missing something in your model?

Experience with poker shows people have a *great* tendency to find patterns in random chance. Lots of players seem to have a knack for outdrawing against great odds. Of course the good ones never believe it -- though they may pretend to to psyche out their opponents.

I would agree that people have a *great* tendency to

believethey can find patterns in random chance. But they are fooling themselves.Even if something is 100% chance, there will always be some people will be in the right tail of the curve. Millions of people probably play poker, and there are going to be quite a few who are significantly luckier than the rest of us.

Go-Fish is a game of luck too, and someone out there is the 'luckiest' go-fish player ever. But he has no more of a knack for it than anyone else.