Is "Red Zone" Performance Real?

Team performance in the red zone is obviously very critical. Teams can make up for a lot of deficiencies if they can move the ball inside the opposition's 20 yard line. But is the red zone real? Certainly, it's tougher to move the ball efficiently inside the 20 because the field is compressed and the defense has less territory to protect. As a consequence, outside the red zone the 2007 league average pass completion rate was 63%, but inside the red zone it was only 56%. So in that sense, the red zone is real.

In this article, I'll compare 2007 quarterback performances inside and outside the red zone. After testing the differences statistically, we'll see if some QBs really have a knack for success inside the 20, or if we're just witnessing the randomness of a small subsample of passes.

Football experts saw a correlation in red zone performance and scoring, and quickly focused a lot of attention on how well a team does inside the 20. It does make sense, to a degree. Teams that are otherwise efficient and can move the ball, but don't put it in the end zone won't score and won't win. A quarterback whose stats are particularly good in the red zone is considered a clutch performer and is believed to have some special ability to perform when it counts most. Likewise, a QB who under-performs in the red zone is seen as lacking what it takes to succeed in the NFL.

"My theory is that the very same abilities that lead to success in the other 80 yards of the football field also lead to success inside the 20."


But what if red zone performance was really just a random subset of overall performance? If we arbitrarily divided the field into any other 20-yard segment and analyzed performance, would we find that some QBs have a special talent between the 40s? After all, pass attempts inside the red zone comprise only 13% of all passes.

My theory is that the very same abilities that lead to success in the other 80 yards of the football field also lead to success inside the 20. A QB who is accurate, aware, can read defenses, and has a strong arm will likely do well in any part of the field. Someone would have to convince me that there is some special talent that becomes more important in the red zone.

Some might say that abilities or flaws are magnified in the red zone because of the compressed field--the density of pass defenders is much higher. If that's true, we should see quarterbacks with good stats do especially well, and below-average quarterbacks should do especially poorly in the red zone.

But that's not what we find. The #1 QB in completion percentage in the red zone last year was Sage Rosenfels. Outside the red zone however, he ranked 20th. Trent Edwards was 29th outside the red zone, but 3rd inside. Looking at the rankings, they seemed very random.

Although completion percentage is generally not the best measure of QB performance, it might be particularly relevant in the red zone for several reasons. Yardage measures are problematic because passes thrown into the end zone are truncated at the goal line. Efficiency measures may not make sense because a 2 yd pass completion from the 2 on 3rd and goal is more meaningful than a 4 yd completion from the 18 on 3rd and 8. Over the course of an entire season and across a full 100-yard field, yards per attempt might tell us a lot more than within a select set of passes near the goal line. In general, a completion in the red zone is always better than an incomplete pass or an interception. Close to the goal line, there isn't often a better or deeper option--any completion is good.

To test whether red zone completion percentage is a special talent or simply a random subsample of overall completion percentage, I used a statistical test known as a t-test. Those familiar with statistics and regression know this is the test that indicates if a variable is significant or not. But it can be used on its own without a regression to test if observed differences are due to a systematic effect or just due to random variation or sample error.

The t-test produces a probability that the observed differences are really just due to chance. Generally, a p-value below 0.05 means a variable is significant--there is a 95% chance there really is a systematic connection between variables. For example, if Matt Cassel completed 4 out of 5 passes in one series of relief for Tom Brady, does that make him more accurate than Brady's 398 for 578? The t-test says no--the sample size is too low and the difference is not big enough to conclude Cassel is more accurate than Brady.

So I performed a t-test on each quarterback's completion percentage inside and outside the red zone. But there was a wrinkle. We already know it's tougher on every quarterback in the red zone because of field compression. We're really interested in whether some QBs significantly over- or under-perform their overall completion percentage. To compare apples to apples, I used a special version of the test that accounts for known expected differences in the means of each group. The NFL average completion percentage is 7.2 points lower for the red zone than outside the red zone, so I used 7.2% as the difference in means.

The table below lists the leading QBs in completion percentage. Their completion percentage in the non-red zone (NRZ) and inside the red zone (RZ) is listed. Next is listed a "value over average" (VOA) number that indicates how a QB over- or under-performed his overall percentage when inside the red zone, accounting for the general increase in difficulty inside the 20. A high positive number means a QB did especially well in the red zone compared to outside it. The final column is the result of the t-test. A number below 0.05 is generally considered significant.

Click on the table headers to sort.



































QBTeamNRZ PctRZ PctRZ VOAT-test
Anderson58.145.5-10.60.43
Boller62.055.3-0.80.75
Brady70.661.65.50.86
Brees66.971.415.30.04
Bulger57.864.78.60.12
Campbell61.052.1-4.00.84
Clemens53.939.4-16.70.62
Cutler64.456.90.80.98
Edwards54.867.911.80.04
Favre67.659.43.30.93
Garcia65.053.3-2.80.63
Garrard65.158.92.80.81
Griese63.446.7-9.40.34
Harrington62.850.0-6.10.56
Hasselbeck63.755.4-0.70.90
Huard62.656.70.60.89
Jackson59.544.0-12.10.42
Kitna64.156.10.00.91
Manning67.356.0-0.10.55
Manning56.056.80.70.19
McNabb63.947.8-8.30.19
Palmer65.859.33.20.85
Pennington70.754.8-1.30.35
Rivers62.447.8-8.30.28
Roethlisberger66.958.32.20.91
Romo66.153.6-2.50.86
Rosenfels62.676.920.80.04
Schaub68.053.3-2.80.43
Warner62.958.32.20.67
Young63.650.0-6.10.46


Note that there are in fact three QBs with statistically significant differences in completion percentage inside the red zone. Brees, Edwards, and Rosenfels all had significantly better than expected performance in the red zone. No QBs were significantly worse than expected. So this is proof that those three QBs have a special ability to perform in the compressed field inside the 20, right?

"It appears that despite all analysis to the contrary, there is nothing special about any particular quarterback's ability inside the red zone as compared to outside the 20."

Not exactly. The table above lists 30 of the league's leading passers, and we would expect a handful of QBs to appear significant just by chance (this is known as a "Type I" statistical error). Further, the distribution of t-test values is evenly spread from 0.93 to 0.04. There is no bunching of values toward the significance level of 0.05.

From this, it appears that despite all analysis to the contrary, there is nothing special about any particular quarterback's ability inside the red zone as compared to outside the 20. It's simply a random subsample of overall performance. If a QB is a good passer, he'll probably be good inside the 20.

If this conclusion is correct, it should adjust our view of some of the QBs in the table above. (Sort by the red zone value over average (RZ VOA) column to see the most over- and under-performing QBs.) Take Trent Edwards for example. His over-performance inside the red zone probably makes him appear to be more productive a passer than he truly is. He's not likely to over-perform inside the 20 in 2008 like he did in 2007. Chances are Buffalo fans will be disappointed next year.

The opposite could be said of QBs such as Kellen Clemens. He under-performed in the red zone in 2007. Although his overall numbers aren't terribly great, chances are he will not under-perform inside the 20 as severely as he did last season. It's not a guarantee Clemens will improve, but he would likely trend toward his overall performance level and not repeat his dismal 39% completion rate in the red zone.

This result also means that prediction models and handicappers that overweight red zone performance are reducing their predictive power. They are chasing the past randomness of a small subset of performance. The same is probably also true of the myriad of stats "splits" such as "under the lights" or "road division games " or any other silly and arbitrary classification.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

15 Responses to “Is "Red Zone" Performance Real?”

  1. Tarr says:

    This makes for a nice companion piece to your 3rd down performance analysis, although the approach is not quite as conclusive, due to this being based solely on completion percentage. It's still possible that there are some QB skills that are relatively more impiortant in the red zone, and these skills are not reflected in completion percentage.

  2. JG says:

    "The opposite could be said of QBs such as Kellen Clemens ... chances are he will not under-perform inside the 20 as severely as he did last season"

    Very probably, yes, but this may not be quite as certain as that players who over-performed by as much won't do it again.

    In this regard, FWIW, I'll just mention Bill James's observation that in baseball's stat categories where overperformance is chance or luck, underpeformance does sometimes to some degree reflect (lack of) skill. The situation need not be symmetrical on the up and down sides.

    One example he used for which there is a ton of data is one-run baseball games. He found that teams that won a lot more than their share of one-run games during a given period of time had zero ability to persist in doing so during a subsequent period, indicating the winning stretch reflected just luck, not any ability to win one-run games.

    However, teams that lost far more than their share of one-run games *did* show some persistance in continuing to lose them thereafter, indicating their under-performance wasn't entirely bad luck.

    It seems to be a "Red Queen competition" thing. The talent level in pro sports is really very competitive and fairly equal overall. Teams and players have to compete like all get out just to stay even and not fall behind. So in situations that are very tight and turned by small things influenced by chance, it is very difficult to get a systematic advantage over the rest of the league. Even a good team will win no more in those situations than luck allows. But if your team is outside the normal distribution with *really bad* talent and/or coaching, it's a lot easier to demonstrate systematic inferiority to the rest of the league, which is visible even in close situations highly influenced by chance.

    Still, James said the great majority of losing in one-run games is merely bad luck. And relating all this to "red zone" NFL QB play is a bit of a stretch. So it may not be worth mentioning here, I admit.

    I do so *only* because you mentioned Kellen Clemens, and I am a decades-long suffering Jets fan. When I watched him this year he looked dreadful. The NFL rating system had him 33rd in a 32-team league, Football Outsiders ranked him 43rd, etc.

    As a Jets fan my gut fully expects our QB of the future to be the *one* NFL QB to ever indisputably embody James's "losing in clutch situations can indeed be a skill".

  3. Brian Burke says:

    Good point that the effect may not be symmetrical. Hadn't considered that. I'd need more years of data to examine that.

    Unfortunately, I'm limited to the data that's available out there. I'd love yds per attempt in the red zone, or using play action, etc. I used data from myway.com (http://sports.myway.com/nfl/stats.html) which is one of the more comprehensive sources for stats. It's where I get by YAC data for my QB rating.

    And speaking of James, it's important that I point out that my analysis here does not prove the non-existence of clutch red-zone performance. It just shows that we almost certainly could never detect it by what we observe. Just like clutch hitting in baseball, it either does not exist, or the effect is so small that it doesn't really matter.

  4. JG says:

    "my analysis here does not prove the non-existence of clutch red-zone performance. It just shows [if it exists] the effect is so small that it doesn't really matter."

    You're right, which means it doesn't exist for any practical purposes, and you really do show that.

    The "asymetrical downside" is just a theoretical nit, even in baseball. Any NFL QB who's really that bad won't be around long enough for anyone to collect a meaningful data set on him, so to speak. Sorry I mentioned it.

    And yes, since James landed a spot on the Red Sox payroll and has been so impressed by Big Papi he's kind of backtracked from "there's no evidence of clutch hitting's existence" to "there's not enough evidence to fully disprove clutch hitting's existence", which may be technically true, but really, for any practical purpose...

  5. Pacifist Viking says:

    I still wonder if at some positions, certain players are more suited to perform well in the red zone than others. No, not because of a "clutch" ability in their psychology. I would ask, "What is fundamentally different about football in the red zone and football outside of the red zone?" The practical answers include less space, no threat of a deep pass, and space limitations at the back of the end zone. Defenders, then, have less space to fill, and don't have to play safe for a big play, and thus everything is more difficult on all offensive players. The back of the end zone, however, allows some plays that offenses wouldn't run elsewhere on the field because there is no safety over the top.

    So, are there certain WRs, TEs, and RBs that are better suited to perform well in a situation where more defenders are clogging space, but new plays are also opened up due to the back of the end zone? I'm not sure. If there are such players, it's not because of cluth ability, I think, but because they have physical attibutes suited to the different situation (size, strength, leaping ability, etc.). Of course those same skills have usefulness all over the field, so such players might not stand out as "extra" good in the red zone. Perhaps the red zone, then, brings certain limitations to certain players who are effective elsewhere on the field, more than it brings advantages to certain players who are suited to the red zone.

  6. Brian Burke says:

    PV-I think you nailed it. Height and jumping ability are probably the key attributes for red zone/goal line passing success. Speed may be the more important attribute outside the red zone. A tall QB and tall receivers might be the key.

    I wonder if we'll come to a point where all teams will use highly specialized red zone personnel. They would just be gargantuan guys without much speed--basketball centers. If I were a GM I'd invite a few of NCAA centers to camp to see what they could in the back of the end zone against 6-ft tall safeties and 6-ft-3 linebackers. Even if they don't pull down the ball, they might draw a lot of interference calls.

  7. Joey says:

    My background hardly compares with yours so I am not going to dispute the results of your statistical reasoning. BUT as they say, figures lie and liars figure. There is something amiss here.

    Looking at team like the Eagles and Donovan McNabb, you can see they are highly productive in the large area of the rield and less so in the more confined and close Red Zone. McNabb can't get around so easily in that space and has to rely totally on his passing and reading skills which often come up short. That's why they can be6thin the league in offensive plays but 12th in points. Too many field goals.

    Forget the numbers. What is needed inside the restricted area known as the Red Zone is quickness. You need to read more quickly, react more quickly and move more quickly to get free because the defenders are on you immediatley. And there is increased pressure.

    Quarterbacks that can handle those circumstances do well. Those that cannot, do less well. A quarterback with great numbers outside the Zone may have okay numbers inside the zone and still not be very good because he starts out with so much. Marginal quarterbaks perhaps have less to work with and any loss of ability under the pressure of the Red Zone might just be too much to easily assimilate.

    To me, numbers aside, some guys perform well in that tight space and others do not. Great QB's perform well everywhere on the field and the Red Zone is no exception. They just dial up what they need and move on.

    But others don't have enough in the tank to be able to dial up more when more is needed. So they fail.

    Football in the Red Zone - in my opinion - is much different football than football outside the Red Zone.

    My name is Joey and I approve of this message - right or wrong lol

    Go Birds.

  8. Derek says:

    Joey, the problem with your analysis is that you only took the 2007 stats Brian listed here. If you go back and look at McNabb's red zone stats for a few more years, you'll see that historically he's been quite good down there.

    Notwithstanding the occasional ball thrown well short of the end zone with five seconds left and no timeouts...

  9. Anonymous says:

    I think the best argument against red zone performance as part of a players skill set is Jerry Rice.

    One year he sets the record for most TD catches (21 or something?), the next—in more games—he catches only 9.

    Any argument crediting Rice with skills resulting in TDs must also ask if he just forget how to catch them next year.

    It's like the oft-used "That guy is a winner." I guess, but what is he if they lose?

  10. Anonymous says:

    I think you should also keep in mind that you haven't adjusted for the number of tests you've performed. So even with those p values you did find with a p = 0.04, there's a 4% chance it's occured due to chance, and since you've performed so many t-tests there's actually a solid chance that one of your "significant" results is itself a result of random variation.

  11. Brian Burke says:

    That's my entire point.

  12. Anonymous says:

    Why not use a Bonferroni (sp?) correction? To account for the number of tests your running, simply divide your alpha level by the number of tests you run and use that as your acceptable alpha level. It'll give you an extraordinarily difficult level, but that unfortunately comes with the territory when you run 30 independent t-tests.

  13. Eric Moore says:

    I think "Chances are Buffalo fans will be disappointed next year" should win the IgNobel Prize for Statistics!

    (Note: I love reading this site, so that's not meant as insult. But with the Bills, if we don't laugh...)

  14. Anonymous says:

    Hey Brian, I just stumbled on this site and have been rather impressed with your content. I'm slowly working my way from the beginning to current day now....

    You mentioned the difficulty in using yards per attempt as a predictor. Did you consider using a metric that measures performance as a % of the distance to goal?

    For instance, a 5 yard pass on the 20 would be equal to a 1 yard pass on the 4 (25% of the distance to goal). Then you could take the average per attempt.

    By the taking the avg per attempt, you'd also get an idea of how long it took to get into the endzone. A perfect score would be 1 (1 TD pass per attempt).

    Obviously this analysis was from several years ago, and it'd be a bear to get the required stats. Just wanted your thoughts.

    Keep up the great work. I look forward to reading the rest!

  15. Busto says:

    I realize this is an old article, but I was looking for year over year red-zone performance for quarterbacks, as a way to prove this same conjecture, and I stumbled upon this article (and you're imo superior approach).

    I have been very interested recently in quantifying luck football, and this piece helps a lot to that end.

Leave a Reply

Note: Only a member of this blog may post a comment.