Play Calling on 3rd and Short Part 1

It’s third down and one yard to go. Should the coach call for a run or pass? We know that according to game theory (and from simple experience), it’s best for a coach to call for a randomized mix. The question is, what is the optimum mix? In other words, what are the best run/pass ratios in 3rd down situations? We’ll see if coaches are making the right calls on 3rd down, focusing on 3rd and 1 as a case in point.

Game theory tells us that, at the optimum mix of strategies, the average utility of both strategies will be equal. The optimum mix of strategies is also known as the Nash equilibrium. In football terms, that means that on 3rd and 1 yard to go, if a team is getting a 1st down 60% of the time when running and 30% when passing, that team is passing too frequently.

If this just sounds like theoretical mumbo-jumbo, and you don’t buy that the payoff for running and passing should always be equal at the optimum ratio, think of it this way: If the conversion rate for running is higher than that for passing, why pass at all? The answer is obvious enough—If offenses only ran, defenses would stack the line with all 11 defenders and the success rate would plummet. So the best tactic would be to run more often until defenses counter your tendency. Defenses would be forced to commit to run defense more and more until passing becomes more successful. When the conversion rate for running and for passing equalize, that’s the Nash equilibrium where the overall conversion rate will be highest.

A look at conversion rates across the NFL for 3rd downs at various ‘to go’ distances initially suggests offenses pass too often on 3rd and short. Notice how the conversion rate is higher for running than for passing for short yardage situations.

On 3rd and 1, teams are successful in 70% of attempts when running, but in only 58% of attempts when passing. Not until we get to 3rd down and 5 yards to go do the conversion rates completely equalize.

Except for the 3rd and very short situations, I’m struck by how close the conversion rates for running and passing are. Running and passing were equally successful for 3rd down and mid- to long-yardage situations. For example, for the 1011 3rd and 7 situations in the past 8 years, both running and passing were successful 38% of the time. Teams are running in that situation only 12% of the time, so defenses are usually expecting a pass.

Running therefore becomes far easier than if teams ran more often on 3rd and 7. Whether they realize it or not, NFL coaches are doing an excellent job finding the Nash equilibrium, at least on 3rd and 5 or more. The graph below plots the NFL-wide 3rd down run/pass balance for various ‘to go’ distances. Passing is the preferred option at longer distances for obvious reasons.

Converting a 1st down is not, however, the only measure of success. It’s always better to have a 1st down and more yards (except on 3rd and goal). So passing seems to offer an advantage over running because its average gain is longer. On the other hand, passing adds the slight possibility of an interception. I'll examine those considerations in part 2 by comparing the average gain for both play types and by analyzing the expected points scored after run attempts and after pass attempts on 3rd down.

Note: The data are from all regular season 3rd down run and pass plays from 2000 through 2007, except for those within 2 minutes of the end of a half and all plays from within field goal range defined as inside the 35 yard line. I excluded late minute plays to exclude situations where teams were tending towards the pass for clock management purposes. Plays within FG range could skew the results because teams may make non-optimum decisions knowing they can kick a FG if unsuccessful. Conversion rates include 1st downs due to penalties. One shortcoming in the data is that QB scrambles are counted simply as runs. However, scrambles comprise a very small minority of plays, so the general conclusions here should not be affected.

15 Responses to “Play Calling on 3rd and Short Part 1”

1. Anonymous says:

love the post. question though--doesn't the notion of reaching the equilibrium assume that other team's properly adjust to your team's tendencies? In other words, suppose your team ALWAYS runs on 3rd and 1, and other teams DON'T adjust to this tendency. Then it seems like you are making the right move since you are exploiting the advantage that running has in this situation.

(This is just a theoretical question here--I am ignoring the other issues such as expected yardage gain and expected points for that drive)

2. Brian Burke says:

You're absolutely right. Theoretically, the best thing for an offense to do to a defense that is over-defending the pass is to run every time.

But even the dullest defensive coordinators would realize the tendency and react. That's why I would suggest running more and more often on 3rd and short until I started to see the defenses respond.

3. Mr.Ceraldi says:

hi Brian;
this comment is in regard to your post regarding the effectiveness of using a random strategy in play calling.

it seems to me this would also apply to pitchers in baseball and pitch calling..

in fact after one of the playoff games this year the hitters on the losing team were complaining that they couldn't hit the pitcher because hispitch selection was completly unpredictable,there was " no pattern" or "sense to the pitch selection"
i thought this was an interesting example as to the effectivness of instituting a random strategy to an enviroment nowadays, in sports coaching which is obsessed with predictablity, and intellectual prior preperation.

As you know I am engaged in the general issue of random/luck in sports (primarily NHL hockey)in regards to scoring
however, once again you have opened up another topic of interesting debate and questioning extending the impact of randomness to game strategy.thanks for another great post!

4. Brian Burke says:

Yeah, I was thinking along the same lines watching the baseball playoffs last night. One of the announcers/commentators was saying that the next pitch should be a "fastball, slightly high and inside. That's the only right pitch."

If that's true, then the batter knows this too, so this is the exact wrong pitch. And around we go.

Therefore, you can't do better than to randomize, which means the batter has to "look" for pitches in a random way too.

I'm sure it's the same in hockey--skates left or right, passes back or forward, goes for the 5-hole or for a high slap shot, etc. The defender has to guess.

And that means, a huge part of all these games really are random. People debate me about luck all the time. It really spoils the whole determinist and romantic narrative of destiny in sports that a lot of people buy into.

If we could have the 2007 Patriots replay the 2007 Giants 100 times, the Patriots would have won 75 or so of the games. We were just treated to one of the 25 games in which the Giants win.

5. Mr.Ceraldi says:

Great comments Brian..I have the same discussion with my buddies all the time who likewise refuse to acknowledge the role random/luck plays in sports outcomes ....while its somewhat easier for the 'masses' (who buy into the 'romantic narrative'...often for a personal or collective psycological need for MYTH..)to accept when the underdog wins by luck (like the Giants) it is almost impossible for them to 'get' that the favorite wins often because of randomness as well! I postulate that the lack of acknowledgement of these % of games (where the favorite wins by luck )
leads to an overall distortion and reduction of the true impact of randomness
How many times have you heard a commentator
state when a favorite wins by luck? (" they were the better team anyways", etc etc)
In other words almost always when the expected team wins it is attributed to their skill advantage alone, regardless of the objective facts as to how they won?(random penalties called, random turnovers,etc ditribution of first downs etc

Also even when the underdog wins often masses will attribute it to poor play by the favorite, poor preparation etc etc.

I have mentioned that Tango has a formula for calculating the amount of luck involved in the outcome in each sport it is astonishingly high

(For example last year in the Stanley Cup)
Detroit won the cup and universally were
celebrated as the best team etc etc.
However, in the first two games there were a large number of 'suspicious' non penalty calls
especially early on in each game
(In fact the TV ran all the evidence in a montage)
These non-calls were a result of good fortune or luck for Detroit (or league intervention if one is a conspiricy theorist) there is no skill involved in having refs miss penalties
Of course the commentators brushed off Pittsburgh's complaints as whinnng.

in any event the average hockey game only has 5 goals (unlike football which has I believe an average of 9 scoring events?) so the margin of error between teams is the slightest - scoring is so difficult and rare that luck is a huge factor
Using Tango's formulas I calculated that skill
may only account for at most 20% of each outcome)

In a related non- sports topic one of the best and most controversial articles I have eve read was from a russian mathametician (his name escapes me right now)who argues that global warming cannot be proven because predicting weather over any significant period of time is not possible there are simple to many random events and any attempt to do so is humans simply projecting order or pattern over random events(for there obsessive need for control)Of course the alarmists do not wish to here this

P.S. Great outright washington pic (+11)

stunning actually !Boy the effects on hyperbole media hype is alive and well
even to a pragmatic statistican as myself!
I simply could not follow your objective evidence and thus lost my weekly office pool by one game ARGH!

6. Sasha says:

I like your article, but why do you assume that defensive play calling is not random? What I mean is that you only talk about offensive play calling adjusting to defensive tendencies, but what if the defensive calling is also random? I guess what I am getting at is the objective question of what is better to run or to pass in 3rd and short does not depend on logical actions of either side. Or at least should not depend on that. Instead what is interesting is looking at what is more probable (i.e., in a random situation what is more probable to get a first down running or passing?) Intuitively it seems that running is easier since there are less "things" that have to go right....

7. GIEFF says:

YI agree with Sasha. This is a flawed analysis, beacuse you are ignoring the knowledge gained by both sides immediately before the snap (e.g. the defense has 9 in the box, or the offense has 4 wideouts). Decisions are not made in a vacuum - the defense doesn't need to justinformation about THE CURRENT PLAY.

Completely theoretically, I agree - defenses should be more keyed in to stop the run on 3rd and short, and offenses should run more. But what about situations where the offense sees 9 guys in the box? Or where the defense sees a 3-TE set and decides to crowd the line? I'd bet the lines are a lot closer in these situations, if not reversed.

8. Brian Burke says:

What the article says is that *given* the recent historical mix of defensive strategies, the run is underutilized. And *given* the pre-snap knowledge gained by both sides, offenses are currently too biased toward the pass.

It does not matter when the call is made--at the line, from the sideline, or in the coach's office a week before the game. The fact remains that running is more effective than passing on 3rd and short, and offenses should do it more often.

You're also mistaken that defenses should be more "keyed in against the run." That would be a big mistake given the current strategy mix of NFL offenses. In fact, defenses should bias more toward defending the pass unless and until offenses begin exploiting the run more often.

9. Happy says:

I loved this article (as I see many of your readers did). Of course the real situation is more complex since a 1st down isn't the only possible outcome. There is also the possibility of a long play (touchdown) and a turnover or a sack.

I have noticed one tendency that should weigh even more heavily in favor of running on 3rd and short; when defenses stack the line to stop a 1st down the runner is often able to score if he can get through to where the secondary should be. I don't have any statistics on this though.

I think the principle of the Nash Equilibrium would still apply; but I would think a correction factor would have to be added so that %1st down + c1%touchdown - c2%turnover for run would equal %1st down + c1%touchdown - c2%turnover for pass. To make matters even more complex the constants themselves (c1, c2) would be functions of field position.

- Happy

10. Brett says:

Brian- I assume 3rd and 1 includes everything from inches up to a whole yard. If this is the case, then I believe your numbers will naturally be slightly skewed in favor of the run because teams almost always run and convert on 3rd and inches, but they tend to pass more on 3rd and a full yard to go, obviously with less success. The difference becomes less significant in longer to-go distances.

11. Brian Burke says:

Yes. I think I address that in the follow-on parts to this article. If you look at 3rd and 2, the same discrepancy exists in favor of running suggesting the bias may not be so significant.

12. Anonymous says:

coaches drive me bananas when they throw an incompletion on third and 2 or less in scoring position/four down territory,then fail on 4th down, or opt to kick the FG.

13. whispers says:

The Nash equilibrium only applies to two player games. You cannot expect it to apply to an average of several two player games. As an extreme example, if you have two teams, and Team A gets a 1st down on 3rd and 1 50% of the time while rushing, but 0% of the time while passing, and Team B gets a 1st down on 3rd and 1 0% of the time while rushing but 100% of the time while passing, then Team A should always run on 3rd and 1 while Team B should always pass on 3rd and 1, and your summary stats will say that on average, offenses get 1st on 3rd and 1 50% of the time while rushing, while they get it 100% of the time while passing.
Still, it's an interesting idea. If you assume more homogeneity across teams, your point might hold up.
I think what happens in practice is that teams rush if they feel a high likelihood of success rushing, but if they don't feel that way, they pass, regardless of whether the pass succeeds or fails with high percentage. Which would support your thesis.
It's just not a Nash equilibrium.

14. Anonymous says:

@ whispers: In game theory, Nash equilibriumis a solution concept of a game involving two or more players...... 2 or more...

15. Anonymous says:

If on 3rd and 5, running and passing have equal chances of conversion, then why do you think they should pass more than run on 3rd and 5?

Leave a Reply

Note: Only a member of this blog may post a comment.