tag:blogger.com,1999:blog-38600807.post8048620136125316625..comments2018-06-02T14:19:34.554-04:00Comments on Advanced Football Analytics (formerly Advanced NFL Stats): Fourth Downs in the New Overtime: First PossessionBrian Burkenoreply@blogger.comBlogger20125tag:blogger.com,1999:blog-38600807.post-6339426126214557442012-12-31T13:13:37.505-05:002012-12-31T13:13:37.505-05:00Sorry I didn't replay to Mike S' comment. ...Sorry I didn't replay to Mike S' comment. I think that's an important consideration. With more time on my hands I could do things like that. Like he mentioned, the effect of sampling bias could cut either way. The bias would affect conversion rates as well as basic EP and WP estimates.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-71937681723155064602012-12-18T02:36:10.744-05:002012-12-18T02:36:10.744-05:00mike s. -- That's probably the most important ...mike s. -- That's probably the most important comment I have seen on this site. As you suggest, it has ramifications for all of Burke's situational work. It's unfortunate he didn't reply.MPhttps://www.blogger.com/profile/04813956972730765912noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-33114300332881965882012-11-30T11:13:28.784-05:002012-11-30T11:13:28.784-05:00I happen to have been looking into this question m...I happen to have been looking into this question myself recently. Based on my own work and a few sanity checks, I think some of your win probability curves might be wrong. (If you want to see my own write-up, it is available here: http://www.reddit.com/r/NFLstatheads/comments/141xqy/original_research_4th_down_decisions_under_the/)<br /><br />1. I used data from regulation punting situations occurring in the 1st or 3rd quarters to figure out how likely punting teams at a given yard line were to score first. (I justify the use of regulation data to approximate sudden death data in the link I provided above.) Because a punt initiates sudden death, team who scores first = winner. The resulting curve is here: http://i.imgur.com/HY8os.png. My estimate of the win probability for the punting team is significantly lower than yours on the opponent's side of the field.<br /><br />2. I think there might be issues with you go-for-it curves, too. To take an example: your curves say that a first possession team with a 4th-and-1 on their own 10 has about a 50% chance of winning if they go for it. But we know this number is too high because the conversion rate is something like 70% and a failed attempt is almost a certain loss. For the math to work out, you would have to say that a first possession team with a first down at their own ~11 has a 70% chance of winning, which is way too high (my estimate is 40%).<br /><br />I think that the way you derived your curves might be the source of our disagreeing values. If I understand your methodology correctly, you always chose the value of time that maximized WP according to your standard regulation WP algorithm. Because your WP values are consistently higher than mine (and in some places I think obviously too high even without reference to my curves), I worry that you may be unjustified in deriving your values in that way.kloverrnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-63287716476985758662012-11-28T10:10:42.693-05:002012-11-28T10:10:42.693-05:00Sometimes a team gets a 15-yard penalty (for unspo...Sometimes a team gets a 15-yard penalty (for unsportsmanlike conduct, for example) that gets assessed on the kickoff. How about an onside kick now, either at the start of OT or at the onset of state 2? I don't know enough math to crunch the numbers, but I think it would be interesting to see.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-8457659299476645022012-11-28T08:36:00.084-05:002012-11-28T08:36:00.084-05:00Outstanding. Great analysis and discussion. Thanks...Outstanding. Great analysis and discussion. Thanks Brian. Ken R.https://www.blogger.com/profile/13086158597256834776noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-44655433004397489082012-11-27T20:07:59.224-05:002012-11-27T20:07:59.224-05:00What about an onside kick attempt at the onset of ...What about an onside kick attempt at the onset of state #2? It would be pretty interesting to see a team try.<br /><br />Recovery by the kicking team guarantees a win, while a failure to recover gives (using MarkP's numbers above) P(FG)=0.247 and P(TD)=0.342, therefore P(no score)=0.411. After a second field goal, I use the assumption referred to above that the team with the ball at the start of state #3 has a 60% chance of winning.<br /><br />If R is the chance of recovering the onside kick this gives:<br />WP = R*1 + (1-R)*0.411*1 + (1-R)*0.247*0.6<br /><br />For R=0.26, WP=0.67, and for R=0.6, WP=0.82. <br /><br />Obviously this needs to be compared to the base odds of winning at the start of state #2 given a regular kickoff, but I find it interesting that a team that tries an onside kick and fails still has a >50% chance of winning the game.Eric Moorehttps://www.blogger.com/profile/01496717842529415018noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-28037153690618183492012-11-27T16:47:11.415-05:002012-11-27T16:47:11.415-05:00MFLoGrasso:
That's why it is an interesting q...MFLoGrasso:<br /><br />That's why it is an interesting question, and I wanted to run down the outcomes with various numbers. Also: I made a fairly large mistake the first time through. With the kickoff now at the 35, an onside recovery (or not) occurs at the 46 instead of the 41, which has surprisingly large implications here. The WP's are now WP(kick away) = 0.433; WP(non-recovery) = 0.333; WP(recovery) = 0.675.<br /><br />Plugging this into the equation now yields a break-even of 29% onside recovery, well within the success for unexpected onside attempts. It will be interesting to see how teams actually perform in overtime, and if any one starts attempting occasional onside kicks.MarkPhttps://www.blogger.com/profile/02337868053749239010noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-6957365571235942452012-11-27T15:59:35.762-05:002012-11-27T15:59:35.762-05:00wait... what? on 10 yardline with 5 to go you go f...wait... what? on 10 yardline with 5 to go you go for it? I would imagine the fieldgoal is a virtual certainty, and your chance of winning would have to be extremely high. What's the odds your opponent gets 3 conversions or higher AND then kicks a fieldgoal? If you go for it, maybe you have a 40% chance of converting and winning right there and of course, here is why the go for it decision is correct, if you go for it and fail, your opponent STILL has a very small chance of putting together 3 conversions or whatever he needs AND making a fieldgoal. And with the additional 10 yard field position if you fail, this time a fieldgoal wins it automatically. Even though after the failure, the fieldgoal beats you, most likely it will result in you getting the ball back and having a chance to do it again and that's IF you fail your 40% conversion or whatever. The fieldgoal is close to a sure thing but isn't completely. If your opponent gets a big play on you regardless of what field position he has, it's over anyways. If you can prevent that and prevent the unlikelihood of allowing multiple conversions bak to back and a fieldgoal, you probably win.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-38343593965389613722012-11-27T15:37:32.342-05:002012-11-27T15:37:32.342-05:00Brian do you account for sampling bias in your wor...Brian do you account for sampling bias in your work? Not just this post but in general? For example, it's entirely possible that the better offense are on the field more because they convert more so the 1st and 2nd down data is not representative of the average team but instead skewed slightly to over represent the best offensive teams and under represent the worst defensive teams which can't get their offense on the field often... It's possible that either the best offenses have more 3rd down situations because they are on the field more, or FEWER because they convert on 1st and 2nd down more often and have fewer 3rd downs. I was glad to see you used 3rd down data rather than 4th down data to represeent the decisions teams should make on 4th down as I felt that the teams like the Patriots would line up and attempt to draw the other team offsides, if they do they get a free play, if they don't they read the defensive matchups and only run the play if they have a favorable matchup, otherwise they take a time out of take the delay of game. then they punt or kick a fieldgoal. This ability to only hike the ball when the matchups are favorable is really only available on 4th down. I suppose though the teams that are down by a tone because they are bad on offense may be more desperate and could skew things the other way. Nevertheless, even on 3rd down there may be a sampling bias because the data doesn't randomly and evenly represent the same number of downs for each team per year.<br /><br />I suggest each year counting the team with the least amount of 1st downs, the least amount of 2nd and the least amount of 3rd downs and randomly selecting just enough data so that you have an even representation of each particular down for each team and excluding the remaining. I am hesitant on whether or not to also suggest that you do that with each down and distance or even groups of down and distance (3rd and 7+ as one group, 3rd and 4-6 as another and 3rd and 3 or less as another)... Although there still may be sampling bias from teams that end up in 3rd and long because they are bad making the 3rd and long conversion rates potentially unfairly low, if you reduce the sample size too much per year it could increase the variance even if you give it a more accurate representative sample. Perhaps instead just removing or reducing the "outliers" or data (teams) with a large amount of a certain range of down and distances.<br />mike snoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-34204155486791146252012-11-27T15:05:09.385-05:002012-11-27T15:05:09.385-05:00I ignored the possibility of a safety.I ignored the possibility of a safety.Brian Burkehttps://www.blogger.com/profile/12371470711365236987noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-24344710575309514272012-11-27T14:03:34.382-05:002012-11-27T14:03:34.382-05:00MarkP:
I had no idea which numbers to use. I was ...MarkP:<br /><br />I had no idea which numbers to use. I was just wanted to use numbers that indicated a slight drop-off from kicking away if the kick wasn't recovered but a larger increase if it was recovered (which I suspect will occur).MFLoGrassohttps://www.blogger.com/profile/16421311347964163796noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-41205757786163489222012-11-27T13:34:57.422-05:002012-11-27T13:34:57.422-05:00MFLoGrasso:
I agree that the estimates still aren...MFLoGrasso:<br /><br />I agree that the estimates still aren't known, but I think that the current Markov states are a reasonable current stand in. If you run through the event tree assuming starting field position of own 22 for kicking away (and kicking away to start state 2) and using the Markov model (the only assumption I keep from the event tree is 60% chance of the team with the ball winning in state 3), you still get WP(kick away)= 0.433 and a break even of just over 60% recovery.<br /><br />When I first started running the numbers, I expected to lower the threshold for onside, but that isn't how they came out. I agree that we don't know the actual WP's, but I think that these estimates are likely closer to reality than your initial intuitions. I agree that an occasional onside kick might be worth it (especially in situations favoring high variance), but not at the frequency your initial estimate would have suggested.MarkPhttps://www.blogger.com/profile/02337868053749239010noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-68201385227769488902012-11-27T13:07:16.505-05:002012-11-27T13:07:16.505-05:00One question I have is with the narrowing down of ...One question I have is with the narrowing down of situations to where it maximizes the team's chance of winning as far as time remaining goes (for that opening drive). <br /><br />I think given the current conservative mindset of coaches, both at the end of regulation AND in overtime you may see teams kick field goals more -- and playing it safe for the field goal -- rather than making the complete effort to score a TD and win the game. Obviously this is counterintuitive, but falls in line with the idea of coaches playing not to lose rather than to win.Keith Goldnerhttps://www.blogger.com/profile/16510947295485321744noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-45423761718569182942012-11-27T13:00:43.822-05:002012-11-27T13:00:43.822-05:00MarkP:
But the issue is that the numbers in the e...MarkP:<br /><br />But the issue is that the numbers in the event tree are simply placeholders, if I understand correctly. Brian's point in this post is that he hasn't had enough data to accurately assess the situation.<br /><br />Also, I don't think there would ever be an obvious onside kick in OT. The kicking team would always line up to kick away, so there would be some surprise component if it goes onside. However, as time goes on, as certain teams show a tendency to onside kick, the receiving team (if coached well) will begin to put better hands guys as their front blockers, likely meaning they are not as capable blockers as the standard return unit. This should lead to poorer returns when the ball is kicked away, increasing the WP of kicking away (increasing the required recovery probability threshold) while also reducing the actual recovery probability.<br /><br />I see an eventuality where a well-coached team will look at the return team placed on the field to start OT and have someone on the field make the decision to kick away or go onside depending on the personnel.MFLoGrassohttps://www.blogger.com/profile/16421311347964163796noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-29300885774924823592012-11-27T12:41:44.845-05:002012-11-27T12:41:44.845-05:00MFLoGrasso:
Based on the event tree, WP(kick away...MFLoGrasso:<br /><br />Based on the event tree, WP(kick away) is actually only 0.444. <br /><br />Assuming that a failed onside gives the opposing team the ball at the 41, and that the Markov model still applies {P(FG) = 0.247; P(TD) = 0.342}, WP(non-recovery) = 0.25748. <br /><br />Based on the Markov model for recovering at your own 41 {P(FG) = .155, P(TD)=.234} WP(recovery) = 0.5446.<br /><br />Plugging that into your equation gives us: <br />(0.444 - 0.25748) / (0.5446 - 0.25748) = 0.649<br /><br />Making a recovery rate of roughly 65% of onside kicks the break even point. Earlier analysis (http://www.advancednflstats.com/2009/09/onside-kicks.html) showed that onsides are successful only 26% of the time. They are successful 60% of the times when they are unexpected, but still not to the 65% threshold.MarkPhttps://www.blogger.com/profile/02337868053749239010noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-7169665279391896452012-11-27T10:20:35.499-05:002012-11-27T10:20:35.499-05:00The safety falls into the turnover/punt > SD sc...The safety falls into the turnover/punt > SD scenario (SD over at that moment) so it is accounted for...Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-76409132024841942722012-11-27T10:17:04.945-05:002012-11-27T10:17:04.945-05:00Here's an analysis I would find interesting on...Here's an analysis I would find interesting once you get a decent handle on the overall scenario. What would the WP analysis be for onside kicking to start OT? Based on the rules (and I just verified this on the NFL website):<br /><br />"A.R. 16.2 ONSIDE KICK<br />On the opening kickoff of overtime, Team A legally recovers the ball at the A41.<br />Ruling: A's ball, first-and-10 on A41. A kickoff is considered an opportunity to possess for the receiving team. Team B is considered to have had an opportunity to possess the ball."<br /><br />So, if the kicking team recovers, SD is triggered immediately. If not, the kicking team has surrendered about 25-35 yards of field position for the first possession, and the receiving team is still not immediately in FG range. The simple formula would be to onside kick if:<br />P(recovery)*WP(recovery)+P(non-recovery)*WP(non-recovery) > WP(kick away)<br />which simplifies to onside kick if:<br />P(recovery) > (WP(kick away)-WP(non-recovery))/(WP(recovery)-WP(non-recovery))<br /><br />So, if kicking away to start OT gives the kicking team a 50% chance of winning, failing to recover an onside kick reduces this to 45%, but recovering the onside kick increases this to 65%. Then it would make sense to onside kick at the beginning of OT if you think you have a probability of recovering the kick higher than (0.50-0.45)/(0.65-0.45) = 0.05/0.20 = 0.25.MFLoGrassohttps://www.blogger.com/profile/16421311347964163796noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-55634767976846236282012-11-27T09:33:07.275-05:002012-11-27T09:33:07.275-05:00Isn't one of the possible terminal events for ...Isn't one of the possible terminal events for the team with initial possession a safety and thus a loss? I know it is a small probability, but it would still be a distinct outcome in your event tree.David Avraamideshttps://www.blogger.com/profile/07642457177220572135noreply@blogger.comtag:blogger.com,1999:blog-38600807.post-66661442572247009362012-11-27T08:23:23.826-05:002012-11-27T08:23:23.826-05:00Well the drop-off you're seeing inside the six...Well the drop-off you're seeing inside the six is due to the distance to gain being equal to the distance to goal (ex: there is no such thing as a 4th and 6 from the 4.) <br /><br />So the way I read that is you always go for it with goal-to-go from the 6 or closer. And you always go for it needing 6 yards for fewer out to around the 15 or so.Justinhttp://sotsohockey.blogspot.comnoreply@blogger.comtag:blogger.com,1999:blog-38600807.post-76691507748042866362012-11-27T08:14:55.376-05:002012-11-27T08:14:55.376-05:00The second oddity is the sudden drop-off in WP for...The second oddity is the sudden drop-off in WP for 4th down attempts at the opponent's 20 yd-line and 10-yd line. Those discontinuities are due to the increasing difficulty of converting 4th downs inside the red zone. Obviously, the true estimates are going to be smooth and continuous. But it was easy to correct for that in the final analysis, which follows below.<br /><br />I'm not sure that's what I see on the graph, can you please explain?<br /><br />Great post !Anonymousnoreply@blogger.com