Methodology
Know what is measured. Know what is assumed.
The simulator separates the historical game state, the coaching recommendation, and the estimated value of the options offered.
01 · Historical evidence
Start with the documented situation.
Scenario checkpoints use official gamebooks, play-by-play records, and team reporting. Score, clock, down, distance, ball location, and timeouts are preserved rather than randomized into new combinations.
Each completed rep provides sources and the actual outcome. Field art and player movement illustrate the situation; they are not film reconstructions.
02 · Decision value
Compare complete paths.
When a model comparison is available, the review combines the chance of success with the chance of winning after both success and failure. It shows the inputs so you can see which assumption changes the call.
Currently, 19 scenarios have fitted comparisons covering fourth-down and ordinary conversion decisions. Other scenarios use coaching analysis and score or clock arithmetic.
How the estimates are built
- Training data
- NFL play-by-play from 2016–2022
- Calibration
- 2023 season
- Held-out evaluation
- 2024 season
- Model family
- Histogram gradient-boosted trees
The win model uses score, remaining time, down, distance, field position, and both teams’ timeouts. The conversion model uses down, distance, and field position. The kick model uses distance. Games played on the replay dates are excluded from training, calibration, and evaluation.
The held-out win model’s Brier score is 0.1564, compared with 0.2500 for a constant-rate baseline. Lower is better. This measures prediction error on observed outcomes; it does not validate what would have happened under a different coaching decision.
Ordinary two-point tries use a historical conversion estimate of about 48.4%. A tested machine-learning model did not outperform the baseline for those tries. Long kicks beyond 58 yards include an explicit distance adjustment because the sample is sparse.
What can change the recommendation?
These are league-average, retrospective estimates. They omit team strength, personnel, injuries, tracking data, weather, and the specific play call. They do not reproduce everything a coaching staff knew at the time.
Policy calculations also make simplifying assumptions: a conversion gains the necessary yards, a failure gains none, and no penalty or return occurs. Punt distance and play duration are assumptions, not certain outcomes. Small gaps between options should not be treated as proven advantages.
Overtime possession choices, surprise onside kicks, intentional concessions, and personnel-timeout tradeoffs do not yet have fitted policy comparisons. Those reps identify the factors supporting the coaching recommendation instead.
How feedback and timing work
Progress grades follow the authored coaching rubric. When the fitted model favors a different option, the review presents that disagreement. The “best” estimated call means the highest estimated win probability among the offered paths under the stated assumptions.
In historical strategic reps, the 30-second decision timer creates training pressure; it does not consume the historical game clock. Final-kick drills separately simulate execution and clock operation.
Data and references
- nflverse play-by-play data · Model input, credited to nflverse contributors.
- nflverse data license
- nflverse schedule data · Game dates, weeks, and playoff rounds.
- nfl4th · Reference for the long-distance kicking adjustment.
Official gamebooks and scenario-specific reporting are linked in each completed rep.
Model version: league-hgb-2026-10-14-v1. Data and estimates prepared October 14, 2026.