NCAA Tournament Prediction Model

Technical Reference — March Machine Learning Mania 2026

1. Overview

This model predicts the probability that Team A beats Team B for every possible NCAA tournament matchup. It combines a gradient-boosted decision tree (LightGBM, 44 features) with a logistic regression (KenPom features, 5 features) in a 70%/30% weighted ensemble.

Key design principles:

• Leave-One-Season-Out CV: every evaluation is genuinely out-of-sample. No within-season leakage.
• Matchup-level prediction: the model predicts P(A beats B) directly, not team strength. All features are differences (Team A minus Team B).
• Multi-source consensus: combining 11 data sources reduces dependence on any single rating system.
• Conservative regularization: shallow trees (depth 3), low learning rate (0.013), and modest L1/L2 penalties. The primary regularization comes from tree structure constraints, not penalty terms. The model is designed not to overfit the ~1,445 historical tournament games.

2. Architecture

The pipeline has five stages:

3. Data Sources

4. Feature Engineering

Feature Categories

Features are organized into tiers by source and type. All enter the model as differences: feature_A - feature_B.

Full Feature List (44 LightGBM features)

Key Derived Features

Four Factors (Dean Oliver's framework): offensive and defensive versions of effective FG%, turnover rate, offensive rebound rate, and free throw rate. These are tempo-free — they measure quality independent of pace.

Matchup interactions: How offensive vs defensive the matchup is overall (symmetric — invariant to team ordering):

BPI-KenPom divergence: when BPI and KenPom disagree about a team's ranking, it signals that one system sees something the other doesn't. This feature ranks 5th by importance.

Elo rating: our custom Elo system (documented at /elo). Processes every regular-season game since 1985 with margin-of-victory scaling and 75% season carryover. Ranks #1 by feature importance.

5. Model Training

Model 1: Seed-Only Logistic Regression (Floor Baseline)

A single feature: seed_diff. This is the minimum-viable model that any NCAA bracket predictor should beat. Log-loss: 0.566.

Model 2: KenPom Logistic Regression

5 features with GridSearchCV over regularization strength C:

This is the "strong baseline" — hard to beat because AdjEM alone is highly predictive. Log-loss: 0.547.

Model 3: LightGBM

44 features with Optuna-tuned hyperparameters:

Log-loss: 0.538. The key hyperparameters are max_depth=3 (very shallow trees) and learning_rate=0.013 (very slow), which prevent overfitting on a small training set.

Training Protocol

Leave-One-Season-Out (LOSO): for each of the 22 seasons (2003-2025, excluding 2020), train on all other seasons and predict the held-out season. This is more conservative than k-fold because it tests temporal generalization — the model must predict a season it has never seen.

6. Ensemble

The final prediction blends LightGBM and KenPom LR:

The optimal weight is found by grid search over LOSO predictions. The ensemble achieves 0.53651 log-loss — better than either model alone.

Other ensemble strategies tested

• Stacking meta-learner: logistic regression on all three models' OOF predictions. Did not improve over simple weighted average.
• Isotonic calibration: post-hoc calibration on ensemble predictions. Tested via LOSO to avoid leakage. Did not improve — the ensemble is already well-calibrated (ECE = 0.014).

7. Injury Adjustments

Injuries are incorporated via a Win Shares replacement model that adjusts KenPom AdjEM for missing players:

The discount factor accounts for games already missed: if a player has already been out for most of the season, their absence is already reflected in the team's stats (discount = 0.2 if >50% missed).

The adjustment is split: AdjOE gets half the impact, AdjDE gets the other half. This is conservative — in reality, losing a scorer mostly hurts offense, but we don't have enough data to model the split precisely.

8. Bracket Simulation

The model generates championship probabilities via Monte Carlo simulation:

The simulation handles edge cases: play-in games, teams at the same seed competing for region slots, and the "bye" mechanism when fewer than 64 teams are in the field.

Optional: Rating Perturbation

An experimental mode adds $N(0, \sigma)$ noise to each team's AdjEM per simulation, modeling the uncertainty in team ratings. This compresses the probability distribution: the favorite's probability decreases while underdogs gain. The notebook comparison model uses $\sigma = 2.5$.

9. Validation

All metrics are computed out-of-sample via LOSO cross-validation on 1445 historical tournament games across 22 seasons:

Expected Calibration Error (ECE) = 0.014 means the model's predicted probabilities closely match actual outcomes. When the model says 70%, the team wins about 70% of the time.

Per-Round Performance

The model performs best in early rounds (where seed advantages are largest) and degrades in later rounds (where remaining teams are more evenly matched):

• R64: highest accuracy (seed favorites win ~75%)
• Sweet 16 / Elite 8: log-loss increases as matchups become more competitive
• Final Four / Championship: most uncertain — essentially coin flips between elite teams

10. Feature Ablation

Which feature groups matter most? We remove each tier entirely and measure the log-loss impact:

The ablation also reveals significant redundancy: KenPom's AdjEM, Massey's POM rank, and Elo are highly correlated. Removing any one is partially compensated by the others. But removing the entire consensus tier (8 features spanning Massey systems, BPI, derived divergence, and poll trajectory) is devastating because it eliminates all cross-system signal.

11. Market Comparison

We compare model championship probabilities to prediction market prices from Kalshi, Polymarket, and sportsbook sharp consensus:

• Model systematically higher on: defensive/efficiency teams (Houston, Illinois, Michigan St) — the model rewards KenPom metrics that markets may underweight
• Markets systematically higher on: brand-name and momentum teams (Michigan, Florida, Kansas) — markets may overweight recent narratives and fan attention

The model's predictions are not blended with market data for the primary submission. Markets are used only for post-hoc comparison and identifying potential value picks. An optional market-blend submission uses 70% model + 30% market-implied pairwise probabilities.