Walk-Forward Validation¶

Summary¶

Walk-forward validation (also called walk-forward optimization or rolling forward) is the gold standard for validating sports betting models. Unlike k-fold cross-validation which randomly splits data, walk-forward validation respects temporal order: the model is trained on historical data, then tested on future data that was not available at training time.

The process: train on period 1 → test on period 2 → expand training window to include period 2 → test on period 3 → repeat. This mimics real deployment where today's model is built on all historical data and used to predict tomorrow's games.

Walk-forward validation prevents look-ahead bias (using future information to make predictions) and gives an honest estimate of how the model would have performed in real-time. The client's spec explicitly requires walk-forward validation across 2010, 2014, 2018, and 2022 World Cup group stages — specifically because k-fold cross-validation is inappropriate for time-series betting data.

Key Concepts¶

Expanding window: Training set grows over time (all history to T1, all history to T2, etc.). Most common for betting models.
Rolling window: Training set stays fixed size (last N games), slides forward. Better when team characteristics change significantly.
Purged cross-validation: Removes a buffer zone between training and test sets to prevent information leakage from near-future data influencing training features.
Look-ahead bias: Using information that wouldn't have been available at prediction time. Walk-forward prevents this by design.
Out-of-sample testing: Data the model has never seen during training. Walk-forward ensures every test period is out-of-sample.
Walk-forward optimization: Iterating through the data to find optimal parameters that work across all walk-forward periods.

Process¶

Period 1 (2010 WC) → train on 2006-2010 → test on 2010 WC
Period 2 (2014 WC) → train on 2006-2014 → test on 2014 WC  
Period 3 (2018 WC) → train on 2006-2018 → test on 2018 WC
Period 4 (2022 WC) → train on 2006-2022 → test on 2022 WC

For each period:
  1. Train model on all data up to tournament start
  2. Generate predictions for all matches
  3. Compare to actual outcomes
  4. Record metrics: ROI, CLV, Brier score, hit rate

Python Implementation¶

import pandas as pd
import numpy as np
from datetime import datetime

def walk_forward_validate(df, train_start, test_start, test_end, features, target, model_class, params={}):
    """
    Perform single walk-forward validation fold.

    Args:
        df: full historical DataFrame
        train_start: start date for training
        test_start: start date for test period
        test_end: end date for test period
        features: list of feature column names
        target: target column name
        model_class: sklearn-compatible model class
        params: model hyperparameters

    Returns:
        DataFrame of predictions for test period
    """
    train_df = df[(df['date'] >= train_start) & (df['date'] < test_start)]
    test_df = df[(df['date'] >= test_start) & (df['date'] <= test_end)]

    if len(train_df) < 50 or len(test_df) < 5:
        return None

    X_train = train_df[features]
    y_train = train_df[target]
    X_test = test_df[features]

    model = model_class(**params)
    model.fit(X_train, y_train)

    predictions = test_df.copy()
    predictions['pred_prob'] = model.predict_proba(X_test)[:, 1]
    predictions['fold'] = f"{test_start.date()}_{test_end.date()}"

    return predictions

def run_walk_forward(df, tournaments, features, target, model_class, params={}):
    """
    Run walk-forward validation across multiple tournaments.
    tournaments: list of (train_cutoff, test_start, test_end) tuples
    """
    all_results = []

    for train_end, test_start, test_end in tournaments:
        result = walk_forward_validate(
            df, 
            train_start=df['date'].min(),  # expanding window
            test_start=test_start,
            test_end=test_end,
            features=features,
            target=target,
            model_class=model_class,
            params=params
        )
        if result is not None:
            all_results.append(result)

    combined = pd.concat(all_results)

    # Aggregate metrics
    metrics = {
        'total_bets': len(combined),
        'roi': (combined['pnl'].sum() / combined['stake'].sum()) if 'pnl' in combined.columns else None,
        'brier_score': ((combined['pred_prob'] - combined['outcome'])**2).mean(),
        'clv_mean': combined['clv'].mean() if 'clv' in combined.columns else None,
        'hit_rate': combined['correct'].mean() if 'correct' in combined.columns else None
    }

    return combined, metrics

# Tournament schedule for walk-forward
world_cup_tournaments = [
    # (train_end, test_start, test_end)
    ('2010-06-10', '2010-06-11', '2010-07-11'),  # 2010 WC
    ('2014-06-12', '2014-06-13', '2014-07-13'),  # 2014 WC
    ('2018-06-14', '2018-06-15', '2018-07-15'),  # 2018 WC
    ('2022-11-20', '2022-11-21', '2022-12-18'),  # 2022 WC
]

Notes¶

Walk-forward is computationally expensive but gives the most honest validation picture for betting models
Key metric: does the model beat the closing line (positive CLV) consistently across all four World Cups?
The client specifically tests for walk-forward validation (not k-fold), so this is a hard requirement for the MVP
For World Cup specifically: with only 64 matches per tournament, each fold has small sample — aggregate across all 4 tournaments for statistical power
The expanding window approach is appropriate for World Cup since team strength accumulates over years