Log-Loss (Cross-Entropy)¶

Summary¶

Log-loss (also called cross-entropy loss or logarithmic loss) is a scoring rule that measures the quality of probabilistic predictions. For binary outcomes, it is the negative average of log-probabilities assigned to the actual outcomes. It heavily penalizes confident wrong predictions — predicting 99% when the outcome is 0% scores much worse than predicting 51%.

Log-loss is the cost function used in logistic regression and is equivalent to the negative log-likelihood of the data under a Bernoulli model. Lower log-loss is better, with 0 being perfect.

In sports betting models, log-loss is used both as a training objective (to optimize probabilistic classifiers) and as a validation metric. Unlike Brier score which quadratically penalizes errors, log-loss penalizes exponentially — a prediction of 0.01 when the true outcome is 1 contributes -ln(0.01) = 4.6 to the loss, while Brier would contribute only 0.9801.

Key Concepts¶

Cross-entropy: Log-loss is the cross-entropy between the predicted distribution and the true distribution. For binary: H(p,q) = -[y log(p) + (1-y) log(1-p)]
Strictly proper: Like Brier score, log-loss is a strictly proper scoring rule — optimal when predicted probabilities match true probabilities
Penalizes overconfidence: A prediction of 0.001 for an event that occurs is catastrophic (ln(0.001) = -6.9), much worse than predicting 0.3 (ln(0.3) = -1.2)
Log-loss vs Brier: Log-loss is more sensitive to confident mistakes; Brier is more sensitive to calibration errors. For betting models, both should be tracked.
Multi-class log-loss: For soccer 1X2 predictions, use categorical cross-entropy: LL = -Σ y_i log(p_i)
Normalization: Log-loss can be normalized (divide by log(2)) to report in bits; or compared to baseline (climatology)

Formulas¶

Binary log-loss:
$$LL = -\frac{1}{N} \sum_{i=1}^{N} [y_i \log(p_i) + (1 - y_i) \log(1 - p_i)]$$

Where y_i = actual outcome (1 or 0), p_i = predicted probability.

Multi-class log-loss (categorical):
$$LL = -\frac{1}{N} \sum_{i=1}^{N} \sum_{c=1}^{C} y_{ic} \log(p_{ic})$$

For soccer 1X2: C=3 classes (home win, draw, away win), y is one-hot encoded.

Relationship to Brier score: For binary events near 50%, log-loss and Brier are similar. Differences grow at the extremes.

Normalized log-loss:
$$NLL = \frac{LL}{LL_{\text{climatology}}}$$

Where climatology = -[p̄ log(p̄) + (1-p̄) log(1-p̄)] with p̄ = mean outcome rate.

Python Implementation¶

import numpy as np
from sklearn.metrics import log_loss

def log_loss_binary(predicted_probs, actual_outcomes):
    """
    Calculate binary log-loss.
    """
    eps = 1e-15  # clipping to avoid log(0)
    p = np.clip(predicted_probs, eps, 1 - eps)
    return -np.mean(actual_outcomes * np.log(p) + (1 - actual_outcomes) * np.log(1 - p))

def log_loss_multiclass(predicted_probs_matrix, actual_outcomes_onehot):
    """
    Calculate multi-class log-loss (categorical cross-entropy).
    For 1X2 soccer predictions.

    Args:
        predicted_probs_matrix: Nx3 array of predicted probabilities (rows sum to 1)
        actual_outcomes_onehot: Nx3 array of one-hot actual outcomes
    """
    eps = 1e-15
    p = np.clip(predicted_probs_matrix, eps, 1 - eps)
    # Normalize rows to sum to 1
    p = p / p.sum(axis=1, keepdims=True)
    return -np.mean(np.sum(actual_outcomes_onehot * np.log(p), axis=1))

def log_loss_skill_score(predicted_probs, actual_outcomes):
    """
    Log-loss skill score vs. climatology baseline.
    """
    ll_model = log_loss_binary(predicted_probs, actual_outcomes)

    p_bar = np.mean(actual_outcomes)
    p_base = np.full_like(actual_outcomes, p_bar, dtype=float)
    ll_baseline = log_loss_binary(p_base, actual_outcomes)

    return 1 - (ll_model / ll_baseline)

# Example: soccer 1X2 multi-class
# predicted: [home_prob, draw_prob, away_prob] for each match
predictions = np.array([
    [0.55, 0.25, 0.20],  # match 1: home predicted most likely
    [0.30, 0.35, 0.35],  # match 2: draw/away uncertain
    [0.70, 0.20, 0.10],  # match 3: home heavily favored
])
# outcomes: 1=home win, 2=draw, 3=away win
outcomes = [1, 3, 1]  # match 1: home win, match 2: away win, match 3: home win
y_onehot = np.zeros((3, 3))
for i, outcome in enumerate(outcomes):
    y_onehot[i, outcome - 1] = 1

ll = log_loss_multiclass(predictions, y_onehot)
print(f"Multi-class log-loss: {ll:.4f}")

Notes¶

Log-loss is the primary training objective for neural network and gradient boosting sports prediction models — it directly optimizes probability calibration
For betting model evaluation: track both log-loss (penalizes overconfident wrong predictions) and Brier score (penalizes calibration errors)
Log-loss is more volatile than Brier — small sample sizes (64 World Cup matches) make it less reliable as a standalone metric
Use log-loss for model selection during training; use Brier score + CLV for final model evaluation
The Expected Calibration Error (ECE) metric complements both: bucket predictions by probability range, compute weighted |avg_pred - empirical_freq|, sum