Bias vs Variance Decoded

The Story: Vikram Needs to Predict Home Prices

The Data — Home Price vs Square Footage

Vikram plots all 21 homes. As square footage rises, price rises — but not perfectly. Real data always has noise. This is the dataset every model will be trained and tested on.

The Train / Test Split — The Foundation of All Evaluation

Before training any model, Vikram splits his 21 homes into two groups. Blue dots are the training set — the model learns from these. Orange dots are the test set — hidden during training, used only to measure real-world performance. This split is the key to revealing bias and variance.

The Complex Model — "The Memoriser" (Overfitting)

Vikram's first attempt is an extremely complex curve — high-degree polynomial that snakes through every single training point. It memorises the training data perfectly. Train error = 0.

Proving High Variance — The Same Model, a Different Training Set

Variance means: how much does the model's performance change when you train it on a different sample of the same data? Vikram shuffles his data and creates a second train/test split — different blue dots, different orange dots. He trains the same type of complex wiggly model again.

🎬 Watch It Live — The Variance Dance of the Complex Model

Two static snapshots are one thing — but variance is a behaviour, not a photograph. Watch the same complex model type retrained on four different training samples of the same underlying data. Notice how the green curve twists into a wildly different shape each time, even though the true trend (behind the noise) never changed.

The Simple Model — "The Oversimplifier" (Underfitting)

Vikram's second attempt is the simplest possible model: a straight line. It cannot capture the slight upward curve in the data. Even on the training set, it makes consistent errors because a line simply cannot fit a curve.

Proving High Bias + Low Variance — The Linear Model on Two Splits

🎬 Watch It Live — The Stubborn Line of the Simple Model

Now watch the opposite behaviour. Same four training samples, same underlying data — but this time Vikram fits a straight line. Notice how the line barely twitches between samples. It is stable, predictable... and wrong in exactly the same way every single time.

The Balanced Fit — "The Generalist" (The Sweet Spot)

Vikram's third model is a smooth, moderately curved function — complex enough to follow the general upward curve but not complex enough to memorise every noise fluctuation. This is the balanced fit.

🎮 The Complexity Dial — Press "Next" to Step Through Underfit → Balanced → Overfit

This time you control the model complexity. Press the Next button below to move through five stages, one at a time, watching exactly how the fitted green curve, the train/test errors, and the bias-variance verdict change with each turn of the dial.

The Complete Picture — All Three Models Side by Side

Why — The Mechanism Behind Each Behaviour

🎬 The Bias-Variance Dartboard — Every Model in Four Quadrants

Every trained machine-learning model can be summarised by a dart pattern on a target. The bullseye is the true answer. Each dart is one model trained on one training set. Where the darts land tells you everything about bias and variance. Watch the darts hit live below.

🎬 The U-Shape — Total Error vs Model Complexity

If you plot bias, variance, and total error against model complexity, you get the single most important graph in machine learning. Bias falls as complexity rises. Variance climbs. Total error dips into a U-shape — and the bottom of that U is the sweet spot every practitioner is chasing. Watch a marker travel along the complexity axis to see where each of Vikram's models lands.

Python — Reproducing Vikram's Three Models

import numpy as np
from sklearn.pipeline        import Pipeline
from sklearn.preprocessing   import PolynomialFeatures
from sklearn.linear_model    import LinearRegression
from sklearn.model_selection import train_test_split, ShuffleSplit
from sklearn.metrics         import mean_squared_error

np.random.seed(42)

# ── Generate home price data (quadratic + noise) ─────────────
sqft  = np.random.uniform(600, 2500, 80)
price = 0.00004 * sqft ** 2 + 0.05 * sqft + np.random.randn(80) * 12
X     = sqft.reshape(-1, 1)
y     = price

# ── Three model pipelines ─────────────────────────────────────
models = {
    'Underfit (degree=1, linear)'    : Pipeline([
        ('poly',  PolynomialFeatures(1)),
        ('model', LinearRegression()),
    ]),
    'Balanced (degree=3)'            : Pipeline([
        ('poly',  PolynomialFeatures(3)),
        ('model', LinearRegression()),
    ]),
    'Overfit (degree=15, complex)'   : Pipeline([
        ('poly',  PolynomialFeatures(15)),
        ('model', LinearRegression()),
    ]),
}

# ── Run 2 different train/test splits (like the PDF slides) ──
splits = [42, 99]   # two different random seeds = two different training sets

print(f"{'Model':<32} {'Split':>6} {'Train MSE':>10} {'Test MSE':>10} {'Gap':>8}")
print("-" * 72)

for name, model in models.items():
    test_errors = []
    for i, seed in enumerate(splits, 1):
        X_tr, X_te, y_tr, y_te = train_test_split(
            X, y, test_size=0.25, random_state=seed
        )
        model.fit(X_tr, y_tr)
        tr_mse = mean_squared_error(y_tr, model.predict(X_tr))
        te_mse = mean_squared_error(y_te, model.predict(X_te))
        test_errors.append(te_mse)
        print(f"{name:<32} {'Split '+str(i):>6} {tr_mse:>10.1f} {te_mse:>10.1f} {te_mse-tr_mse:>8.1f}")

    variance_signal = abs(test_errors[1] - test_errors[0])
    print(f"{'':32} {'Variance (test diff):':>18} {variance_signal:>9.1f}")
    print()

Golden Rules