Data Preparation / Data Preprocessing Slides 📂 Introduction · 9 of 13 47 min read

Feature Scaling in Machine Learning

A practical, visual guide to feature scaling for machine learning. Learn Min-Max, Z-score, Robust and Max-Abs scaling with worked examples; see exactly which algorithms need scaling and which don't; avoid the data-leakage trap; pick the right scaler with a simple decision tree; and build a leak-proof scikit-learn pipeline.

Feature Scaling in Machine Learning

Putting every feature on the same footing so distance, gradient and regularisation algorithms judge them fairly. Scaling isn't bureaucracy — it's mathematics.
Min-Max & Z-Score Robust & Max-Abs Which Algorithm? Leak-Proof Pipelines

Press Next → or use ← → arrow keys

Section 01

Why Feature Scaling Is Essential

🏥 REAL STORY · F1 0.54 → 0.81
The triage model that ignored kidney failure
A Delhi hospital's KNN triage model used age (18–90), blood pressure (80–200) and creatinine (0.5–12). Because BP's numbers were largest, it owned 96% of every distance — so a critically ill patient with high creatinine but normal BP was ranked low priority. After StandardScaler, creatinine could finally speak: F1 rose from 0.54 to 0.81.
Before — three wildly different ranges salary BP creatinine creatinine is invisible to the model → After — equal footing all three now heard equally
Section 01 · The Rule

Scale or Skip? One Simple Test

scale
📏 Distance
KNN, K-Means, SVM — Euclidean distance lets the biggest range dominate.
scale
📉 Gradient
Neural nets, linear & logistic regression — step size scales with input magnitude.
scale
➗ Regularised
Ridge, Lasso, ElasticNet, PCA — the penalty is unfair unless features share a scale.
skip
🌳 Tree splits
Decision Tree, Random Forest, XGBoost, Isolation Forest — split on thresholds, scale-invariant.
optional
🎲 Naïve Bayes
Its per-feature variance term already absorbs different scales.
check skew
📊 Any model
If |skew| > 1, reshape with a log/power transform before scaling.
🧭
The one-line heuristic

If the algorithm involves distance, gradient, or regularisation → scale. If it involves tree, split, or threshold → skip.

Section 02

The Four Core Scalers

📏 Min-Max
(x − min) / (max − min)
→ [0, 1]. Keeps shape. Neural nets & images. Fragile to outliers.
📐 Z-Score / Standard
(x − μ) / σ
→ mean 0, std 1. The default for linear models, SVM, PCA.
🛡️ Robust
(x − median) / IQR
Centres on median. Outliers can't distort it.
🧮 Max-Abs
x / |max(x)|
→ [−1, 1]. Keeps zeros — ideal for sparse TF-IDF.
🎯
Same goal, different maths

All four rescale — but Min-Max & Max-Abs use the extremes (fragile to outliers), Standard uses mean & σ (partly fragile), and Robust uses median & IQR (outlier-proof).

Section 02 · By Hand

One Value, Four Scalers — Worked

Min-Max · age
(35 − 18) / (90 − 18)
= 17 / 72 = 0.236
Z-Score · salary
(500k − 420k) / 180k
= 80 / 180 = 0.444
Robust · purchase
(8000 − 6500) / 5200
= 1500 / 5200 = 0.288
Max-Abs · TF-IDF
0.42 / 0.91
= 0.462
✍️
Every scaler is just arithmetic on learned stats

Each method stores a few numbers at fit time — min & max, or mean & σ, or median & IQR, or the max absolute value — then applies the same formula to every row. That's why the fitted scaler is the model of your data's scale.

Section 03

Min-Max Normalisation → [0, 1]

A fraud-detection network wouldn't converge — transaction amount (₹50–₹4.2M) produced gradients millions of times larger than failed-login counts (0–8). Loss thrashed for 300 epochs. MinMax fixed it: clean convergence in 45.

Before — raw ₹ (right-skew) ₹0₹24k →MinMax After — [0,1] · same shape 0.01.0 
from sklearn.preprocessing import MinMaxScaler
sc = MinMaxScaler(feature_range=(0,1))
sc.fit(X_train[num])           # train only
X_train[num] = sc.transform(X_train[num])
X_test[num]  = sc.transform(X_test[num])
💥
Shattered by one outlier

Add a single ₹5,000,000 value and every normal point is crushed into 0.00–0.01. Min-Max only when the column is already outlier-free.

Section 04

Z-Score Standardisation → μ=0, σ=1

A bank read its loan model's coefficients as credit_score 1.42 vs salary 0.000003 — "473,000× more important." Pure scale artifact. After standardising, salary emerged the strongest predictor (2.3) and all coefficients were finally comparable.

Before — salary dwarfs all age salary credit emp yrs → After — all σ=1, centred at 0 μ = 0 
from sklearn.preprocessing import StandardScaler
sc = StandardScaler().fit(X_train[num])       # learns mean_ & var_ from train
X_train[num] = sc.transform(X_train[num]);  X_test[num] = sc.transform(X_test[num])
pd.DataFrame(X_train[num]).agg(['mean','std']).round(2)   # ≈ 0 and 1 ✅
Section 05

Robust Scaling — Median & IQR

An insurer's claims ran ₹50k–₹120k normally, with legitimate catastrophes over ₹4.2M. StandardScaler dragged the mean to ₹340k and squashed normal claims near zero (R² 0.71). RobustScaler kept them well-spread (R² 0.84).

MinMax ✗ crushed to 0 Standard ~ mean pulled right Robust ✓ normal spread kept ₹4.2M outlier 
from sklearn.preprocessing import RobustScaler
sc = RobustScaler(quantile_range=(25.0,75.0)).fit(X_train[num])   # median + IQR
# the ₹4.2M outlier: StandardScaler ≈ 6.3σ  ·  RobustScaler ≈ 68.6 (stays far, doesn't distort)
Section 06

Max-Abs & the Full Comparison

ScalerFormulaOutputOutlier-proofKeeps 0sBest for
MinMax(x−min)/(max−min)[0,1]nonoNeural nets, image pixels
Standard(x−μ)/σ(−∞,∞)partialnoLinear, PCA, SVM
Robust(x−Q2)/IQR(−∞,∞)yesnoLegit extreme values
MaxAbsx/|max|[−1,1]noyesSparse, TF-IDF, NLP
 
from sklearn.preprocessing import MaxAbsScaler
X_sparse = TfidfVectorizer(max_features=5000).fit_transform(df['review_text'])
X_scaled = MaxAbsScaler().fit_transform(X_sparse)   # divides by |max|, keeps sparsity & zeros
Section 07

Which Algorithms Need Scaling?

AlgorithmVerdictWhy
KNN · K-Means⚠ scaleEuclidean distance — big ranges dominate
SVM⚠ scaleMargin is set by the largest-scale feature
Neural networks⚠ scaleGradients ∝ input magnitude → explode / vanish
Linear · Logistic · Ridge · Lasso⚠ scaleCoefficients & penalties depend on scale
PCA⚠ scaleChases variance — hijacked by the biggest feature
Decision Tree · RF · XGBoost✓ skipSplit on thresholds — scale-invariant
Naïve Bayes~ optionalPer-feature variance absorbs scale
🌳
Trees genuinely don't care

A split like "salary > 500,000" gives the same partition whether salary is in rupees or scaled to 0–1. Scaling a tree model wastes effort and can make features harder to interpret.

Section 07 · Impact

The Payoff — Accuracy Before vs After

Model accuracy: unscaled vs scaled KNN +29 SVM +22 Neural Net +26 LogReg +11 Random Forest +0 unscaledscaled
📊
KNN gains most, Random Forest gains nothing

Distance and gradient models leap when features share a scale; the tree ensemble is flat because it never used magnitude in the first place. The chart is the scale/skip rule, measured.

Section 08

The Data-Leakage Trap

✗ WRONG — fit on everything, then split Full dataset scaler.fit(ALL)sees test stats split 💥leak ✓ CORRECT — split first, fit on train split first scaler.fit(TRAIN)learns here only transform both ✅honest 
# ✅ BEST — a Pipeline is leak-proof by design
pipe = Pipeline([('scaler', StandardScaler()), ('model', LogisticRegression())])
pipe.fit(X_train, y_train);  pipe.score(X_test, y_test)   # scaler fits on train inside fit()
Section 09

Choosing the Right Scaler

Sparse? (TF-IDF, counts) yes → MaxAbsScaler no → legit outliers? yes → RobustScaler no → NN / image? yes → MinMaxScaler no → Standard Default when unsure: StandardScaler
🧭
Four questions, one answer

Sparse? MaxAbs. Legit outliers? Robust. Neural net / images? MinMax. Otherwise? StandardScaler — the safe default for the majority of models.

Section 09 · Side by Side

Same Features, Four Scalers

Raw salary dwarfs all MinMax all bounded 0 → 1 Standard centred at 0, σ=1 Robust centred on median, spread = IQR
Section 10

A Multi-Scaler Production Pipeline

std = Pipeline([('imp', SimpleImputer('median')),
                ('sc', StandardScaler())])
rob = Pipeline([('imp', SimpleImputer('median')),
                ('sc', RobustScaler())])
mm  = Pipeline([('imp', SimpleImputer('median')),
                ('sc', MinMaxScaler())])
cat = Pipeline([('imp', SimpleImputer('most_frequent')),
                ('oh', OneHotEncoder(
                        handle_unknown='ignore'))])

pre = ColumnTransformer([
    ('std', std, normal_num),
    ('rob', rob, skewed_num),
    ('mm',  mm,  bounded),
    ('cat', cat, cat_cols)])

pipe = Pipeline([
    ('prep',  pre),
    ('model', GradientBoostingClassifier())])
pipe.fit(X_train, y_train)
joblib.dump(pipe, 'production.pkl')
X_train (raw) Standardnormal Robustskewed MinMaxbounded OneHotcateg. ColumnTransformer GradientBoosting → ŷ
Pro · Debug

Common Pitfalls & Their Fixes

SymptomThe trapThe fix
One feature dominates KNN/SVMNo scaling at allStandardScaler / MinMaxScaler
Inflated validation scoreFit scaler on full dataFit on X_train only — use a Pipeline
Normals crushed near zeroMinMax with an outlierRobustScaler
Skew survives scalingScaling a skewed columnlog1p / power transform first
Sparse matrix blows up in RAMStandardScaler centres zerosMaxAbsScaler (keeps sparsity)
Wasted effort, worse interpretabilityScaling a tree modelSkip — trees are scale-invariant
Model fails at inferenceSaved model, not the scalerjoblib.dump(pipeline)
Section 11 · Part 1

The Golden Rules — 1 to 4

📐 FEATURE SCALING · RULES 1–4
1
Scale distance/gradient/regularised models (KNN, SVM, NN, linear, Ridge, Lasso, K-Means, PCA). Never scale trees (RF, XGBoost).
2
Fit on train only, then transform train and test separately — fitting on the full set leaks statistics and inflates metrics.
3
StandardScaler is the default. Switch only for a specific reason — outliers, sparsity, or a bounded-input model.
4
RobustScaler for legitimate extremes. Mean & σ are distorted by outliers; median & IQR are not.
Section 11 · Part 2

The Golden Rules — 5 to 8 & Takeaway

📐 FEATURE SCALING · RULES 5–8
5
MinMax for neural nets & images that need bounded [0,1] inputs — but not when outliers are present.
6
Check skew before scaling. If |skew| > 1, log/power-transform first — scaling never fixes shape.
7
Verify the output. describe() — Standard should give mean≈0, std≈1; MinMax should give min 0, max 1.
8
Save the whole fitted pipeline, not just the model — every point at inference must pass the same fitted scaler.
🎯
Scaling is mathematics, not bureaucracy

When the triage model ignored creatinine because blood pressure had bigger numbers, it wasn't a data or model problem — it was a scale problem. Four lines of code moved F1 on critical patients from 0.54 to 0.81.

📐 End of tutorial · Press ← to review, or click Restart

1 / 18