Handling Imbalanced Data
Press Next → or use ← → arrow keys
The Class Imbalance Problem
On a 99:1 split, "always predict the majority" scores 99% while being completely useless. The rarest class is usually the most important one — and it gets the least attention.
How Bad Is Your Imbalance?
counts = df['target'].value_counts(); ratio = counts.max() / counts.min()
print(f"Imbalance ratio: {ratio:.0f}:1") # severity guides the treatment
Counter(y_train) # Counter({0: 9910, 1: 90}) → 110:1, severeStep 1 — Fix the Metric, Not the Model
99.1% acc · 0% recall · 0/90 caught
97.1% acc · 80% recall · 72/90 caught
The "worse" model on accuracy is dramatically better in reality. That's why you must read the confusion matrix and the minority-class metrics — never the headline accuracy alone.
Precision, Recall & the Right Curve
Recall = of all real frauds, how many caught? Precision = of all flagged, how many were real? F1 balances the two.
ROC is inflated by the huge pile of true negatives, so it flatters weak models. AUC-PR is the honest primary metric on severe imbalance.
Six Ways to Rebalance
class_weight='balanced'.scale_pos_weight.Try class weights first, add SMOTE for severe imbalance, tune the threshold to hit your business target, and reach for combined methods only when the boundary is noisy.
Class Weights — the Two-Line Fix
An insurance team spent three weeks tuning XGBoost on a 100:1 fraud set — recall stuck at 31%. Adding class_weight='balanced' doubled it to 64%. Two lines, no new architecture.
# auto-computed inverse-frequency weights
LogisticRegression(class_weight='balanced')
RandomForestClassifier(class_weight='balanced')
# manual: minority gets ~100× the weight
w = compute_class_weight('balanced', classes=np.unique(y_train), y=y_train)
print(dict(enumerate(w))) # {0: 0.505, 1: 50.5}
# XGBoost equivalent
xgb.XGBClassifier(scale_pos_weight=neg/pos, eval_metric='aucpr')It's two characters of code, costs nothing, changes no data, and often delivers ~80% of the benefit of heavier resampling. Only escalate if it isn't enough.
SMOTE — Synthesising the Minority
A hospital's rare-condition model (100:1) reached only 42% recall with weights alone. SMOTE generated 9,520 synthetic cases (down to 5:1) and recall leapt to 87% — an estimated 180 extra patients caught per year.
SMOTE doesn't duplicate rows — it places new points between real minority neighbours, forcing the model to learn a denser, more accurate boundary around the rare class.
SMOTE Done Right — Split First
from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline # imblearn!
pipe = Pipeline([
('scaler', StandardScaler()),
('smote', SMOTE(sampling_strategy=0.5,
random_state=42)),
('model', RandomForestClassifier())])
pipe.fit(X_train, y_train) # SMOTE on train only
pipe.predict(X_test) # test untouched ✓
# BorderlineSMOTE / ADASYN = harder-case variantsNever SMOTE the full dataset before splitting — synthetic points made from train rows leak into test, giving fake-brilliant scores that collapse in production.
sampling_strategy=0.5 makes the minority 50% of the majority (a 2:1 ratio). Forcing a perfect 1:1 usually over-synthesises and hurts precision — 2:1 or 3:1 is plenty.
Undersampling & Combined Cleanup
# drop majority rows (fast, lossy)
RandomUnderSampler(sampling_strategy=0.5)
# Tomek: remove borderline majority
TomekLinks()
# SMOTE + cleanup = best of both
from imblearn.combine import \
SMOTETomek, SMOTEENN
SMOTETomek(random_state=42) # over + de-noise
SMOTEENN(random_state=42)Oversampling keeps all your data but adds synthetic rows; undersampling is faster but discards real majority examples. Combined methods do both, then clean the seam between classes.
Threshold Tuning — 0.5 Is Almost Always Wrong
p, r, t = precision_recall_curve(y_val, model.predict_proba(X_val)[:,1])
best = t[np.argmax(2*p*r/(p+r+1e-10))] # threshold that maximises F1
y_pred = (model.predict_proba(X_test)[:,1] >= best).astype(int) # not 0.5!Which Strategy Wins? It Depends
SMOTE + class weights gives the best F1 (0.77) — the balanced all-rounder. Need maximum recall and can tolerate false alarms? Drop the threshold. The choice is a business decision about which error costs more.
The Full Imbalanced-Data Pipeline
from imblearn.pipeline import Pipeline # NOT sklearn
pipe = Pipeline([
('prep', preprocessor),
('smote', SMOTE(sampling_strategy=0.5)),
('model', RandomForestClassifier(
class_weight='balanced'))])
cv = StratifiedKFold(5, shuffle=True)
cross_val_score(pipe, X_train, y_train,
cv=cv, scoring='average_precision') # AUC-PR
pipe.fit(X_train, y_train)
joblib.dump({'pipeline': pipe,
'threshold': best}, 'model.pkl')sklearn's Pipeline would resample the test set too — catastrophic leakage. imblearn's applies SMOTE only inside fit() and skips it at predict time. This one import is the difference between honest and fake metrics.
Common Pitfalls & Their Fixes
| Symptom | The trap | The fix |
|---|---|---|
| 99% accuracy, 0 frauds caught | Optimising accuracy | Precision, recall, F1, AUC-PR |
| Perfect CV, collapses in prod | SMOTE before the split | Split first · imblearn Pipeline |
| Test set secretly resampled | sklearn Pipeline + SMOTE | Use imblearn Pipeline |
| A fold has zero frauds | Plain KFold | StratifiedKFold |
| Great model, poor results | Default 0.5 threshold | Tune on validation, save it |
| SMOTE makes near-duplicates | <10 minority samples | Class weights instead |
| Recall high, everyone complains | Ignoring false-alarm cost | Set threshold to the business cost |
The Golden Rules — 1 to 4
fit() and skips it at prediction.The Golden Rules — 5 to 8 & Takeaway
The zero-fraud model wasn't a modelling failure — it was a problem-formulation failure. Choose the right metric first, then the right treatment, then build the pipeline in the right order. Everything before the model decides whether it can even learn what matters.
⚖️ End of tutorial · Press ← to review, or click Restart