Quartiles, IQR and Outlier Detection

The Story That Explains Everythingवह कहानी जो सब कुछ समझाती है一个讲清一切的故事

You are an HR analyst at a tech company. The CEO asks you for the "typical" salary. You pull the data for 12 employees:

आप एक tech company में HR analyst हैं। CEO आपसे "आम" salary पूछते हैं। आप 12 कर्मचारियों का data निकालते हैं:

你是一家科技公司的 HR 分析师。CEO 让你报告“典型”工资。你调出了 12 名员工的数据：

The mean salary is £88,917. You report this to the CEO. Every junior developer in the room is confused — they earn about £33,000, and the "average" is £88,917? The CTO's £380,000 has dragged the mean above what 9 of the 12 employees earn.

Mean salary £88,917 है। आप यही CEO को बताते हैं। कमरे में हर junior developer उलझन में है — वे लगभग £33,000 कमाते हैं, और "average" £88,917? CTO के £380,000 ने mean को 12 में से 9 कर्मचारियों की कमाई से ऊपर खींच दिया है।

平均工资是 £88,917。你把这个数报给了 CEO。会议室里每位初级开发都很困惑 —— 他们只挣约 £33,000，“平均”却是 £88,917？ CTO 的 £380,000 把均值拉到了 12 人中 9 人的工资之上。

What Are Quartiles?Quartiles क्या हैं?什么是四分位数？

Quartiles split a sorted dataset into four parts, each holding about 25% of the values. There are three quartile values:

Quartiles एक sort किए dataset को चार हिस्सों में बाँटते हैं, हर हिस्से में लगभग 25% values. तीन quartile values होती हैं:

四分位数把排好序的数据分成四份，每份约含 25% 的数值。共有三个四分位数：

What Is the IQR?IQR क्या है?什么是 IQR？

The Interquartile Range (IQR) is the distance between Q1 and Q3. It covers the middle 50% of the data — the typical range — and ignores the extreme low and high values.

Interquartile Range (IQR) Q1 और Q3 के बीच की दूरी है। यह data के बीच के 50% को cover करता है — आम range — और बहुत नीची व बहुत ऊँची values को अनदेखा करता है।

四分位距（IQR）是 Q1 到 Q3 的距离。它涵盖数据中间的 50% —— 典型范围 —— 并忽略极低和极高的值。

Step-by-Step CalculationStep-by-Step गणना分步计算

Let us use a simpler dataset first: [4, 7, 9, 12, 15, 18, 21, 24, 28, 35, 38, 45]

पहले एक आसान dataset लें: [4, 7, 9, 12, 15, 18, 21, 24, 28, 35, 38, 45]

先用一个简单的数据集：[4, 7, 9, 12, 15, 18, 21, 24, 28, 35, 38, 45]

How to Detect Outliers Using the IQRIQR से Outliers कैसे पकड़ें如何用 IQR 检测离群点

The IQR method uses fences — boundaries beyond which a value is flagged as an outlier. Box plots use exactly this rule.

IQR तरीका fences इस्तेमाल करता है — ऐसी सीमाएँ जिनके बाहर की value outlier मानी जाती है। Box plots ठीक यही नियम इस्तेमाल करते हैं।

IQR 方法使用围栏（fences） —— 超出这些边界的值被标记为离群点。箱线图用的正是这条规则。

Applying the fences to the salary dataSalary data पर fences लगाना把围栏用到工资数据上

Python ImplementationPython ImplementationPython 实现

Manual calculation (hand method)हाथ से गणना (हाथ वाला तरीका)手动计算（手算方法）

salaries = [32000, 33000, 34000, 48000, 49000, 51000,
            72000, 75000, 78000, 95000, 120000, 380000]

salaries_sorted = sorted(salaries)
n = len(salaries_sorted)

# Q2 — median
mid = n // 2
q2 = (salaries_sorted[mid - 1] + salaries_sorted[mid]) / 2

# Q1 — median of lower half
lower_half = salaries_sorted[:mid]
q1 = (lower_half[len(lower_half)//2 - 1] + lower_half[len(lower_half)//2]) / 2

# Q3 — median of upper half
upper_half = salaries_sorted[mid:]
q3 = (upper_half[len(upper_half)//2 - 1] + upper_half[len(upper_half)//2]) / 2

iqr = q3 - q1

lower_fence = q1 - 1.5 * iqr
upper_fence = q3 + 1.5 * iqr

print(f"Q1: £{q1:,.0f}")
print(f"Q2: £{q2:,.0f}")
print(f"Q3: £{q3:,.0f}")
print(f"IQR: £{iqr:,.0f}")
print(f"Lower fence: £{lower_fence:,.0f}")
print(f"Upper fence: £{upper_fence:,.0f}")

outliers = [s for s in salaries if s < lower_fence or s > upper_fence]
print(f"Outliers: {outliers}")

Using NumPyNumPy से使用 NumPy

import numpy as np

salaries = [32000, 33000, 34000, 48000, 49000, 51000,
            72000, 75000, 78000, 95000, 120000, 380000]

q1  = np.percentile(salaries, 25)
q2  = np.percentile(salaries, 50)
q3  = np.percentile(salaries, 75)
iqr = q3 - q1

lower_fence = q1 - 1.5 * iqr
upper_fence = q3 + 1.5 * iqr

salaries_arr = np.array(salaries)
outliers = salaries_arr[(salaries_arr < lower_fence) | (salaries_arr > upper_fence)]

print(f"Q1:  £{q1:,.0f}")
print(f"Q2:  £{q2:,.0f}")
print(f"Q3:  £{q3:,.0f}")
print(f"IQR: £{iqr:,.0f}")
print(f"Upper fence: £{upper_fence:,.0f}")
print(f"Outliers: {outliers}")

# Why are these different from the hand calculation (41,000 and 86,500)?
# NumPy's default method interpolates between data points ("linear").
# Other textbook methods give other answers — none of them is "wrong":
for m in ["linear", "midpoint", "lower", "higher"]:
    a, b = np.percentile(salaries, [25, 75], method=m)
    print(f"method={m:9s} Q1 = £{a:,.0f}   Q3 = £{b:,.0f}")

Using pandaspandas से使用 pandas

import pandas as pd

salaries = [32000, 33000, 34000, 48000, 49000, 51000,
            72000, 75000, 78000, 95000, 120000, 380000]

df = pd.DataFrame({"salary": salaries})

q1  = df["salary"].quantile(0.25)
q3  = df["salary"].quantile(0.75)
iqr = q3 - q1

lower_fence = q1 - 1.5 * iqr
upper_fence = q3 + 1.5 * iqr

# Flag outliers as a new column
df["is_outlier"] = (df["salary"] < lower_fence) | (df["salary"] > upper_fence)

print(df)
print(f"\nOutlier rows:")
print(df[df["is_outlier"]])

Visualising with a box plotBox plot से दिखाना用箱线图可视化

import matplotlib.pyplot as plt
import numpy as np

salaries = [32000, 33000, 34000, 48000, 49000, 51000,
            72000, 75000, 78000, 95000, 120000, 380000]

fig, axes = plt.subplots(1, 2, figsize=(12, 5))

# Box plot — outliers shown as circles beyond the whiskers
axes[0].boxplot(salaries, vert=True, patch_artist=True,
                boxprops=dict(facecolor="#3776AB", alpha=0.6))
axes[0].set_title("Salary Box Plot")
axes[0].set_ylabel("Salary (£)")
axes[0].yaxis.set_major_formatter(
    plt.FuncFormatter(lambda x, _: f"£{x:,.0f}")
)

# Same data without outlier for comparison
salaries_clean = [s for s in salaries if s <= 154750]
axes[1].boxplot(salaries_clean, vert=True, patch_artist=True,
                boxprops=dict(facecolor="#34d399", alpha=0.6))
axes[1].set_title("Salaries Without Outlier")
axes[1].set_ylabel("Salary (£)")
axes[1].yaxis.set_major_formatter(
    plt.FuncFormatter(lambda x, _: f"£{x:,.0f}")
)

plt.suptitle("IQR Outlier Detection — Box Plots", fontsize=13)
plt.tight_layout()
plt.show()

This draws two box plots side by side. Matplotlib uses the same 1.5 × IQR rule for the whiskers (whis=1.5) and shows points beyond them as circles.

यह दो box plots साथ-साथ बनाता है। Matplotlib whiskers के लिए वही 1.5 × IQR नियम (whis=1.5) इस्तेमाल करता है और उनसे बाहर के points को गोलों के रूप में दिखाता है।

这段代码并排画出两个箱线图。Matplotlib 的须线使用同样的 1.5 × IQR 规则（whis=1.5），超出的点以圆圈显示。

What to Do When You Find OutliersOutliers मिलें तो क्या करें发现离群点后该怎么办

Finding an outlier is only the first step. What you do next depends on why it exists:

Outlier मिलना सिर्फ़ पहला कदम है। आगे क्या करना है, यह इस पर निर्भर है कि वह क्यों है:

发现离群点只是第一步。接下来怎么做，取决于它为什么存在：

Remove, cap, or use the medianहटाएँ, cap करें, या median इस्तेमाल करें删除、截断，还是用中位数

import numpy as np

salaries = [32000, 33000, 34000, 48000, 49000, 51000,
            72000, 75000, 78000, 95000, 120000, 380000]

q1  = np.percentile(salaries, 25)
q3  = np.percentile(salaries, 75)
iqr = q3 - q1

lower_fence = q1 - 1.5 * iqr
upper_fence = q3 + 1.5 * iqr

# Option 1 — Remove outliers
clean = [s for s in salaries if lower_fence <= s <= upper_fence]
print(f"After removal: {len(clean)} values, mean = £{sum(clean)/len(clean):,.0f}")

# Option 2 — Cap outliers (Winsorisation)
# Replace outlier with the fence value instead of removing
capped = [max(lower_fence, min(upper_fence, s)) for s in salaries]
print(f"After capping: mean = £{sum(capped)/len(capped):,.0f}")

# Option 3 — Use median (unaffected by outliers)
median = np.median(salaries)
print(f"Median (no action needed): £{median:,.0f}")

Capping (also called winsorising) replaces the CTO's £380,000 with the upper fence (£138,875 with NumPy's quartiles), so the row stays in the data but can no longer dominate the mean.

Capping (जिसे winsorising भी कहते हैं) CTO के £380,000 को upper fence (NumPy के quartiles से £138,875) से बदल देता है, ताकि row data में रहे पर mean पर हावी न हो सके।

截断（也叫缩尾，winsorising）把 CTO 的 £380,000 替换成上围栏（用 NumPy 四分位数时为 £138,875），这一行留在数据里，却再也无法主导均值。

IQR vs Z-Score — Two Methods ComparedIQR vs Z-Score — दो तरीकों की तुलनाIQR vs Z 分数 — 两种方法对比

import numpy as np

salaries = [32000, 33000, 34000, 48000, 49000, 51000,
            72000, 75000, 78000, 95000, 120000, 380000]

# ── IQR Method ──────────────────────────────────────────────
q1, q3  = np.percentile(salaries, [25, 75])
iqr     = q3 - q1
iqr_outliers = [s for s in salaries
                if s < q1 - 1.5*iqr or s > q3 + 1.5*iqr]
print(f"IQR outliers:               {iqr_outliers}")

# ── Z-Score Method ───────────────────────────────────────────
mean = np.mean(salaries)
std  = np.std(salaries, ddof=1)
z_scores = [(s - mean) / std for s in salaries]
z_outliers = [s for s, z in zip(salaries, z_scores) if abs(z) > 2]
print(f"Z-score outliers (|z| > 2): {z_outliers}")
print(f"CTO z-score: {z_scores[-1]:.2f}")
print(f"Z-score outliers (|z| > 3): {[s for s, z in zip(salaries, z_scores) if abs(z) > 3]}")

# With n values, no point can ever have |z| above (n - 1) / sqrt(n)
n = len(salaries)
print(f"Largest possible |z| with n = {n}: {(n - 1) / np.sqrt(n):.2f}")

Here both rules catch the CTO, but only just: the CTO's z-score is 3.05, and with 12 values no point can ever exceed |z| = 3.18. Push the CTO salary in the simulator below and watch the two rules behave differently.

यहाँ दोनों नियम CTO को पकड़ते हैं, पर मुश्किल से: CTO का z-score 3.05 है, और 12 values में कोई point कभी |z| = 3.18 से ऊपर नहीं जा सकता। नीचे के simulator में CTO की salary बदलें और दोनों नियमों का अलग व्यवहार देखें।

这里两条规则都抓到了 CTO，但只是勉强：CTO 的 z 分数是 3.05，而 12 个值里任何点的 |z| 都不可能超过 3.18。在下面的模拟器里调整 CTO 的工资，看两条规则表现有何不同。

When the Fences Fire Too Often — Skewed Dataजब Fences बहुत बार चलें — Skewed Data围栏触发太频繁时 — 偏斜数据

Complete Pipeline — Detect, Investigate, Handleपूरी Pipeline — पकड़ें, जाँचें, संभालें完整流程 — 检测、调查、处理

import numpy as np
import pandas as pd

# Full dataset — house prices in a neighbourhood (£)
prices = [185000, 192000, 178000, 205000, 195000, 210000,
          188000, 201000, 198000, 185000, 2100000, 175000]

df = pd.DataFrame({"price": prices})

# Step 1 — Calculate quartiles and IQR
q1  = df["price"].quantile(0.25)
q2  = df["price"].quantile(0.50)
q3  = df["price"].quantile(0.75)
iqr = q3 - q1

lower_fence = q1 - 1.5 * iqr
upper_fence = q3 + 1.5 * iqr

print("── Summary Statistics ──────────────────")
print(f"Q1 (25th percentile): £{q1:,.0f}")
print(f"Q2 (Median):          £{q2:,.0f}")
print(f"Q3 (75th percentile): £{q3:,.0f}")
print(f"IQR:                  £{iqr:,.0f}")
print(f"Lower fence:          £{lower_fence:,.0f}")
print(f"Upper fence:          £{upper_fence:,.0f}")

# Step 2 — Flag outliers
df["is_outlier"] = (
    (df["price"] < lower_fence) |
    (df["price"] > upper_fence)
)

print("\n── Outliers Detected ───────────────────")
print(df[df["is_outlier"]])

# Step 3 — Compare mean before and after
mean_with    = df["price"].mean()
mean_without = df[~df["is_outlier"]]["price"].mean()
median       = df["price"].median()

print("\n── Impact of Outlier ───────────────────")
print(f"Mean with outlier:    £{mean_with:,.0f}")
print(f"Mean without outlier: £{mean_without:,.0f}")
print(f"Median (robust):      £{median:,.0f}")

One £2.1 million house lifts the mean from £192,000 to £351,000, while the median stays at £193,500. Before deciding what to do, check the listing: is it a typo (£210,000?) or a real mansion in the same street?

एक £2.1 million का घर mean को £192,000 से £351,000 कर देता है, जबकि median £193,500 पर रहता है। क्या करना है, यह तय करने से पहले listing जाँचें: क्या यह typo है (£210,000?) या उसी गली में सचमुच का बँगला?

一套 £210 万的房子把均值从 £192,000 拉到 £351,000，而中位数仍是 £193,500。决定怎么处理之前，先核对挂牌信息：是笔误（£210,000？），还是同一条街上真有一座豪宅？

Golden RulesGolden Rules黄金法则