Hierarchical Clustering

The Story That Explains Hierarchical Clusteringवह कहानी जो Hierarchical Clustering समझाती है一个讲清层次聚类的故事

Hierarchical clustering is one of the oldest and easiest-to-explain clustering methods in unsupervised machine learning. Unlike K-Means — where you must choose the number of clusters before you begin — hierarchical clustering builds a full nested structure of groups and lets you choose the level of detail after you have seen the data.

Hierarchical clustering, unsupervised machine learning के सबसे पुराने और समझाने में सबसे आसान clustering तरीकों में से एक है। K-Means के उलट — जहाँ शुरू करने से पहले clusters की संख्या चुननी पड़ती है — hierarchical clustering groups का पूरा nested ढाँचा बनाता है और data देखने के बाद detail का स्तर चुनने देता है।

层次聚类是无监督机器学习中最古老、也最容易解释的聚类方法之一。与 K-Means 不同 —— K-Means 必须在开始前确定簇的个数 —— 层次聚类会构建一个完整的嵌套分组结构，让你在看过数据之后再决定要多细的分组。

Hierarchical Clustering in Pictures — Understand It Before the Mathsतस्वीरों में Hierarchical Clustering — गणित से पहले समझें图解层次聚类 — 先看懂，再学公式

Six words are enough to read every dendrogram on this page. Learn them from the pictures first.

इस page का हर dendrogram पढ़ने के लिए छह शब्द काफ़ी हैं। पहले इन्हें तस्वीरों से सीखें।

读懂本页所有树状图，只需要六个词。先通过图片学会它们。

Everyday trees you already knowरोज़ के पेड़ जिन्हें आप पहले से जानते हैं你早已熟悉的日常“树”

Two Flavours — Agglomerative vs Divisiveदो तरीके — Agglomerative vs Divisive两种方式 — 凝聚式 vs 分裂式

There are two directions to build a hierarchy: from the bottom up, or from the top down.

Hierarchy बनाने की दो दिशाएँ हैं: नीचे से ऊपर, या ऊपर से नीचे।

构建层次结构有两个方向：自底向上，或自顶向下。

Step by Step: How Agglomerative Clustering WorksStep by Step: Agglomerative Clustering कैसे काम करता है一步一步：凝聚式聚类如何运作

Now with 12 points on a mapअब map पर 12 points के साथ现在换成地图上的 12 个点

Linkage Criteria — How to Measure Distance Between ClustersLinkage Criteria — Clusters के बीच दूरी कैसे मापें连接准则 — 如何度量簇与簇之间的距离

Once two points form a cluster, how far is that cluster from the others? The rule that answers this is the linkage criterion. It is the most important choice in hierarchical clustering.

जब दो points एक cluster बना लेते हैं, तो वह cluster बाकियों से कितना दूर है? इसका जवाब देने वाला नियम linkage criterion है। Hierarchical clustering में यही सबसे ज़रूरी चुनाव है।

两个点合成一个簇之后，这个簇离其他簇有多远？回答这个问题的规则叫作连接准则（linkage criterion）。它是层次聚类中最重要的选择。

Distance Metrics — What Does "Similar" Mean?Distance Metrics — "मिलता-जुलता" का मतलब क्या?距离度量 — “相似”到底指什么？

Before measuring the distance between clusters, we need the distance between two points. The choice changes the result a lot.

Clusters के बीच दूरी मापने से पहले हमें दो points के बीच की दूरी चाहिए। यह चुनाव नतीजे को बहुत बदल देता है।

在度量簇间距离之前，我们需要先定义两点之间的距离。这个选择会极大地影响结果。

The Dendrogram — Reading It Like a ProDendrogram — Pro की तरह पढ़ना树状图 — 像专家一样阅读

Time and Memory — How Big Can the Data Be?समय और Memory — Data कितना बड़ा हो सकता है?时间与内存 — 数据能有多大？

Hierarchical clustering needs the distance between every pair of points. That is what limits it. Here are real timings from SciPy on 5-feature data:

Hierarchical clustering को points के हर जोड़े के बीच दूरी चाहिए। यही इसकी सीमा है। 5-feature data पर SciPy की असली timings ये हैं:

层次聚类需要每一对点之间的距离，这正是它的瓶颈。下面是 SciPy 在 5 个特征的数据上的真实计时：

import time
import numpy as np
from scipy.cluster.hierarchy import linkage
from sklearn.cluster import KMeans

rng = np.random.default_rng(0)
KMeans(5, n_init=1, random_state=0).fit(rng.normal(size=(500, 5)))   # warm-up run
print(f"{'n':>6} | {'distance matrix':>15} | {'Ward linkage':>12} | {'K-Means (K=5)':>13}")
for n in [1000, 2000, 4000, 8000, 16000]:
    X = rng.normal(size=(n, 5))
    mem_mb = n * (n - 1) / 2 * 8 / 1e6            # condensed matrix, 8 bytes per float
    t0 = time.time(); linkage(X, method='ward'); t_hc = time.time() - t0
    t0 = time.time(); KMeans(5, n_init=1, random_state=0).fit(X); t_km = time.time() - t0
    print(f"{n:>6} | {mem_mb:>12.0f} MB | {t_hc:>10.2f} s | {t_km:>11.2f} s")

Each time n doubles, the memory grows 4× and the Ward time grows roughly 4× too — that is O(n²). K-Means stays almost flat. (Your times will differ by machine; the growth pattern will not.)

हर बार n दोगुना होने पर memory 4× बढ़ती है और Ward का समय भी लगभग 4× — यही O(n²) है। K-Means लगभग सपाट रहता है। (आपकी machine पर समय अलग होगा; बढ़ने का pattern नहीं।)

n 每翻一倍，内存增长 4 倍，Ward 的耗时也大约增长 4 倍 —— 这就是 O(n²)。K-Means 几乎不变。（不同机器的耗时会不同，但增长规律不会变。）

Evaluating Cluster QualityCluster Quality जाँचना评估聚类质量

There are no true labels, so we use internal scores that only look at the data and the clusters.

कोई असली labels नहीं हैं, इसलिए हम internal scores इस्तेमाल करते हैं जो सिर्फ़ data और clusters देखते हैं।

没有真实标签，所以我们使用只看数据和簇本身的内部指标。

Python Implementation — Full WalkthroughPython Implementation — पूरा WalkthroughPython 实现 — 完整演练

A complete implementation using scipy (for the dendrogram) and scikit-learn (for pipelines). Every output below is the real output of the code.

scipy (dendrogram के लिए) और scikit-learn (pipelines के लिए) से पूरा implementation. नीचे का हर output code का असली output है।

使用 scipy（画树状图）和 scikit-learn（用于流水线）的完整实现。下面每个输出都是代码真实运行的结果。

Step 1 — Import Libraries and Generate DataStep 1 — Libraries import करें और Data बनाएँ第 1 步 — 导入库并生成数据

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# scipy — dendrogram plotting
from scipy.cluster.hierarchy import dendrogram, linkage, fcluster
from scipy.spatial.distance   import pdist

# sklearn — clustering + evaluation
from sklearn.cluster           import AgglomerativeClustering
from sklearn.preprocessing     import StandardScaler
from sklearn.metrics           import silhouette_score, davies_bouldin_score
from sklearn.datasets          import make_blobs

# Reproducibility
np.random.seed(42)

# Generate 3-cluster synthetic data
X, y_true = make_blobs(
    n_samples=150,
    centers=3,
    cluster_std=0.8,
    random_state=42
)

print(f"Dataset shape: {X.shape}")

Step 2 — Scale FeaturesStep 2 — Features scale करें第 2 步 — 缩放特征

# Always scale before distance-based algorithms
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

print("Mean per feature:  ", X_scaled.mean(axis=0).round(4) + 0.0)   # + 0.0 turns -0.0 into 0.0
print("Std  per feature:  ", X_scaled.std(axis=0).round(4))

Step 3 — Build the Linkage Matrix and Plot the DendrogramStep 3 — Linkage Matrix बनाएँ और Dendrogram plot करें第 3 步 — 构建连接矩阵并画树状图

# Build the full linkage matrix using Ward's method
Z = linkage(X_scaled, method='ward', metric='euclidean')

# Z is an (n-1) x 4 matrix; each row:
# [cluster_i, cluster_j, distance, count_in_new_cluster]
np.set_printoptions(suppress=True)          # plain numbers, no 1e+01
print("Linkage matrix shape:", Z.shape)
print("First 3 merges:\n", Z[:3].round(3))
print("Last 3 merge heights:", Z[-3:, 2].round(2))

# ── Plot the dendrogram ─────────────────────────────────────
plt.figure(figsize=(14, 6))
plt.title('Hierarchical Clustering Dendrogram (Ward Linkage)', fontsize=14)
plt.xlabel('Sample Index')
plt.ylabel('Distance (Ward)')

dendrogram(
    Z,
    color_threshold=5.0,     # 5 sits inside the big gap (1.49 → 13.06) → 3 colours
    leaf_font_size=8,
    above_threshold_color='grey'
)

# Draw a horizontal cut line at height = 5
plt.axhline(y=5.0, color='red', linestyle='--', label='Cut threshold')
plt.legend()
plt.tight_layout()
plt.show()

Each row of Z is one merge: [cluster i, cluster j, merge height, size of the new cluster]. Ids 0–149 are the original points; id 150 is the first new cluster, and so on. The last three heights (1.49 → 13.06 → 20.43) show a huge gap, which is why color_threshold=5.0 gives 3 colours.

Z की हर row एक merge है: [cluster i, cluster j, merge height, नए cluster का size]. Ids 0–149 असली points हैं; id 150 पहला नया cluster है, और इसी तरह आगे। आख़िरी तीन heights (1.49 → 13.06 → 20.43) एक बड़ा gap दिखाती हैं, इसीलिए color_threshold=5.0 3 रंग देता है।

Z 的每一行是一次合并：[簇 i, 簇 j, 合并高度, 新簇大小]。编号 0–149 是原始点；150 是第一个新簇，依此类推。最后三个高度（1.49 → 13.06 → 20.43）之间有巨大间隙，所以 color_threshold=5.0 会得到 3 种颜色。

Step 4 — Extract Clusters and EvaluateStep 4 — Clusters निकालें और जाँचें第 4 步 — 提取簇并评估

# ── Method A: scipy fcluster (cut dendrogram at K=3) ───────
labels_scipy = fcluster(Z, t=3, criterion='maxclust')

# ── Method B: sklearn AgglomerativeClustering ──────────────
# Production-grade, integrates with sklearn pipelines
hac = AgglomerativeClustering(
    n_clusters=3,
    linkage='ward',
    metric='euclidean'
)
labels_sklearn = hac.fit_predict(X_scaled)

# ── Evaluation metrics ──────────────────────────────────────
sil  = silhouette_score(X_scaled, labels_sklearn)
dbi  = davies_bouldin_score(X_scaled, labels_sklearn)

print(f"Silhouette Score  : {sil:.4f}  (higher = better, max=1.0)")
print(f"Davies-Bouldin    : {dbi:.4f}  (lower  = better, min=0.0)")
print(f"Cluster sizes     : {np.bincount(labels_sklearn)}")

Step 5 — Silhouette for Every Cut to Find the Best KStep 5 — सबसे अच्छा K खोजने के लिए हर Cut का Silhouette第 5 步 — 对每种切法计算轮廓系数，找出最佳 K

# Cut dendrogram at K = 2 to 8 and score each
k_range = range(2, 9)
sil_scores = []

for k in k_range:
    labels = fcluster(Z, t=k, criterion='maxclust')
    sil_scores.append(silhouette_score(X_scaled, labels))

best_k  = k_range[int(np.argmax(sil_scores))]
best_sc = max(sil_scores)
print(f"Best K = {best_k}  →  Silhouette = {best_sc:.4f}")

for k, sc in zip(k_range, sil_scores):
    bar = '█' * int(sc * 50)
    marker = ' ← BEST' if k == best_k else ''
    print(f"  K={k}  {bar}  {sc:.4f}{marker}")

K = 3 wins clearly (0.88), matching the 3 blobs we generated and the big gap in the dendrogram.

K = 3 साफ़ जीतता है (0.88), जो हमारे बनाए 3 blobs और dendrogram के बड़े gap से मेल खाता है।

K = 3 明显胜出（0.88），与我们生成的 3 个团以及树状图中的大间隙一致。

Hierarchical vs K-Means — When to Use WhichHierarchical vs K-Means — कब कौन सा层次聚类 vs K-Means — 何时用哪个

Common Pitfalls and How to Avoid Themआम ग़लतियाँ और उनसे कैसे बचें常见陷阱及规避方法

🏪 Customer Segmentation for a Retail BankRetail Bank के लिए Customer Segmentation零售银行的客户分群

Step 1 — Load and ExploreStep 1 — Load करें और देखें第 1 步 — 加载并查看

import numpy as np
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import AgglomerativeClustering
from sklearn.metrics import silhouette_score
from scipy.cluster.hierarchy import dendrogram, linkage, fcluster
import matplotlib.pyplot as plt

rng = np.random.default_rng(42)

# Simulated bank customers from 4 hidden behaviour types.
# (Real data would come from the bank; the algorithm never sees 'types'.)
types = {          #  age  income  credit  products  monthly_txns  count
    'professional': (45,  85000,   745,     4,        55,          180),
    'budget':       (42,  32000,   600,     2,        20,          220),
    'digital':      (29,  52000,   690,     3,        75,          200),
    'pre_retire':   (60,  50000,   665,     2,        14,          200),
}
parts = []
for name, (age, inc, cred, prod, txn, m) in types.items():
    parts.append(pd.DataFrame({
        'age':           rng.normal(age, 4, m).round().clip(21, 75),
        'annual_income': rng.normal(inc, inc * 0.12, m).round(-2),
        'credit_score':  rng.normal(cred, 25, m).round().clip(300, 850),
        'num_products':  rng.poisson(prod, m).clip(1, 8),
        'monthly_txns':  rng.normal(txn, 6, m).round().clip(1, None),
    }))
df = pd.concat(parts, ignore_index=True).sample(frac=1, random_state=42).reset_index(drop=True)

print(df.describe().round(1).loc[['count', 'mean', 'std', 'min', 'max']])

Step 2 — Preprocess and Choose KStep 2 — Preprocess करें और K चुनें第 2 步 — 预处理并选择 K

# Scale all features to zero mean, unit variance
scaler  = StandardScaler()
X_scaled = scaler.fit_transform(df)

# Build Ward linkage matrix
Z = linkage(X_scaled, method='ward')

# Test K from 2 to 8
results = []
for k in range(2, 9):
    labels = fcluster(Z, t=k, criterion='maxclust')
    sil    = silhouette_score(X_scaled, labels)
    results.append({'k': k, 'silhouette': round(sil, 4)})

best = max(results, key=lambda x: x['silhouette'])
print(f"{'K':>4}  {'Silhouette':>12}")
for r in results:
    star = ' ← BEST' if r['k'] == best['k'] else ''
    print(f"  {r['k']:>2}  {r['silhouette']:>12.4f}{star}")

K = 4 has the highest silhouette (0.47), but K = 3 and 5 are close. That is normal for real customer data: the silhouette says "about 3–5 groups", and the dendrogram gap plus business sense pick the final K. A silhouette around 0.45 means real but overlapping groups.

K = 4 का silhouette सबसे ऊँचा है (0.47), पर K = 3 और 5 पास हैं। असली customer data में यह आम है: silhouette कहता है "लगभग 3–5 groups", और dendrogram gap व business की समझ आख़िरी K चुनते हैं। लगभग 0.45 का silhouette मतलब असली पर कुछ overlap वाले groups.

K = 4 的轮廓系数最高（0.47），但 K = 3 和 5 也很接近。这在真实客户数据中很常见：轮廓系数告诉你“大约 3–5 组”，再由树状图间隙和业务判断决定最终的 K。0.45 左右的轮廓系数意味着组是真实存在的，但有一定重叠。

Step 3 — Final Clustering (K = 4) and Segment ProfilesStep 3 — Final Clustering (K = 4) और Segment Profiles第 3 步 — 最终聚类（K = 4）与客户群画像

# Apply final model with K=4
hac = AgglomerativeClustering(n_clusters=4, linkage='ward')
df['segment'] = hac.fit_predict(X_scaled)

# Profile each segment with mean values
profile = df.groupby('segment').agg(
    count=('age', 'size'),
    avg_age=('age', 'mean'),
    avg_income=('annual_income', 'mean'),
    avg_credit=('credit_score', 'mean'),
    avg_products=('num_products', 'mean'),
    avg_txns=('monthly_txns', 'mean')
).round(1)

print(profile.to_string())

Step 4 — Interpret and Name the SegmentsStep 4 — Segments समझें और नाम दें第 4 步 — 解读并命名客户群

🧬 Gene Expression Clustering in BioinformaticsBioinformatics में Gene Expression Clustering生物信息学中的基因表达聚类

Step 1 — Simulate Gene Expression DataStep 1 — Gene Expression Data simulate करें第 1 步 — 模拟基因表达数据

import numpy as np
import pandas as pd
from scipy.cluster.hierarchy import dendrogram, linkage, fcluster
from scipy.spatial.distance   import pdist, squareform
from sklearn.metrics           import silhouette_score
import matplotlib.pyplot       as plt
import seaborn                  as sns

np.random.seed(99)

conditions = ['D0', 'D1', 'D2', 'D4', 'D7', 'D14', 'D21', 'D28']

# Create 3 co-expression groups of ~17 genes each
# Group A: genes that peak early then drop off
base_A = np.array([3.0, 5.2, 7.8, 9.1, 6.4, 3.5, 1.8, 1.2])
# Group B: genes that rise late and stay high
base_B = np.array([1.0, 1.2, 1.5, 4.8, 6.2, 7.1, 7.4, 7.9])
# Group C: genes that are switched off step by step
base_C = np.array([8.5, 7.2, 5.1, 3.3, 2.1, 1.5, 1.0, 0.8])

rows = []
for i in range(17):
    rows.append(base_A + np.random.normal(0, 0.4, 8))
for i in range(17):
    rows.append(base_B + np.random.normal(0, 0.4, 8))
for i in range(16):
    rows.append(base_C + np.random.normal(0, 0.4, 8))

gene_names = [f"GENE_{i+1:02d}" for i in range(50)]
expr = pd.DataFrame(rows, index=gene_names, columns=conditions)

print("Expression matrix shape:", expr.shape)
print(expr.head(3).round(2))

Step 2 — Correlation Distance MatrixStep 2 — Correlation Distance Matrix第 2 步 — 相关距离矩阵

# 1 − Pearson correlation as distance between genes
corr_dist = pdist(expr.values, metric='correlation')
corr_matrix = squareform(corr_dist)

print("Distance matrix shape:", corr_matrix.shape)
print(f"Min dist: {corr_dist.min():.4f}  |  Max dist: {corr_dist.max():.4f}")
print(f"Median dist: {np.median(corr_dist):.4f}")

Step 3 — Cluster Genes with Average LinkageStep 3 — Average Linkage से Genes cluster करें第 3 步 — 用平均连接对基因聚类

# Average linkage on precomputed correlation distances
Z_genes = linkage(corr_dist, method='average')

# Find optimal K via silhouette
best_k, best_sil = 2, -1
for k in range(2, 8):
    labels = fcluster(Z_genes, t=k, criterion='maxclust')
    sil    = silhouette_score(corr_matrix, labels, metric='precomputed')
    if sil > best_sil:
        best_k, best_sil = k, sil
    print(f"  K={k}  silhouette={sil:.4f}")

print(f"\nOptimal K = {best_k}  (silhouette = {best_sil:.4f})")

# Apply final clustering
gene_labels = fcluster(Z_genes, t=best_k, criterion='maxclust')
expr['cluster'] = gene_labels

Step 4 — Biological InterpretationStep 4 — Biological व्याख्या第 4 步 — 生物学解读

for c in sorted(expr['cluster'].unique()):
    genes_in = expr[expr['cluster'] == c].drop('cluster', axis=1)
    mean_expr = genes_in.mean()
    print(f"\n── Cluster {c}  ({len(genes_in)} genes) ──────────────")
    print("Mean expression per timepoint:")
    for cond, val in mean_expr.items():
        bar = '■' * int(val)
        print(f"  {cond:>4}: {bar:<12} {val:.2f}")
    print(f"  Genes: {', '.join(genes_in.index[:5].tolist())} ...")
# Picture: clustered heatmap with the same dendrogram on the rows
# sns.clustermap(expr.drop(columns="cluster"), row_linkage=Z_genes, col_cluster=False, cmap="viridis")

Note: fcluster numbers clusters 1, 2, 3 in its own order, so "Cluster 1" here is the late-rising group (genes 18–34), not genes 1–17. Always read the profile, never assume the number.

ध्यान दें: fcluster clusters को अपने क्रम में 1, 2, 3 number देता है, इसलिए यहाँ "Cluster 1" देर से बढ़ने वाला group है (genes 18–34), genes 1–17 नहीं। हमेशा profile पढ़ें, number पर भरोसा न करें।

注意：fcluster 按自己的顺序给簇编号 1、2、3，所以这里的“Cluster 1”是后期上升组（基因 18–34），而不是基因 1–17。一定要看画像，不要想当然地看编号。

scikit-learn API Quick Referencescikit-learn API Quick Referencescikit-learn API 速查

Run on the scaled bank data from Case Study 01:

Case Study 01 के scaled bank data पर चलाया गया:

在案例 01 缩放后的银行数据上运行：

from sklearn.cluster import AgglomerativeClustering

# Full parameter reference
model = AgglomerativeClustering(
    n_clusters=4,          # int or None (if distance_threshold used)
    metric='euclidean',    # 'euclidean','manhattan','cosine','precomputed'
    linkage='ward',        # 'ward','complete','average','single'
    distance_threshold=None, # set this instead of n_clusters to cut by distance
    connectivity=None,     # connectivity matrix (e.g. from kneighbors_graph)
    compute_full_tree='auto',# 'auto', True, False
    compute_distances=False  # set True if you need .distances_ attribute
)

labels = model.fit_predict(X_scaled)

# Useful post-fit attributes
print("n_clusters_ : ", model.n_clusters_)    # actual number of clusters
print("n_leaves_   : ", model.n_leaves_)      # number of leaves (= n samples)
print("n_connected_: ", model.n_connected_components_) # connectivity components

Golden Rules — Hierarchical ClusteringGolden Rules — Hierarchical Clustering黄金法则 — 层次聚类