Rosenblatt's Perceptron Algorithm

The Story That Separates Deep Learning from ML

Classical Machine Learning is a toolkit of mathematical models (logistic regression, SVMs, decision trees, Random Forest) that learn patterns from structured, human-prepared features. A data scientist must decide which features to extract before training even begins.

Deep Learning is a sub-field of ML that uses layered artificial neural networks to learn hierarchical feature representations directly from raw data. The network builds its own internal vocabulary — edges, shapes, textures, concepts — layer by layer, without being told what to look for.

The Hierarchy — How They Fit Together

Before going further, a quick map so you never confuse the terms:

Feature Engineering — The Dividing Line

Inside a Neural Network — The Core Mechanism

A neural network is built from layers of neurons. Each neuron receives inputs, multiplies them by learned weights, adds a bias, and passes the result through an activation function. Stack enough of these layers and the network can approximate any function — a property called the Universal Approximation Theorem.

How Learning Happens — Backpropagation

ML vs Deep Learning — Side-by-Side Comparison

Real-World Examples — Where Each Wins

Diagram — Layers Learning Representations

The power of depth is best understood by watching what each layer actually learns in a computer vision network.

Python Code — Classical ML vs Deep Learning on the Same Task

Let's train both approaches on the MNIST handwritten digit dataset (28×28 pixel greyscale images, 10 classes, 60K train / 10K test). The contrast shows exactly where the work lives in each paradigm.

Part A — Classical ML (Random Forest on flattened pixels)

# ── Classical ML approach: flatten image → feature vector → model ──
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
from tensorflow.keras.datasets import mnist

# Load MNIST (28×28 grayscale images)
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# ── Feature Engineering (manual): flatten 28×28 = 784 pixel values
X_train_flat = X_train.reshape(-1, 784) / 255.0  # normalise 0–1
X_test_flat  = X_test.reshape(-1, 784)  / 255.0

# ── No deeper feature design — we hand the raw pixels to the model
rf = RandomForestClassifier(n_estimators=100, n_jobs=-1, random_state=42)
rf.fit(X_train_flat, y_train)

y_pred = rf.predict(X_test_flat)
print(f"Random Forest Accuracy: {accuracy_score(y_test, y_pred):.4f}")

Part B — Deep Learning (CNN — learns its own features)

# ── Deep Learning approach: raw pixels → CNN learns features itself ──
import numpy as np
import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.datasets import mnist

# Load and reshape for CNN (needs channel dimension)
(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train = X_train[..., np.newaxis] / 255.0  # shape: (60000, 28, 28, 1)
X_test  = X_test[...,  np.newaxis] / 255.0

# ── Architecture: no manual features — the Conv layers find them ──
model = models.Sequential([
    # Block 1 — learns edges and simple patterns
    layers.Conv2D(32, (3,3), activation='relu', input_shape=(28,28,1)),
    layers.MaxPooling2D((2,2)),

    # Block 2 — learns higher-order shapes
    layers.Conv2D(64, (3,3), activation='relu'),
    layers.MaxPooling2D((2,2)),

    # Flatten and classify
    layers.Flatten(),
    layers.Dense(64, activation='relu'),
    layers.Dropout(0.3),
    layers.Dense(10, activation='softmax')  # 10 digit classes
])

model.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

model.fit(X_train, y_train,
         epochs=5,
         batch_size=128,
         validation_split=0.1,
         verbose=1)

test_loss, test_acc = model.evaluate(X_test, y_test, verbose=0)
print(f"CNN Accuracy: {test_acc:.4f}")

When to Use Which — Decision Guide

Common Deep Learning Architectures — Quick Map