Deep Learning vs Machine Learning
Press Next → or use ← → arrow keys
Hand-Crafted vs Learned Features
All deep learning is machine learning — it just adds layered neural networks that learn hierarchical representations straight from raw data. But not all ML is deep learning.
The AI Family Tree
Each layer lives inside the one above it. Today's LLMs are deep-learning models, which are machine-learning models, which are a form of AI — a set of nesting dolls, not competing fields.
Feature Engineering — the Dividing Line
Classical ML is a kitchen of specialist chefs handing the machine tidy measurements — sweetness, salt, texture. Deep learning is a robot that tastes the raw ingredients and works out the balance itself.
| Classical ML pipeline | by |
|---|---|
| Collect raw data | engineer |
| Hand-extract features | expert |
| Scale / encode | scientist |
| Map features → prediction | algorithm |
| Deep-learning pipeline | by |
|---|---|
| Collect raw data | engineer |
| Feed raw data into network | algorithm |
| Layer 1 learns edges | network |
| Layer N learns faces → prediction | network |
Deep learning removes the hand-engineering — but demands thousands to millions of labelled examples and serious GPU hours. Classical ML often learns from a few hundred rows on a laptop.
Inside a Neuron
Why Activations Matter
Stack linear layers and the whole thing collapses to a single linear map — no matter how deep. The non-linear activation between layers is what lets a network bend, curve and compose simple parts into complex concepts. ReLU is the modern default; sigmoid/softmax live at the output for probabilities.
How Learning Happens — Backpropagation
Picture a faulty product at the end of an assembly line. The manager walks the blame backwards through every worker. Backprop does exactly that with calculus — pushing the error back through the layers and assigning each weight its share.
Each epoch: predict forward, measure the loss, backprop the gradients, let the optimiser (SGD, Adam) nudge every weight downhill. After enough epochs the loss flattens — the network has converged.
ML vs Deep Learning — Side by Side
| Property | Classical ML | Deep Learning |
|---|---|---|
| Feature extraction | manual (expert) | automatic (learned) |
| Data needed | hundreds of rows | thousands–millions |
| Compute | CPU / laptop | GPU/TPU · hours–weeks |
| Interpretability | often explainable | black box |
| Best data | tabular / structured | images, text, audio, video |
| Tabular performance | excellent (XGBoost often wins) | competitive, rarely better |
| Unstructured performance | poor | state-of-the-art |
| Transfer learning | no | yes — reuse pre-trained models |
Start with classical ML for tabular data — faster, more interpretable, often just as accurate. Move to deep learning for images, audio and text, where hand-crafting features is too costly or simply impossible.
Where Each One Wins
The deciding question is rarely "which is better?" — it's "is my signal already in tidy columns, or buried in raw pixels, waveforms and words?" That answers it almost every time.
Layers Learn a Hierarchy of Meaning
Edges and textures exist in all natural images, so a net trained on ImageNet can be fine-tuned on a 500-image medical set: freeze the early layers, retrain only the final classifier. Classical models can't share internal representations like this — the heart of transfer learning.
MNIST — Random Forest vs CNN
# Classical ML — flatten pixels, no structure
X = X_train.reshape(-1, 784) / 255.
RandomForestClassifier(n_estimators=100).fit(X, y)
# → 0.9705
# Deep learning — Conv layers find the features
models.Sequential([
layers.Conv2D(32, (3,3), activation='relu'),
layers.MaxPooling2D(),
layers.Conv2D(64, (3,3), activation='relu'),
layers.MaxPooling2D(),
layers.Flatten(),
layers.Dense(10, activation='softmax')])
# → 0.9921The forest treats every pixel independently — blind to spatial structure. The CNN learns that edges form curves and curves form digits: a hierarchy no classical model can discover on its own.
When to Use Which — Six Signals
The Deep-Learning Architecture Map
CNNs see space, RNNs/LSTMs remember sequence, Transformers attend to everything at once. Nearly all of today's frontier models are Transformers — but each shape still fits its data best.
The Boundary Is Moving
For years XGBoost owned structured data. Now attention-based models like TabTransformer and FT-Transformer are closing the gap — and transfer learning across tables is an active research front. The "use trees for tables" rule still holds for most projects today, but the frontier is genuinely shifting.
Don't pick a camp — benchmark. On your own data, a tuned gradient-boosted tree and a modern tabular Transformer are both a few lines away. Let the validation score decide.
The One-Sentence Summary
You hand the algorithm facts → it learns a decision. The features are your job.
You hand the algorithm raw data → it learns what facts to extract, then the decision. The features are its job.
Deep learning adds one thing: it learns the representation itself. That makes it unbeatable on unstructured data — and expensive, data-hungry and opaque on everything else. Choose by the shape of your data, not the hype.
🧠 Next up: The Biological Neuron & the McCulloch-Pitts Model · Press ← to review