Covariance Explained
Press Next → or use ← → arrow keys
The Intuition — Do They Move Together?
Covariance measures how much two variables change together relative to their own means. It takes each point's distance above or below its mean, multiplies the pair, and averages — producing a single number whose sign reveals the direction of the relationship.
Covariance is the raw material for correlation, the covariance matrix, and PCA. Nail this one idea — joint variability around the means — and a whole layer of multivariate statistics opens up.
What The Sign Tells You
Covariance's sign is trustworthy, but its magnitude isn't directly meaningful — a covariance of 500 isn't "bigger" than one of 5 in any comparable way, because it depends entirely on the units. Read the sign for direction; wait for correlation to judge strength.
Why The Sign Works — Four Quadrants
Split the cloud by the two mean lines. A point above both means, or below both, gives a positive product (+·+ or −·−). A point above one and below the other gives a negative product. Covariance sums these votes: if the green quadrants dominate, Cov > 0; if red dominates, Cov < 0; if they cancel, Cov ≈ 0.
Population vs Sample Covariance
Variance multiplies a variable's deviation by itself: (x−x̄)². Covariance
multiplies X's deviation by Y's: (x−x̄)(y−ȳ). In fact Cov(X, X) = Var(X)
— variance is just covariance of a variable with itself. Same (n−1) logic, same Bessel correction.
Five Students, Step By Step
Study hours X = [2, 4, 6, 8, 10] (mean 6) and scores Y = [50, 60, 70, 85, 95] (mean 72). Multiply each pair of deviations:
| Student | x − x̄ | y − ȳ | product |
|---|---|---|---|
| A (2, 50) | −4 | −22 | +88 |
| B (4, 60) | −2 | −12 | +24 |
| C (6, 70) | 0 | −2 | 0 |
| D (8, 85) | +2 | +13 | +26 |
| E (10, 95) | +4 | +23 | +92 |
Every product is positive (or zero), so the sum is a solid +230; divide by n−1 = 4 and Cov = 57.5. The positive sign confirms study hours and scores move together. But "57.5 hour·marks" is an awkward unit — which is exactly the problem correlation will fix.
The Units Problem — Magnitude Is Slippery
Switch study time from hours to minutes and the covariance jumps from 57.5 to 3450 — even though the relationship is identical. Covariance has no fixed range and rides on the units of X and Y, so you can never compare raw covariances across datasets. That single flaw is why correlation exists.
Standardize It → Correlation
Divide the covariance by the product of the two standard deviations — r = Cov(X,Y) / (σₓ·σᵧ)
— and the units cancel out. The result is the correlation coefficient: the same directional information,
now squeezed into a comparable −1 to +1 scale. Covariance is the engine; correlation is
the readable dial.
Covariance vs Correlation
| Property | Covariance | Correlation (r) |
|---|---|---|
| Range | −∞ to +∞ | −1 to +1 |
| Depends on units? | Yes | No (unitless) |
| Magnitude interpretable? | Hard | Easy |
| Tells direction? | Yes | Yes |
| Symmetric? | Yes — Cov(X,Y)=Cov(Y,X) | Yes |
| Best for | Matrices, PCA, theory | Reporting & comparing strength |
Both answer "which direction do they move together?" and both are symmetric. Use covariance inside the maths — covariance matrices, PCA's eigen-decomposition, portfolio variance. Use correlation whenever a human needs to judge how strong the link is.
The Covariance Matrix
Stack every pairwise covariance into a grid and you get the covariance matrix: each variable's variance sits on the diagonal, and the covariances fill the off-diagonal — always symmetric (top-right = bottom-left). This matrix is exactly what PCA eigen-decomposes to find the directions of greatest variance.
Why Covariance Matters
Because covariance underlies correlation, it inherits the same caveats: a non-zero covariance shows variables move together, not that one causes the other, and a zero covariance only rules out a linear link — a curved relationship can still hide beneath it.
Covariance In Python
import numpy as np X = [2, 4, 6, 8, 10] Y = [50, 60, 70, 85, 95] np.cov(X, Y) # 2×2 matrix; np.cov defaults to ddof=1 (sample) ✓ # [[ 10. 57.5 ] # [ 57.5 332.5 ]] → Cov(X,Y) = 57.5 # ── pandas ── df.cov() # full covariance matrix df['hours'].cov(df['score']) # 57.5 — one pair
Unlike np.var() and np.std() (which default to the population
divisor), np.cov() already uses ddof=1 — the sample formula. So the matrix
it returns is the sample covariance out of the box. The diagonal holds each variable's variance; the
off-diagonal is the covariance you want.
Six Rules For Covariance
You Now Understand Covariance
Covariance multiplies how far two variables stray from their means and averages it — the sign reveals whether they move together or apart. Its magnitude is unit-bound and unbounded, so we standardize it into correlation for comparison. As a matrix, it's the raw material of PCA and portfolio risk.
You've now seen the pair: covariance and its standardized twin, correlation. Next, watch the covariance matrix come alive inside PCA, and step up to regression, which models the exact line two variables trace out.
↗️ End of tutorial · Press ← to review, or click Restart