lesson
45 min

Scaling and PCA

Put features on the same scale so no column bullies the others, and shrink many columns into a few with PCA.

Objective

By the end of this lesson, you can scale features, explain why it matters for clustering, and describe what PCA does in plain words.

Why scaling matters

K-means measures distance. Suppose one column is income (tens of thousands) and another is days over 90 degrees (tens). A difference of 1,000 dollars would count far more than a difference of 30 hot days. The income column would decide everything.

Scaling puts columns on a similar range so each one gets a fair vote.

Min-max scaling maps each column to the range 0 to 1:

new value = (value minus the minimum) divided by (maximum minus minimum)

Run it. Two made-up columns on very different scales become comparable. Add a fifth county with income 120000 and see how every scaled income value shifts.
Python 3
Ctrl+Enter to run | Tab to indent | Python runs in your browser via Pyodide

Another common choice is standard scaling: subtract the mean and divide by the standard deviation. scikit-learn's StandardScaler does this.

PCA: fewer columns, most of the story

Real datasets can have dozens of columns. PCA (principal component analysis) builds a few new columns, called components, that capture as much of the variation as possible. It helps you:

  • Plot many-column data in two dimensions.
  • Speed up models.
  • Spot rows that do not fit the pattern.

The cost: components are blends of the original columns, so they are harder to explain.

from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.cluster import KMeans

X = StandardScaler().fit_transform(load_iris().data)
X2 = PCA(n_components=2).fit_transform(X)
groups = KMeans(n_clusters=3, n_init=10, random_state=0).fit_predict(X2)

import matplotlib.pyplot as plt
plt.scatter(X2[:, 0], X2[:, 1], c=groups)
plt.title("Iris flowers in two PCA components, colored by cluster")
plt.show()

A national example

Judgment check

Which columns you include decides what "similar" means. Include race or zip code, and clusters may simply sort people by race or neighborhood. Choose columns on purpose, and write down why each one is there.

Quiz: Scaling check

  1. 01Why scale features before k-means?

    Choose one.

  2. 02What is a downside of PCA?

    Choose one.

Ask. Answer. Build together.