Scaling and PCA
Put features on the same scale so no column bullies the others, and shrink many columns into a few with PCA.
Objective
By the end of this lesson, you can scale features, explain why it matters for clustering, and describe what PCA does in plain words.
Why scaling matters
K-means measures distance. Suppose one column is income (tens of thousands) and another is days over 90 degrees (tens). A difference of 1,000 dollars would count far more than a difference of 30 hot days. The income column would decide everything.
Scaling puts columns on a similar range so each one gets a fair vote.
Min-max scaling maps each column to the range 0 to 1:
new value = (value minus the minimum) divided by (maximum minus minimum)
Another common choice is standard scaling: subtract the mean and divide by the standard deviation. scikit-learn's StandardScaler does this.
PCA: fewer columns, most of the story
Real datasets can have dozens of columns. PCA (principal component analysis) builds a few new columns, called components, that capture as much of the variation as possible. It helps you:
- Plot many-column data in two dimensions.
- Speed up models.
- Spot rows that do not fit the pattern.
The cost: components are blends of the original columns, so they are harder to explain.
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.cluster import KMeans
X = StandardScaler().fit_transform(load_iris().data)
X2 = PCA(n_components=2).fit_transform(X)
groups = KMeans(n_clusters=3, n_init=10, random_state=0).fit_predict(X2)
import matplotlib.pyplot as plt
plt.scatter(X2[:, 0], X2[:, 1], c=groups)
plt.title("Iris flowers in two PCA components, colored by cluster")
plt.show()
A national example
Judgment check
Which columns you include decides what "similar" means. Include race or zip code, and clusters may simply sort people by race or neighborhood. Choose columns on purpose, and write down why each one is there.
Quiz: Scaling check
Discussion
Ask. Answer. Build together.
Loading…