Classifiers: Sorting Into Categories
What classification is, how logistic regression and decision trees work, and how to train them.
Objective
By the end of this lesson, you can explain what a classifier does, train two kinds of classifiers, and describe how a score becomes a label.
What classification is
Classification is supervised learning that predicts a category. Spam or not spam. Will this pothole report be urgent or routine? Which of three flower types is this?
Most classifiers first produce a score (often a probability from 0 to 1). Then a threshold turns the score into a label. "If the spam score is 0.5 or higher, call it spam." Choosing that threshold is a judgment call, and it is covered in the next lesson.
A scoring rule you can see
Before any machine learning, here is a hand-made scorer. It counts warning words and uses a threshold of 2.
With a threshold of 1, the real field trip message becomes spam. A parent might miss it. Lower thresholds catch more spam and block more real mail.
Two classic classifiers
Logistic regression. Despite its name, it classifies. It adds up weighted inputs, like linear regression, then squeezes the total into a probability between 0 and 1. It is fast and easy to explain: each input has one weight.
Decision tree. A flowchart of yes/no questions learned from data. "Is the word count over 50? Does it contain a link?" Trees are easy to read when small and easy to overfit when large.
Train them in Colab
This uses the Breast Cancer Wisconsin (Diagnostic) dataset from the UC Irvine Machine Learning Repository (CC BY 4.0), built into scikit-learn. It is a teaching dataset of measurements from cell images.
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0, stratify=y)
logreg = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X_train, y_train)
tree = DecisionTreeClassifier(max_depth=3, random_state=0).fit(X_train, y_train)
print("Logistic regression accuracy:", logreg.score(X_test, y_test))
print("Decision tree accuracy:", tree.score(X_test, y_test))
stratify=y keeps the same mix of categories in the train and test sets.
A national example
Quiz: Classifier check
Discussion
Ask. Answer. Build together.
Loading…