lesson
55 min

Images and Convolution

How networks see: images as grids of numbers, filters that detect edges, and a digit classifier trained on public data.

Objective

By the end of this lesson, you can explain how a computer stores an image, what a convolution filter detects, and train a small image classifier.

Images are grids of numbers

A grayscale image is a grid of numbers. Each number is one pixel's brightness, often 0 (black) to 255 (white). A color image has three grids: red, green, and blue.

Convolution: sliding a filter

A convolution slides a small grid of weights, called a filter, across the image. At each spot, it multiplies and adds. The result is large where the image matches the filter's pattern.

Run it. The image is a bright square on a dark background. The filter detects left-right edges. Big positive and negative numbers mark the square's edges. Zeros mean no edge.
Python 3
Ctrl+Enter to run | Tab to indent | Python runs in your browser via Pyodide

A convolutional neural network (CNN) learns its own filters. Early layers learn edges. Later layers combine edges into shapes, and shapes into objects.

Train a digit classifier

The MNIST dataset is 70,000 small images of handwritten digits, built into Keras. Run this in Google Colab. Choose Runtime, then Change runtime type, then GPU for faster training. It still works without one, just slower.

from tensorflow import keras

(X_train, y_train), (X_test, y_test) = keras.datasets.mnist.load_data()
X_train, X_test = X_train[..., None] / 255.0, X_test[..., None] / 255.0  # scale to 0-1

model = keras.Sequential([
    keras.Input(shape=(28, 28, 1)),
    keras.layers.Conv2D(16, 3, activation="relu"),
    keras.layers.MaxPooling2D(),
    keras.layers.Conv2D(32, 3, activation="relu"),
    keras.layers.MaxPooling2D(),
    keras.layers.Flatten(),
    keras.layers.Dense(10, activation="softmax"),   # one output per digit
])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy", metrics=["accuracy"])
model.fit(X_train, y_train, epochs=3, validation_split=0.1)
print("Test accuracy:", model.evaluate(X_test, y_test, verbose=0)[1])

After it trains, find the test images it got wrong and look at them. Some are hard even for people. That is useful evidence about the limits of the task.

A national example

Judgment check

Image models used on people, like face recognition, carry serious risks of misidentification and surveillance (recall the NIST finding in Module 11). Practice on digits, plants, or objects. Do not build models that identify real people from photos, and never use photos of minors.

Image project check

0 of 5 done

Ask. Answer. Build together.