Layers and Activations
Stack neurons into layers, add activation functions, and build your first neural network in Keras.
Objective
By the end of this lesson, you can describe the layers of a neural network, explain what an activation function does, and build a small network in Keras.
Layers
A neural network connects neurons in layers:
- Input layer: one value per feature (column).
- Hidden layers: neurons that combine the inputs in new ways. "Hidden" just means they are in the middle.
- Output layer: the prediction. One neuron for a number or a yes/no score, or one per category.
Each hidden layer can learn more complex patterns built from the layer before it. A network with many hidden layers is called deep. That is where "deep learning" comes from.
Activation functions
Between layers, each neuron passes its total through an activation function. Without one, stacking layers would add up to just one big straight-line model. Activations let the network learn curves.
Two common ones:
- ReLU: if the total is negative, output 0. Otherwise, output the total. Simple and fast. Used in most hidden layers.
- Sigmoid: squeezes any total into the range 0 to 1. Used for yes/no outputs, because the result reads like a probability.
Your first network in Keras
Keras is a beginner-friendly library for neural networks that runs on TensorFlow. It is preinstalled in Google Colab. This network solves XOR, which a single neuron could not.
import numpy as np
from tensorflow import keras
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
y = np.array([0, 1, 1, 0]) # XOR
model = keras.Sequential([
keras.Input(shape=(2,)),
keras.layers.Dense(8, activation="relu"), # hidden layer
keras.layers.Dense(1, activation="sigmoid"), # output: a score from 0 to 1
])
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.fit(X, y, epochs=500, verbose=0)
print(model.predict(X).round(2))
Results vary a little between runs, because the starting weights are random. If it has not learned XOR, run it again or raise the epochs.
Judgment check
Neural networks are powerful and hard to explain. For a table of a few dozen columns, a regression or a small tree often does as well, and you can explain it to the person affected. Use a neural network when the data is images, sound, or text, or when simpler models clearly fall short on your test set.
Quiz: Layers check
Discussion
Ask. Answer. Build together.
Loading…