lesson
45 minGrouping and Summaries
Split a table into groups, summarize each group, and compare them fairly.
Objective
By the end of this lesson, you can group rows by a column, compute a summary for each group, and compare groups using rates instead of raw counts.
Split, apply, combine
Grouping means three steps:
- Split the rows into groups (for example, by state).
- Apply a summary to each group (sum, mean, count).
- Combine the answers into a new, smaller table.
Here is the idea in plain Python with a dictionary:
Run it. Each row is a made-up county with its state, population, and number of households without internet. The code totals each state. Then compute the rate (no_internet divided by households) for each state.
Python 3
Ctrl+Enter to run | Tab to indent | Python runs in your browser via Pyodide
Counts versus rates
State BB has more households without internet (11,000 versus 4,500). But BB is also much bigger. The rate tells a different story: AA is 4,500 of 25,000 (18 percent), and BB is 11,000 of 100,000 (11 percent).
Raw counts mostly measure size. To compare places of different sizes, use rates or per-person numbers.
The same thing in pandas
Run this in Google Colab:
import pandas as pd
df = pd.DataFrame({
"state": ["AA", "AA", "BB", "BB"],
"households": [20000, 5000, 90000, 10000],
"no_internet": [3000, 1500, 9000, 2000],
})
by_state = df.groupby("state")[["households", "no_internet"]].sum()
by_state["rate"] = by_state["no_internet"] / by_state["households"]
print(by_state.sort_values("rate", ascending=False))
A national example
Quiz: Grouping check
Discussion
Ask. Answer. Build together.
Loading…