Tables and DataFrames
Read a CSV file, inspect it with pandas, and learn the five questions to ask of any new dataset.
Objective
By the end of this lesson, you can load a CSV file, look at its rows, columns, and types, and answer five first questions about it.
CSV files
A CSV (comma-separated values) file is a plain-text table. Each line is a row. Commas separate the columns. Almost every public data site offers CSV downloads.
Python can read CSV text with its built-in csv module. This runs in your browser:
pandas
pandas is a free Python library for tables. Its table is called a DataFrame: rows and named columns, like a spreadsheet you control with code. pandas reads types for you and has hundreds of tested functions.
pandas does not run in the box above. Use a free online notebook like Google Colab for this part.
Run it yourself
- Open Google Colab and choose New notebook. pandas is already installed there.
- Download a CSV from data.gov. A good first pick is any state- or county-level table from the U.S. Department of Agriculture or the Census Bureau. Upload it with the folder icon on the left.
- Paste this into a cell, change the file name, and press Shift+Enter.
import pandas as pd
df = pd.read_csv("your_file.csv")
print(df.shape) # (rows, columns)
print(df.head()) # first 5 rows
print(df.dtypes) # type of each column
print(df.isna().sum()) # missing values per column
print(df.describe()) # count, mean, min, max for number columns
Five questions for any new dataset
- Who made it, and why? A survey, a sensor, a government form?
- What is one row? A person, a county, a day, a sale?
- What time period does it cover?
- What is missing? Blank cells, but also missing groups of people or places.
- What is the license? Can you use and share it?
A national example
Reflection
Discussion
Ask. Answer. Build together.
Loading…