lesson
60 min

Test Sets First

Define what good looks like before you build: a test set with common, edge, harmful, and out-of-scope cases, run on every change.

Objective

By the end of this lesson, you can design an evaluation test set, write checks for each case type, and use it as a regression suite that every change must pass.

Evaluation is the definition of done

An evaluation (often shortened to "eval") is a repeatable test of how well a system does its job. Without one, every change is a guess. With one, you can say "version 8 answers 94 of 100 test questions correctly, up from 89, with no new failures in Spanish."

Build the test set before the feature. Writing the expected answers forces the team to agree on what "good" means.

What goes in a test set

Cover four kinds of cases:

Case typeExample for a pantry assistantWhat passing looks like
Common"When are you open Saturday?"Correct hours, cited
Edge"I lost my ID and my kids are with me"Correct ID rules for adults and children
Harmful or sensitive"Did my neighbor get food stamps?"Refuses to share others' information
Out of scope"Is there parking?" (not in documents)Says it does not know, offers a person

Also cover each language, reading level, and typing style your users bring: short texts, misspellings, mixed languages.

Grow the set from real life. Every production failure becomes a new test case, so the same failure cannot ship twice.

Kinds of checks

  • Exact or contains: the answer includes "9-5" and a source tag.
  • Refusal: the answer declines when it should.
  • Schema: structured output validates (Module 3).
  • Human review: a trained person grades a sample, using a written rubric.
  • Model-graded: another model grades answers (next lesson, with caution).

Use the cheapest check that is reliable. Code checks are fast and consistent. Human review is the ground truth.

Run it. One test fails: the system shared private information. Fix the canned answer for that question so it refuses, and run again. Then add a fifth test of your own, an edge case, with its expected content.
Python 3
Ctrl+Enter to run | Tab to indent | Python runs in your browser via Pyodide

The first run downloads Python into your browser (several MB).

Thresholds and regression

Set two kinds of thresholds before launch:

  • Launch bar: for example, at least 90% of common cases correct, and 100% of harmful cases refused.
  • Never-regress list: a set of critical cases that must always pass. One failure blocks the release.

Run the full set on every prompt change, model change, and retrieval change. Keep results over time so you can see drift.

A national example

Quiz: Test set check

  1. 01Which case is most important to include and require 100% on?

    Choose one.

  2. 02A production failure happens. What should happen to your test set?

    Choose one.

Go deeper: Machine Learning, Minus the Hype: A Practical Playbook to Ship Useful Models

Ask. Answer. Build together.