lesson
55 min

Structured Outputs

Make model output machine-checkable with schemas, validate every response, and decide what happens when validation fails.

Objective

By the end of this lesson, you can define a schema for model output, validate responses against it, and design the retry and fallback path for responses that fail.

Why free text breaks systems

When a person reads model output, they can skip a stray sentence. When code reads it, a stray sentence breaks the program. If your app routes intake requests, fills a form, or updates a dashboard, it needs output in a fixed shape.

Structured output means asking the model to answer in a defined format, usually JSON (a text format of keys and values). A schema is the rulebook for that format: which keys must appear, their types, and which values are allowed.

Three layers of control

  1. Ask clearly. Describe the schema in the prompt and give one example.
  2. Constrain. Many providers offer a JSON mode or schema-constrained output that keeps the model's text inside a given structure. Use it when available.
  3. Validate anyway. Even constrained output can hold a wrong value. Your code checks every response before it touches anything else.

Validation catches format errors. It does not catch a well-formed lie. A response can match the schema perfectly and still say "urgency: not_urgent" for someone facing eviction tomorrow. That is why high-stakes fields get human review.

Run it. Four model replies come in; only one passes. Then add a fifth reply that has every key but sets service to an empty string. Does it pass? Add a check that rejects empty values.
Python 3
Ctrl+Enter to run | Tab to indent | Python runs in your browser via Pyodide

The first run downloads Python into your browser (several MB).

When validation fails

Decide this before launch, not during an incident:

FailureResponse
Not valid JSONRetry once, including the error message in the request
Missing or extra keysRetry once with the schema repeated
Value outside allowed listDo not guess. Retry, then send to a person
Fails twiceRoute to a person with the original message attached
High-stakes field (urgency, safety)Always show a person before acting

Log every validation failure with the prompt version. A rising failure rate is an early warning that a model or prompt change went wrong.

A national example

Quiz: Structured output check

  1. 01A response matches the schema perfectly. What do you know for sure?

    Choose one.

  2. 02The urgency field comes back as "ASAP," which is not an allowed value. Best action?

    Choose one.

Go deeper: Software Development, Simply: How to Build the Right Thing, Faster, With Fewer Headaches

Ask. Answer. Build together.