lesson
55 min

LLM-as-Judge, With Caution

Use a model to grade answers at scale only after measuring it against trained people, and know the biases to watch for.

Objective

By the end of this lesson, you can write a judge rubric, measure an LLM judge against human labels, and decide where a judge is and is not trustworthy.

Why teams use model judges

Some answers cannot be checked with simple code. "Is this reply kind, accurate, and at a 6th-grade reading level?" needs judgment. People are the best graders, but grading thousands of answers by hand is slow.

LLM-as-judge means using a language model to grade another model's output against a rubric. It scales. It also has known weaknesses.

Known judge biases

Research and practice have documented several:

  • Leniency: judges tend to pass answers that sound confident and fluent, even when wrong.
  • Position bias: when comparing two answers, judges can favor the one shown first.
  • Length bias: longer answers often get higher scores.
  • Self-preference: a model may favor answers written in its own style.
  • Blind spots: a judge cannot catch errors it does not know are errors, such as an out-of-date local rule.

Calibrate before you trust

Never use a judge until you have compared it with people:

  1. A trained person grades 50 to 100 answers using the written rubric.
  2. The judge grades the same answers.
  3. Measure overall agreement, and agreement on failures: of the answers people failed, how many did the judge fail?
  4. Read every disagreement. Fix the rubric or the judge prompt.
  5. Re-check on fresh samples regularly.
Run it. Overall agreement is 0.8, which sounds good. But the judge caught only half the real failures. Change judge so it fails answer 2 as well. How do both numbers move? Which number would you put on your launch dashboard?
Python 3
Ctrl+Enter to run | Tab to indent | Python runs in your browser via Pyodide

The first run downloads Python into your browser (several MB).

Write a judge rubric

Good judge prompts are narrow. Grade one thing at a time, ask for a reason before the verdict, and give examples of pass and fail.

Judge rubric builder

Write a judge prompt that checks one thing: whether an answer is fully supported by the given source passages.

Who should the AI act as?

What exactly should it do?

What does it need to know?

What shape should the answer take?

How should it sound?

Your prompt

Act as a strict grader who checks answers against source passages.

Do not paste private details (full name, address, ID numbers) into any AI tool.

Where judges do not belong

  • As the only check on harmful cases. People grade those.
  • On questions where the judge lacks the facts, such as local rules not in the passages.
  • As a launch gate without calibration.
  • Grading its own model's outputs without human spot checks.

Quiz: Judge check

  1. 01Your judge shows 90% overall agreement with people. What should you check next?

    Choose one.

  2. 02Which judge prompt design is better?

    Choose one.

Go deeper: Machine Learning, Minus the Hype: A Practical Playbook to Ship Useful Models

Ask. Answer. Build together.