lesson
40 min

Judging New AI Tools

New AI tools arrive every week. Use a repeatable test to decide which ones deserve your time and your community's trust.

Objective

By the end of this lesson, you can evaluate any new AI tool with a written checklist and a small test, and write a short adopt, pilot, or skip recommendation.

Tools change. The test does not.

By the time you finish this course, new models and tools will exist that are not named here. That is fine. You do not need to know every tool. You need a way to judge them.

The evaluation steps

  1. Job. What real task would this do? If you cannot name one, skip it.
  2. Evidence. Is there a public evaluation, a technical report, or an independent test? Marketing pages are not evidence.
  3. Your test. Run it on 10 to 20 real examples where you know the right answer. Score it against your current method.
  4. Data. Where does your input go? Is it stored or used for training? Can you turn that off?
  5. Failure. What happens when it is wrong? Who catches it?
  6. Access. Does it work on a phone, with slow internet, with a screen reader, in other languages?
  7. Exit. Can you get your work out if the tool changes or disappears?

A national example

Practice

Prompt Lab

Turn the seven evaluation steps into a test plan for a new AI tool.

Who should the AI act as?

What exactly should it do?

What does it need to know?

What shape should the answer take?

Your prompt

Act as an evaluator for a community organization that serves families and youth.

Your task: Write a test plan for this AI tool using the seven steps: job, evidence, my test, data, failure, access, exit

Format: A table with one row per step and a column for what I will check, then a one-line adopt, pilot, or skip rule

Do not paste private details (full name, address, ID numbers) into any AI tool.

Before you adopt a new AI tool

0 of 7 done

Ask. Answer. Build together.