lesson
40 minJudging New AI Tools
New AI tools arrive every week. Use a repeatable test to decide which ones deserve your time and your community's trust.
Objective
By the end of this lesson, you can evaluate any new AI tool with a written checklist and a small test, and write a short adopt, pilot, or skip recommendation.
Tools change. The test does not.
By the time you finish this course, new models and tools will exist that are not named here. That is fine. You do not need to know every tool. You need a way to judge them.
The evaluation steps
- Job. What real task would this do? If you cannot name one, skip it.
- Evidence. Is there a public evaluation, a technical report, or an independent test? Marketing pages are not evidence.
- Your test. Run it on 10 to 20 real examples where you know the right answer. Score it against your current method.
- Data. Where does your input go? Is it stored or used for training? Can you turn that off?
- Failure. What happens when it is wrong? Who catches it?
- Access. Does it work on a phone, with slow internet, with a screen reader, in other languages?
- Exit. Can you get your work out if the tool changes or disappears?
A national example
Practice
Prompt Lab
Turn the seven evaluation steps into a test plan for a new AI tool.
Your prompt
Act as an evaluator for a community organization that serves families and youth. Your task: Write a test plan for this AI tool using the seven steps: job, evidence, my test, data, failure, access, exit Format: A table with one row per step and a column for what I will check, then a one-line adopt, pilot, or skip rule
Before you adopt a new AI tool
0 of 7 doneDiscussion
Ask. Answer. Build together.
Loading…