lesson
25 min

When Data Misleads

Spot bad data, small numbers, and missing people, and protect individuals when you report.

What you will learn

By the end of this lesson, you can spot four ways data misleads and protect individuals' privacy in a report.

Four ways data misleads

  1. Bad data. Typos, duplicates, and blanks. One student entered twice makes attendance look higher than it is. Clean the data before you trust it.
  2. Small numbers. If attendance goes from 4 to 6, that is "up 50 percent." It sounds big, but it is two people. Show the counts, not just percentages.
  3. Missing people. Data only counts who shows up in the records. If a sign-up form is only online and only in English, the data will not show the families who could not use it. Ask: who is not in this data?
  4. Correlation is not cause. Two things rising together does not mean one causes the other. Students who attend more might do better because of tutoring, or because they already had more support at home. Be careful what you claim.

Protect individuals in reports

A report about a group can still expose a person. If a report shows "one 9th grader at the Thursday program has a disability," people may figure out who it is.

Habits that protect people:

  • Report groups, not individuals. Share totals and averages.
  • Hide tiny groups. If a group is very small, combine it with another or do not show it. Many agencies follow a rule like this, often called small-cell suppression.
  • Remove names and IDs from data you share or analyze.
  • Limit access to raw records to the people who need them.

Take extra care with data about minors. A dashboard for funders should never let anyone identify a child.

When not to use the number

Do not let a number make a decision alone when:

  • The data is known to be incomplete or messy.
  • The number is too small to be meaningful.
  • The decision affects a specific person, like whether a student stays in a program. Look at the person, not just the record.
  • Collecting the data would put people at risk.

Reflection

Private. Saved only in this browser. No one else can see it.

Go deeper: The Power of Web & Business Analytics: Turning Data Into Decisions

Quiz: Module 8 Quiz: Business Intelligence

  1. 01Where should a BI project start?

    Choose one.

  2. 02Why write a definition for each measure?

    Choose one.

  3. 03Attendance went from 4 to 6, reported as "up 50 percent." What is the better report?

    Choose one.

  4. 04What did the Johns Hopkins dashboard team have to explain?

    Choose one.

  5. 05A funder report shows a group with only one student in it. What should you do?

    Choose one.

Ask. Answer. Build together.