lesson
35 min

Collecting Signals Responsibly

Rules for gathering signal data, keeping only fresh signals, and logging where every record came from.

What you will be able to do

Collect signals within the rules, filter out stale ones with a short program, and record the source of every signal.

Where the data came from is a legal question

In the Temerarii machine, the human gate on the signal layer is list sources. Before a single row enters the machine, a person approves where it came from. That is a legal question before it is a marketing one.

Ask four things about any source:

  1. Is it public business information? A district's grant award is. A teacher's home address is not.
  2. Do the site's terms allow it? Many sites forbid automated collection in their terms of service. Respect that.
  3. Does robots.txt allow it? Most sites publish a file at /robots.txt that says which pages automated tools may visit. Honor it.
  4. Is anyone under 18 involved? If yes, stop. Do not collect data about minors for outreach.

Fresh signals only

A signal has a shelf life. A job posting from last week is a reason to reach out. A job posting from last year is not. Old signals produce messages that feel wrong, because the moment has passed.

Set a freshness window: the number of days a signal stays useful. Then filter. The code below keeps only signals from the last 30 days. The signal data is made up for practice.

If you are on phone data, note that the first run downloads the Python runtime (several MB). You can read the code here first:

from datetime import date

today = date(2026, 3, 31)
window_days = 30

signals = [
    {"org": "Lakeview School District", "event": "after-school grant award", "seen": date(2026, 3, 20), "source": "state agency news page"},
    {"org": "Northside Library", "event": "teen services job posting", "seen": date(2026, 3, 25), "source": "library careers page"},
    {"org": "Hillcrest Youth Center", "event": "new location opened", "seen": date(2025, 11, 2), "source": "local news"},
    {"org": "Eastgate Schools", "event": "tutoring RFP posted", "seen": date(2026, 2, 1), "source": "state procurement site"},
]

fresh = []
for s in signals:
    age = (today - s["seen"]).days
    if age <= window_days:
        fresh.append(s)
        print("KEEP", s["org"], "-", s["event"], "(" + str(age) + " days old, from " + s["source"] + ")")
    else:
        print("DROP", s["org"], "-", str(age), "days old")

print("Fresh signals:", len(fresh), "of", len(signals))
Run the code to keep only signals from the last 30 days. Then change window_days to 7 and run it again.
Python 3
Ctrl+Enter to run | Tab to indent | Python runs in your browser via Pyodide

Log the source on every row

Every signal row should carry three facts: where it came from, when you saw it, and the link to the public page. This is called provenance, the record of where data came from. If anyone ever asks "why did you contact us?", you can answer in one sentence and show the page.

When not to collect

  • When the only way to get the data is to break a site's rules.
  • When the data is about private people rather than organizations.
  • When you would be uncomfortable showing the organization exactly how you found them.

That last test is the simplest. If you could not say it out loud to them, do not do it.

Signal source gate

0 of 6 done

Quiz: Check your understanding

  1. 01What does a freshness window do?

    Choose one.

  2. 02Which signal should you never collect?

    Choose one.

Live session

Go deeper: Consent as signal: the first-party data layer you own

Ask. Answer. Build together.