Collecting Signals Responsibly
Rules for gathering signal data, keeping only fresh signals, and logging where every record came from.
What you will be able to do
Collect signals within the rules, filter out stale ones with a short program, and record the source of every signal.
Where the data came from is a legal question
In the Temerarii machine, the human gate on the signal layer is list sources. Before a single row enters the machine, a person approves where it came from. That is a legal question before it is a marketing one.
Ask four things about any source:
- Is it public business information? A district's grant award is. A teacher's home address is not.
- Do the site's terms allow it? Many sites forbid automated collection in their terms of service. Respect that.
- Does robots.txt allow it? Most sites publish a file at
/robots.txtthat says which pages automated tools may visit. Honor it. - Is anyone under 18 involved? If yes, stop. Do not collect data about minors for outreach.
Fresh signals only
A signal has a shelf life. A job posting from last week is a reason to reach out. A job posting from last year is not. Old signals produce messages that feel wrong, because the moment has passed.
Set a freshness window: the number of days a signal stays useful. Then filter. The code below keeps only signals from the last 30 days. The signal data is made up for practice.
If you are on phone data, note that the first run downloads the Python runtime (several MB). You can read the code here first:
from datetime import date
today = date(2026, 3, 31)
window_days = 30
signals = [
{"org": "Lakeview School District", "event": "after-school grant award", "seen": date(2026, 3, 20), "source": "state agency news page"},
{"org": "Northside Library", "event": "teen services job posting", "seen": date(2026, 3, 25), "source": "library careers page"},
{"org": "Hillcrest Youth Center", "event": "new location opened", "seen": date(2025, 11, 2), "source": "local news"},
{"org": "Eastgate Schools", "event": "tutoring RFP posted", "seen": date(2026, 2, 1), "source": "state procurement site"},
]
fresh = []
for s in signals:
age = (today - s["seen"]).days
if age <= window_days:
fresh.append(s)
print("KEEP", s["org"], "-", s["event"], "(" + str(age) + " days old, from " + s["source"] + ")")
else:
print("DROP", s["org"], "-", str(age), "days old")
print("Fresh signals:", len(fresh), "of", len(signals))
Log the source on every row
Every signal row should carry three facts: where it came from, when you saw it, and the link to the public page. This is called provenance, the record of where data came from. If anyone ever asks "why did you contact us?", you can answer in one sentence and show the page.
When not to collect
- When the only way to get the data is to break a site's rules.
- When the data is about private people rather than organizations.
- When you would be uncomfortable showing the organization exactly how you found them.
That last test is the simplest. If you could not say it out loud to them, do not do it.
Signal source gate
0 of 6 doneQuiz: Check your understanding
Live session
Go deeper: Consent as signal: the first-party data layer you own
Discussion
Ask. Answer. Build together.
Loading…