Exfiltration, Secrets, and Least Privilege
Close the paths data can leak through, keep secrets away from models, and scope every tool and agent to the least access it needs.
Objective
By the end of this lesson, you can identify data exfiltration paths in an AI app, handle secrets safely, and design permission scopes using least privilege.
How data leaks out
Exfiltration means data leaving your system to somewhere it should not go. In AI apps, a hijacked or careless model can leak data through:
- Rendered links and images. If the app displays Markdown images, an injected instruction can make the model output an image whose web address contains private data. The browser loads it and sends the data to an outside server, with no click needed.
- Tool calls. A web request tool, email tool, or file upload tool can carry data out.
- Answers to the wrong person. Retrieval without access filters can show one client's notes to another user.
- Logs and vendors. Raw prompts in third-party logging or analytics tools.
- Prompt extraction. Users coaxing the model to reveal its system prompt and anything in it.
Defenses:
- Do not render images or links from untrusted domains. Use an allowlist.
- Restrict outbound network tools to approved addresses.
- Filter retrieval by the user's permissions before search results reach the model.
- Redact before logging (Module 6), and review vendor settings.
Secrets never touch the model
A secret is a credential: an API key, a database password, a token. Rules:
- Store secrets in a secret manager or environment variables on the server.
- Your code uses secrets to call tools. The model never sees them.
- Never paste a secret into a prompt, a chat, a test file, a screenshot, or a shared doc.
- Give each tool its own credential with narrow scope, so one leak exposes little.
- Rotate (replace) secrets on a schedule and immediately after any suspected leak.
# Run-it-yourself pattern: the key comes from the environment, not the code.
import os
api_key = os.environ["MODEL_API_KEY"] # set outside the code; never commit it
Least privilege
Least privilege means every user, agent, and tool gets the minimum access its job needs, and nothing more. Express it as scopes: named permissions such as cases:read.
The first run downloads Python into your browser (several MB).
Apply least privilege in layers:
- Per agent: a public FAQ bot gets no case data at all.
- Per user: a caseworker's assistant only retrieves that caseworker's clients.
- Per tool: a database account that can read one view, not the whole database.
- Per session: temporary credentials that expire.
A national example
Security review
0 of 8 doneGo deeper: Cybersecurity, Plain and Simple: A Practical Playbook for Owners and Operators
Discussion
Ask. Answer. Build together.
Loading…