Prompt injection is not a prompt problem

Every few months someone publishes a clever jailbreak and the industry responds by adding a sterner paragraph to its system prompts. That response misunderstands the problem. There is no sentence you can add that makes a language model incapable of being persuaded, because persuasion is the interface.

The systems I have built that survive adversarial input do not win the argument. They make the argument irrelevant.

The attack that actually happens

Direct jailbreaks are a demo genre. The real incidents I have been called into all had the same shape: instructions hidden inside content the agent was asked to process.

• A line in a PDF footer, set in white text, telling the agent to forward the document.
• A support ticket whose body contained a fake system message about escalation permissions.
• A vendor invoice with a note instructing the agent to update banking details.
• A web page with hidden markup aimed at a research agent's summarizer.

The attacker never touches your infrastructure. They send you a document, which is a thing your business exists to receive.

Defense one: capability, not instruction

This is the whole game. If the agent's credential cannot send email, no phrasing produces an email. If it cannot write to the payments table, a clever document cannot change bank details.

1. Give each agent its own service identity, scoped to the smallest set of operations its job requires.
2. Enumerate its tools explicitly. No generic query, exec, or fetch tools in an agent that handles untrusted input.
3. Deny network egress by default in any sandbox where the agent runs code.
4. Make destructive operations structurally unavailable rather than discouraged.

Defense two: separate the reader from the actor

The single highest-leverage architectural change I make on client systems. One component reads untrusted material and has exactly one power: return structured findings. A second component, which never sees the raw text, validates those findings against a schema and performs actions.

Injected instructions land in the reader, which cannot act, and the actor receives a validated object with no room for narrative. The attack surface collapses to whatever the schema permits, which you control precisely.

Defense three: gate the irreversible

Sort every action by whether you can take it back.

• Reversible. Drafts, notes, internal status changes. Let the agent proceed.
• Externally visible. Customer email, public posts. Human approval, or a strict allowlist of templates and recipients.
• Financial or destructive. Payments, deletions, permission changes. Always a human, or a one-time signed grant tied to that specific payload.

Approval steps are cheap when the agent has already done the work. A person reads a two-line summary and clicks. That is not friction, that is the value of the automation being preserved.

Defense four: validate outputs, not just inputs

Input validation is standard. Output validation is where injection gets caught late but usefully. Before any write, check that the payload conforms to a narrow schema, that identifiers refer to records the current task is actually about, and that values fall inside sane bounds.

A refund tool that accepts any amount is a liability. One that accepts an amount less than or equal to the original charge, for an order in the current task context, is nearly impossible to abuse regardless of what the model was told.

We had three layers of instruction telling it not to do this. It had the permission to do it. The permission won.

Where guardrail models fit

A classifier in front of the agent that flags instruction-shaped content in retrieved material is genuinely useful. It cuts noise and catches lazy attempts. It is also probabilistic, so it belongs in front of architectural controls rather than instead of them. Log its verdicts. The false negatives become test cases in your eval suite.

Detection, because prevention is never complete

• Alert on tool calls that fall outside a job type's normal pattern. An intake agent calling the email tool is an event.
• Alert on any denied-by-scope attempt. Those are either bugs or attacks, and both deserve attention.
• Keep an append-only audit log of write and irreversible calls with run correlation, per observability.
• Review a sample of runs weekly by hand. Patterns show up to humans that no alert was written for.

The checklist

1. Least-privilege credential per agent, enumerated tools only.
2. Reader and actor separated by a validated schema.
3. Irreversible actions behind a human gate or signed grant.
4. Untrusted content fenced, labeled, and never blended into rules.
5. Output validation with bounds and task-scoped identifiers.
6. Sandbox with no default egress for any code execution.
7. Alerts on anomalous and denied tool calls, plus a weekly read.

The protocol-level version of this, for teams exposing tools over MCP, is in MCP server security and auth. If you want an adversarial read on an agent you already run, that is a core part of the AI production readiness audit.

Frequently asked