AI agent security, what actually breaks

Agent security also includes operational failures. Consider a hypothetical agent with excessive access, a retry that repeats an email, or a staging tool configured with production credentials.

These failures can happen without an attacker. This article examines the permissions and operational checks that limit their consequences. Adversarial input has its own post, prompt injection is not a prompt problem, and the protocol layer is covered in MCP server security and auth.

The incidents are boring, and that is the point

The following five scenarios are illustrative failure modes, not client incident reports.

• A support agent with a full database role, added during a demo, never narrowed. It updated a column it was never meant to see.
• One shared API key across three agents. When it leaked, nobody could say which system to shut off, so all three went down for a day.
• A retry with no idempotency key. The provider timed out, the workflow retried, and the customer got charged twice.
• A logging call that wrote full tool payloads, including the contents of identity documents, into a log tool half the company could read.
• A cron-triggered agent that kept running for six weeks after the project was cancelled, because nothing owned its lifecycle.

Review these scenarios before launch and test the controls relevant to your system.

Blast radius is the only number that matters

Before an agent goes live, answer one question in writing. If this thing does the worst permitted action a thousand times in a row, what happens?

That is blast radius. It is a property of the credential and the tool list, not of the prompt, so it is measurable. Write it down, then shrink it until you can live with the answer.

1. One service identity per agent. Never a shared key, never a developer key, never the deploy key.
2. Scope to the smallest set of operations the job needs, then remove one more and see if anything fails.
3. Rate limit at the tool layer, not just at the provider. Per run, per hour, per tenant.
4. Cap spend and volume with hard stops that page a human instead of degrading quietly.

The agent should never hold the key

The instinct is to put the secret in the agent's environment and let it call the API. That collapses two very different jobs into one. Instead, keep every credential behind a tool the agent calls by name.

The tool holds the secret. It verifies who is calling, validates the arguments against a narrow schema, enforces its own limits, writes an audit record, and returns a small structured result. The agent never sees the credential, so a leaked transcript is embarrassing rather than expensive.

• Verbs, not endpoints. Give it refund_order, not http_request. A generic fetch tool is a credential with extra steps.
• Bounded arguments. Amount less than or equal to the original charge. Recipient inside the current task context. Table allowlisted, not templated.
• Short-lived tokens. Mint per run where the provider supports it. Long-lived secrets are a permanent liability.
• Idempotency by default. Every write tool takes a key derived from the run and the payload. Retries then stop being incidents.

Gate the irreversible, let the rest run

Approval on everything is the same as approval on nothing, because people start clicking without reading. Sort actions by whether you can take them back, then spend the human attention where it counts.

• Reversible. Drafts, internal notes, status changes, enrichment. Unattended.
• Customer visible. Email, SMS, public posts, tickets. Template allowlist plus recipient checks, or a human on anything outside the templates.
• Money or deletion. Payments, refunds, cancellations, permission changes, destructive writes. A human, or a one-time signed grant tied to that exact payload.

The approval should read like a receipt, not a puzzle. Two lines of summary, the exact payload, one button. If a reviewer has to open three tabs to understand what they are approving, the gate is decoration.

In a hypothetical retry failure, one permitted action can reach more customers than intended. Test volume limits as well as the action itself.

Logging that survives an audit

Security work you cannot reconstruct is a story, not a control. Every write and every irreversible call needs a record you can hand to a customer, a partner, or a lawyer without editing.

1. Append-only. If the agent's role can delete audit rows, the audit log is a suggestion.
2. Correlated by run ID, so one incident reads as one story across tools, retries, and handoffs.
3. Record the authorization decision, not just the action. Which policy allowed this, and which identity asked.
4. Redact contents, keep the shape. Field names, sizes, and hashes are enough to debug. Document bodies in a log tool are a second breach waiting to happen.
5. Alert on denied-by-scope attempts. Those are either bugs or probes, and both deserve a look the same day.

The wider version of this, including which metrics are worth a dashboard, is in AI agent observability, what to log. Failures caught there belong in your eval suite the same week, so the same break cannot ship twice.

Ownership and shutdown

An agent with no owner becomes a permanent unmonitored process with production credentials. Give every deployed agent a named human, a review date, and a documented off switch. Then actually use the review date.

• Who gets paged when it misbehaves at 2am.
• What the kill command is, and who has run it once on purpose.
• When the credential rotates, and whether rotation is tested.
• What happens to its access when the project ends.

The pre-launch checklist

1. One scoped service identity per agent. No shared or human keys.
2. Blast radius written down and accepted by a named owner.
3. Credentials behind tools. Verbs with bounded arguments, no generic fetch or exec.
4. Idempotency keys on every write. Retries proven safe.
5. Rate limits and hard spend caps that page instead of degrading.
6. Irreversible actions behind a human gate or a signed one-time grant.
7. Append-only audit log, run-correlated, redacted payloads.
8. Alerts on anomalous and denied tool calls, plus a weekly manual read.
9. Tested kill switch, rotation plan, and a review date on the calendar.

Use the checklist to identify the controls your workflow needs and the evidence for each one. If you have a stalled product build that has to work for a real customer, pilot, or launch, send the project brief. Scope and price are agreed before work starts.

Frequently asked