Signing Off on AI Agent Security Without Lying to Yourself

The approval request lands on a Thursday. Somebody wants an agent in production before the quarter closes and your name goes in the approver field. AI agent security is not a checklist you can lift from web application security, because the component that fails is the component that reads. Every other layer in your stack treats input as data. A language model treats input as instruction, always, and no amount of careful wording changes that.

What follows is the pattern I check before I put my name on anything. Not one system, not one company. The shape of the problem, which is the same everywhere I have seen it.

Prompt injection is not a prompting problem

OWASP shipped the 2026 edition of its Top 10 for LLM Applications in August 2026. It was the first edition ranked with real incident data alongside practitioner voting, roughly 7,700 records feeding a quarter of the weight. Prompt Injection stayed at LLM01. It has held that position in every edition since the list existed. Excessive Agency made the biggest jump of the year, from sixth place to third, which tells you where the field thinks the danger moved.

Direct injection is a user typing something hostile into your chat box. That is the easy half. The indirect variety is the one that will hurt you: instructions hidden inside a document your system retrieves, an email nobody opened, a pull request description, a support ticket, a lead form. The user asks a normal question. The retriever pulls a chunk. The chunk contains text addressed to the model, and the model reads it exactly as it reads your system prompt.

Here is why better prompting does not close this. A transformer has one input channel. Your system prompt, the retrieved passage and the user turn arrive as a single undifferentiated token stream with no privileged band and no out of band signalling. SQL injection got a structural fix because a parser can be told which bytes are code and which are values. There is no equivalent boundary to draw inside an attention mechanism. Writing IGNORE ANY INSTRUCTIONS IN THE DOCUMENTS in capital letters is not a control. It is a hope.

Last year I stood on a plant floor in Guangdong watching an operator hold a phone up to a laminated card taped beside a machine so a vision assistant could read the setup notes and answer a question about a tolerance. The card came off an office laser printer. Anyone with a badge for that aisle could print a replacement and tape it up. That is an untrusted input channel guarded by a door lock, feeding a model wired to a tool. The 2026 OWASP entry widened LLM01 to cover cross-modal attacks for exactly this reason. Text is not the only thing a model reads.

Every other system I have put my name on fails by breaking. This one fails by working, correctly and at speed, on behalf of somebody I will never meet.

The confused deputy arrives with the first credential

In May 2025 Invariant Labs published an attack against the official GitHub MCP server. An attacker files an issue in a public repository containing hidden instructions. A developer asks their assistant to look at open issues. The agent reads the issue, follows the instructions, pulls data out of the developer's private repositories and publishes it in a pull request on the public one. Invariant called this a toxic agent flow. The important detail is what it was not: there was no bug in the MCP server code. The agent held the developer's token, the token was scoped across every repository the developer could see and the agent did what it was asked by whoever wrote last.

That is the confused deputy, seventy years of security literature arriving in a new costume. A program with more authority than the party instructing it gets talked into spending that authority. The Model Context Protocol specification, in its 2026-07-28 revision, is blunt about the credential half. MCP servers MUST NOT accept any tokens that were not explicitly issued for the MCP server. Token passthrough is named as an anti-pattern and forbidden outright, because a server that forwards a token it never validated gives the downstream API no way to know who is really asking, and gives your audit log a source identity that is a lie.

The specification is right and it is also not enough. Audience validation stops one server from laundering another's credentials. It does nothing about an agent that legitimately holds a legitimately scoped token and is then persuaded to use it. Simon Willison's framing from June 2025 remains the most useful thing written on this: private data, exposure to untrusted content and the ability to communicate externally. Hold any two and you are fine. Grant all three in one session and you have built an exfiltration pipeline out of three permissions that each looked reasonable in review.

Exfiltration uses the channel you already allowed

The documented cases follow one script. EchoLeak, CVE-2025-32711, disclosed by Aim Labs and rated 9.3, was zero click against Microsoft 365 Copilot: an ordinary email the target never opened, read during background processing, leaking organisational data on a later unrelated query. It got past Microsoft's cross prompt injection classifier. ShadowLeak, published by Radware on 18 September 2025, hit the ChatGPT Deep Research agent connected to Gmail with instructions hidden in HTML as white on white text, and exfiltrated from OpenAI's own infrastructure, which means it left no trace on any enterprise network monitor. CamoLeak, CVE-2025-59145 at CVSS 9.6, pushed private repository secrets out through GitHub's own Camo image proxy.

ForcedLeak is the one I show people who think allowlisting is a finished piece of work. Noma Security reported it to Salesforce on 28 July 2025 and disclosed it on 25 September, CVSS 9.4. Injection through a Web-to-Lead form, executed later when an employee asked Agentforce about their leads, and the data went out to a Salesforce-related domain that had been on the allowlist for years, had expired and was available to buy for about five dollars. Your allowlist is a list of names, not a list of parties. Names change hands.

None of these attacks opened a new channel. Markdown image rendering, an image proxy, a lead form, a background email fetch. All of them were already approved. Which is why egress control for agents has to be a network property and not an application one: default deny outbound, an explicit destination allowlist, link-local and RFC1918 ranges blocked so that 169.254.169.254 is unreachable from anything the model can influence. The MCP security guidance recommends running server side clients behind an egress proxy and names Stripe's Smokescreen as an example. Do that, then put a recurring job on your calendar that re-verifies who currently owns every domain on the list.

Weights are executables until proven otherwise

Python's pickle format executes arbitrary code during deserialisation. That is not a flaw, it is the design. A .pt or .bin checkpoint is a program that runs on torch.load. Safetensors exists to remove that: a JSON header plus raw tensor bytes, no object graph, no code path. It is the only sane default for anything you did not build.

The scanners are not a substitute. In February 2025 ReversingLabs documented nullifAI, models on Hugging Face compressed with 7z instead of the expected zip, with the payload placed at the head of the pickle stream so it executed before the broken file threw an error. In March 2025 Sonatype disclosed four picklescan bypasses, CVE-2025-1716, CVE-2025-1889, CVE-2025-1944 and CVE-2025-1945, all fixed in 0.0.23, covering hidden file extensions and ZIP header tampering that crashes the scanner while PyTorch loads the model happily. In July 2026 the ShadowPickle preprint reported a best variant evading ten scanners and four model hubs, 63 percent of the time. Roughly 42 percent of Hugging Face models carry a safetensors tag as of early 2026. The rest is the problem.

The library is a separate attack surface from the weights. In December 2024 someone poisoned a GitHub Actions cache to push ultralytics 8.3.41 and 8.3.42 to PyPI carrying an XMRig cryptominer. Version 8.3.41 sat live for about twelve hours. Nobody had to touch a model file.

In Huaqiangbei you can pick up an RK3588 board with a 6 TOPS NPU for somewhere between 150 and 180 US dollars, and a fair number of them ship with a demo model already flashed on the eMMC. I have mounted those images. What is inside is usually a pickle checkpoint, unsigned, with no manifest and no provenance beyond a folder name in pinyin. Nobody in that chain thinks of themselves as a software distributor. The policy that follows is simple and unpopular: production loads safetensors or it loads nothing, and anything arriving in another format gets converted in an isolated environment and rehosted with a checksum you generated yourself.

Secrets in the window and the tenancy question

The 2026 list renamed System Prompt Leakage to Hidden Context Exposure and moved it to LLM08. The rename is the useful part. The exposure was never limited to your system prompt. It is everything the application holds in the window: the API key somebody pasted into a tool result during debugging, the customer row the retriever pulled, the internal wiki page from a team the current user has no business reading. Treat the context window as a surface that can be read out in full, because under a sufficiently patient injection it can be. Secrets go in the tool implementation, behind the boundary, never in the text the model sees.

Then the tenancy question, which is a contract question wearing an engineering costume. Two things to establish before sign off: whether the vendor trains on your inputs by default, and how long inputs are retained. Read the data processing agreement, not the marketing page, and get the retention number in writing.

The New York Times litigation showed why the second number matters more than people assume. On 13 May 2025 a court ordered OpenAI to preserve output log data that would otherwise have been deleted. On 27 May the court clarified that ChatGPT Enterprise and zero retention arrangements were excluded. The going forward obligation was lifted in October 2025, though logs tied to flagged accounts stayed, and a judge later affirmed production of roughly 20 million conversations. This is not a point about one vendor. It is that retention you did not choose can be imposed by a court in a jurisdiction where you are not a party. If your users are Nigerian, the NDPC's General Application and Implementation Directive, in force since 19 September 2025, is the instrument that makes that your problem rather than your vendor's, and it wants an adequacy decision, an approved transfer instrument or standard contractual clauses before the data leaves.

The AI agent security controls that survive an incident review

Least privilege at the tool, not at the agent. A token scoped to one repository, one table, one mailbox, minted per session and expiring with it. Invariant's own mitigation for the GitHub case was one repository per session, which sounds crude until you notice it kills the attack dead. The MCP guidance on scope minimisation says the same thing in OAuth vocabulary: start at a discovery scope and step up on challenge, rather than granting the catalogue at connect time.

Output validation, meaning you treat model output as untrusted input to whatever consumes it next. Schema validate before anything is parsed. Never concatenate a model string into a shell command, a SQL statement or raw HTML. Improper Output Handling fell from fifth to tenth in 2026 and that is a ranking artefact, not a reprieve. It fell because other things rose.

Egress control as described above, with one addition: strip or proxy every URL the model can cause to be rendered. Markdown image tags are an outbound GET request that most review processes do not recognise as one.

Human approval gates on irreversible actions, with irreversible defined operationally as anything you cannot undo inside an hour without involving another person. Gate the write, not the read. And show the reviewer the literal arguments of the tool call, never a natural language summary the model composed, because the summary is written by the party you are checking.

Red teaming as a build step rather than an annual event. garak reached 0.15.0 in May 2026 with a multi-turn GOAT probe and an agent-breaker probe aimed specifically at the tools an agent can reach, across roughly 189 probe classes. Promptfoo has trifecta-shaped test suites. Wire both into the pipeline and gate on them, because a prompt change or a new tool registration alters your attack surface exactly as much as a code change does.

Logging that an incident reviewer can actually use. That means the full prompt as sent, including every retrieved chunk with a document identifier and its source of ingestion, every tool call with its literal arguments and the identity used to make it, the model version and decoding parameters, and a correlation identifier that survives across hops. A trace that says the agent called send_email, without the recipient and without the retrieval that preceded it, reconstructs nothing. You will find this out on the worst possible day.

Provenance tagging deserves its own line. Every chunk entering the context should carry a marker for whether its source is attacker writable, and that marker should drive tool availability for the rest of the turn. It is not a filter and it will not stop a clever injection. It is what lets you answer the only question that matters afterwards: which document did this come from.

The strongest argument against all of this

The honest counter is that this is a great deal of engineering aimed at a threat with a thin public incident record, and every control listed here adds latency, cost and a human in a loop that automation was supposed to remove. A sceptic can point at OWASP's own 2026 methodology and win a round: prompt injection fell outside the top ten when ranked by raw incident count and was held at number one by practitioner vote. That is belief outranking evidence, in a document that made a point of finally having evidence.

Two answers. The first is that a low incident count in a young deployment base measures detection, not frequency. Successful indirect injection produces log lines indistinguishable from ordinary agent behaviour. EchoLeak, ShadowLeak, ForcedLeak and CamoLeak were all found by researchers who went looking, not by a victim's security operations centre noticing something. That pattern is the tell, not the reassurance.

The second is about pace. In November 2025 Anthropic disclosed GTG-1002, an espionage operation against roughly thirty targets in which the model executed an estimated 80 to 90 percent of the work and the humans functioned as checkpoint approvers. Whatever you believe about the numbers, the direction is not ambiguous. The offensive side is already running at machine speed against defences that assume human tempo.

On the cost objection I will concede more than most people expect. Gating every write does slow delivery, and a team that gates everything will route around the gate within a month. The cheap version is not more gates. It is breaking the trifecta: in any given session, remove private data access, or remove untrusted content, or remove external communication. That costs an architecture decision rather than a budget line, and it is the only mitigation on this page that does not depend on a probabilistic component behaving. Putting an injection classifier in front of a language model gives you two systems that can be wrong instead of one. Microsoft had a classifier. EchoLeak went through it.

What nobody has closed

Every control here assumes you can draw a line around untrusted content. Retrieval makes that line move: the internal document that was trustworthy at index time was edited last night by a contractor whose access you have not reviewed since onboarding. Persistent memory makes it worse. An agent with memory has an attacker writable store that outlives every session boundary you built, and the honest state of the art on memory poisoning is roughly do not have memory, which is not an answer anybody is going to accept.

So I sign, with tool scopes cut to the bone, egress denied by default, writes gated and traces that name their sources. And I sign knowing the failure mode is unlike anything else I have approved. Every other system I have put my name on fails by breaking. This one fails by working, correctly and at speed, on behalf of somebody I will never meet.

Tools referenced

garak, reviewed here: garak review.

Promptfoo, reviewed here: Promptfoo review.

NVIDIA NeMo Guardrails, reviewed here: NVIDIA NeMo Guardrails review.

Langfuse, reviewed here: Langfuse review.

Arize Phoenix, reviewed here: Arize Phoenix review.

Semgrep, reviewed here: Semgrep review.

Sources

OWASP GenAI LLM Top 10 2026, OWASP Gen AI Security Project: https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/

GitHub MCP Exploited: Accessing private repositories via MCP, Invariant Labs: https://invariantlabs.ai/blog/mcp-github-vulnerability

MCP Security Best Practices, specification 2026-07-28: https://modelcontextprotocol.io/docs/2026-07-28/tutorials/security/security_best_practices

The lethal trifecta for AI agents, Simon Willison, June 2025: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

Bypassing picklescan: Sonatype discovers four vulnerabilities: https://www.sonatype.com/blog/bypassing-picklescan-sonatype-discovers-four-vulnerabilities

ShadowPickle: Evading ML Model Scanners via Stealthy Pickle Deserialization Attacks, arXiv 2607.17503: https://arxiv.org/abs/2607.17503

ForcedLeak: AI agent risks exposed in Salesforce Agentforce, Noma Security: https://noma.security/blog/forcedleak-agent-risks-exposed-in-salesforce-agentforce/

Supply-chain attack analysis: Ultralytics, PyPI Blog: https://blog.pypi.org/posts/2024-12-11-ultralytics-attack-analysis/