Prompt Injection in Production: Real Attack Patterns

EchoLeak (CVE-2025-32711) exfiltrated data from Microsoft 365 Copilot with a single crafted email, no user click required, no malware, no code, just text. It ranks as the first documented case of prompt injection weaponized for concrete data exfiltration in a production AI system, and OWASP has ranked prompt injection as the #1 security risk for LLM applications in 2025. The gap between how most teams think about this threat and how it actually plays out in production is significant enough to warrant a detailed look at the real attack surface, not the textbook version.

Textbook defenses fail because they model the wrong attacker

The standard framing treats prompt injection as a chatbot problem: a user submits malicious input, the model obeys instead of refusing, the system does something bad. Defenses built on this model target the input layer, input sanitization, system prompt hardening, classifier filters that flag suspicious phrasing.

This framing is accurate for simple, single-turn deployments. It breaks down completely the moment the LLM can read external content and invoke tools.

EchoLeak illustrates the gap precisely. The attack chained four distinct bypasses: it evaded Microsoft's XPIA (Cross Prompt Injection Attempt) classifier, circumvented link redaction by using reference-style Markdown formatting, exploited Copilot's auto-fetched image rendering, and abused a Microsoft Teams proxy. Each subsystem had its own validation logic, and none of them had visibility into what the other had already approved or what the other was about to do. The attacker didn't break through one gate; they navigated the space between gates that each assumed the other had checked the content.

According to the EchoLeak paper published at AAAI Fall Symposium 2025, the payload was entirely text. Antivirus, firewalls, and static file scanning were completely ineffective because the attack surface was the model's behavior, not a file's signature.

This is the pattern defenders need to internalize: in an agentic system, the dangerous input isn't the one that breaks the model's safety training. It's the one that arrives through a channel the model is designed to trust, then redirects capabilities the model is already authorized to use.

The real attack surface: trust transitivity across tool chains

When an LLM agent reads an email, a document pulled from a RAG index, or a webpage fetched by a browser tool, the model assigns that content some implicit level of trust based on the channel it arrived through. If the agent is configured to act on instructions found in emails from internal sources, and an attacker can get content into that email channel, the agent's granted capabilities become the attacker's capabilities.

This is what "trust transitivity" means in practice. The attacker doesn't need to compromise the model or the auth layer. They need to get their instructions into a content stream that the agent treats as authoritative, then let the agent's own tool permissions do the work.

Consider the OWASP-documented RAG attack scenario: an attacker modifies a document in a repository that feeds a RAG pipeline. When a legitimate user queries the system, the retrieval step returns the modified content. The model receives what looks like retrieved context from a trusted internal source and follows the embedded instructions. The user's query was clean. The system prompt was clean. The attack surface was the document the system fetched on the user's behalf.

Researcher Rehberger's work on Devin AI reinforces this. He spent $500 of his own money testing the agentic coding assistant and found it completely defenseless against prompt injection, with the agent manipulable into exposing ports and leaking credentials through content it encountered during normal task execution. The agent was doing exactly what it was designed to do, following instructions it encountered in its environment, which was the problem.

The structural issue is that after the initial authentication step, most agentic systems treat all content the agent retrieves as implicitly trusted. The capability grants, calendar write access, email send, file retrieval, code execution, don't get re-evaluated per content source. They were approved once, at setup, and they remain available throughout the session.

Why CVSS scores on LLM CVEs deserve more attention than they're getting

The severity numbers coming out of 2025 and 2026 are not typical. CVE-2025-32711 (EchoLeak) in Microsoft Copilot, CVE-2025-53773 in GitHub Copilot with remote code execution, and a critical Cursor IDE vulnerability all landed at CVSS scores above 9.0. These aren't theoretical research findings; they're production systems used daily by enterprise development and knowledge workers.

Only 34.7% of organizations have deployed dedicated prompt injection defenses according to Vectra AI's 2025 State of Threat Detection report, despite 83% of organizations planning to deploy agentic AI according to Cisco's State of AI Security 2026 report. The gap between deployment pace and defensive readiness is not subtle.

The 53.6% figure is worth sitting with: across frontier models, even after applying the best available defenses including adversarial fine-tuning, the most effective attack technique still succeeded more than half the time against Google Gemini in 2025 testing. Per Pillar Security's late 2024 analysis, 20% of jailbreak attempts succeed, and 90% of successful prompt injection attacks resulted in leakage of sensitive data. The average successful attack took 42 seconds across five interactions, which isn't a slow, sophisticated operation. These are fast, cheap, scalable attacks.

OpenAI's February 2026 launch of Lockdown Mode for ChatGPT came with a public acknowledgment that prompt injection in AI browsers "may never be fully patched." That acknowledgment is the highest-profile industry admission to date that complete defense requires trading off AI functionality, and that some attack surface is structural, not fixable by patching a classifier.

The specific failure mode that defenses keep missing

Most prompt injection defenses are built around detecting malicious intent in the injected content. Flag suspicious phrases, block instruction-like patterns, validate that retrieved content doesn't contain imperative sentences directed at the model. This is directionally correct but architecturally insufficient.

The failure mode is that sufficiently sophisticated injections don't look malicious in isolation. The EchoLeak payload didn't need to be flagged as dangerous; it needed to survive each subsystem's local validation, which it did, because each subsystem was checking for different things. Reference-style Markdown bypassed link redaction because the link redaction logic wasn't looking for that pattern. The image fetch executed because auto-fetching images was a feature, not a bug.

A more reliable defensive orientation is least-privilege enforcement at the tool level, not just the input level. Specifically:

• Grant tool capabilities per session scope, not per agent identity. An agent helping draft a document doesn't need email send access unless that task requires it.
• Treat retrieved external content as untrusted regardless of the channel it arrived through. A document from an internal SharePoint can be compromised. Emails from known senders can be forwarded with injected content appended.
• Log and alert on tool invocations that weren't directly requested by the user in that turn. If the agent sends an email and the user asked it to summarize a report, that gap should generate a flag.
• Evaluate whether the agentic system actually needs the capability, not just whether it can be granted. Capability minimization is a more durable control than classifier tuning, because classifiers can be bypassed and minimization can't.

Anthropic's documentation on building safe agentic systems addresses tool use and permission scoping, though the field is moving faster than guidance can keep up with. The useful principle is to treat any content the agent retrieves as an untrusted input surface, regardless of the source that delivered it.

If you're building agentic workflows with broad tool access and plan to rely on AI agents to automate complex operations, the security model for those agents needs to be designed before the capabilities are granted, not audited afterward. And the teams doing the security review need to understand how AI hallucinations and trust errors compound in retrieval pipelines, because the injection problem and the reliability problem share the same root: the model treats external content with more trust than it deserves.

FAQ

Is prompt injection the same as jailbreaking? Jailbreaking targets the model's own safety training, trying to get it to refuse less. Prompt injection targets the model's behavior in context, inserting instructions through content the model reads rather than through the user's direct input. They can overlap, but in agentic systems, injection is the more dangerous category because it can operate without any interaction from the legitimate user.

What is indirect prompt injection? Indirect prompt injection happens when the malicious instructions don't come from the user's own input but from content the agent retrieves during task execution, a document, a webpage, a database record, an email. The model never "sees" an attacker; it just processes content that contains embedded instructions. This is the attack pattern behind EchoLeak and the OWASP RAG scenario.

Does system prompt hardening stop these attacks? Hardening the system prompt reduces susceptibility to some direct injection attempts, but it doesn't address the trust transitivity problem in agentic pipelines. EchoLeak bypassed Microsoft's dedicated XPIA classifier, which is specifically built to detect cross-prompt injection. System prompt instructions like "ignore external commands" have limited effect when the injected content is formatted to look like retrieved context rather than user commands.

How do capability minimization and least-privilege apply here? The same way they apply in traditional access control: an agent should be granted only the permissions required for the specific task in the current session. Calendar write, email send, and file access should be scoped and revocable, not granted as standing permissions to the agent identity. An attacker who successfully injects instructions can only abuse the capabilities the agent already holds. Narrowing that set at the tool level is the most structurally durable control available right now.

The uncomfortable conclusion from EchoLeak is that the attack worked not because any single system failed, but because each system assumed the other had already validated what it was passing along. That assumption is built into how most agentic pipelines are designed today, and fixing it requires rethinking the trust model between components, not just adding another classifier at the edge.