What is prompt injection?
Prompt injection is a vulnerability in which content fed to a large language model (LLM) changes the model's behaviour or output in ways the developer did not intend. That content can be typed directly by a user, or it can sit inside a web page, email or document the model is asked to read. When an AI Agent can call tools (send email, query data, edit records), the result is no longer just a wrong answer: it can be a real action.
OWASP ranks prompt injection first (LLM01:2025) in the OWASP Top 10 for LLM Applications 2025. Its definition notes that injected content does not need to be visible or readable to humans, as long as the model parses it.
The UK National Cyber Security Centre (NCSC) puts the root cause plainly: current LLMs do not enforce a security boundary between instructions and data inside a prompt (NCSC, December 2025). Applications concatenate developer instructions with untrusted content into one string and then assume the model will keep the two apart. That assumption does not hold.
Direct vs indirect prompt injection
OWASP and NIST both split prompt injection into two classes. NIST AI 100-2 E2025, NIST's taxonomy of adversarial machine learning attacks finalised on 24 March 2025, treats direct and indirect prompt injection as distinct attack types against generative AI systems (NIST).
The indirect form was first described systematically by Greshake et al. (2023). The authors showed that LLM-integrated applications blur the line between data and instructions, so an attacker can exploit an application remotely, without a direct interface, simply by planting prompts in data that is likely to be retrieved (original paper).
| Criterion | Direct | Indirect |
|---|---|---|
| Where the malicious content comes from | Chat box, form field or API parameter the attacker fills in | Web pages, emails, PDFs, documents in a RAG store, tool outputs |
| What the attacker needs | Access to interact with the application | Only a place where the AI will read their content; no access to the interface |
| Who is affected | Usually the attacker's own session | Legitimate users who ask the AI to process that content |
| Example in OWASP LLM01 | A support chatbot told to ignore its guidelines, query private data and send emails | A web page with hidden instructions makes the LLM insert an image linking to an external URL, leaking the conversation |
| How hard to detect | Easier, since it sits in user input | Harder, since it arrives through a legitimate retrieval channel |
How prompt injection differs from jailbreaking
OWASP treats jailbreaking as a form of prompt injection in which the input makes the model disregard its safety rules entirely. For businesses, a more useful distinction is the target: a jailbreak targets the model itself (getting it to say something forbidden), while prompt injection in an application usually targets what the application is allowed to do: the data it can read and the tools it can call.
Intentional and unintentional
OWASP notes that both forms can be unintentional. Its LLM01 example: a company hides an instruction in a job posting to flag AI-written applications; a candidate uses an LLM to polish their CV and unknowingly triggers it. Nobody acted maliciously, yet the outcome was still manipulated.
Why AI Agents raise the stakes
A chatbot that only returns text can, at worst, give a wrong answer or leak its system prompt. An AI Agent is different: it reads data from many sources, decides which tool to call and repeats over several steps. Every data source is an entry point for indirect injection, and every tool is something an attacker can borrow.
OWASP lists the possible outcomes of prompt injection: disclosure of sensitive information, exposure of infrastructure details or system prompts, manipulated content, unauthorised access to functions available to the LLM, arbitrary command execution in connected systems, and manipulation of critical decisions (LLM01). Severity, OWASP says, depends largely on the business context and on how much agency the system is given.
Excessive agency (OWASP LLM06)
OWASP defines excessive agency as the vulnerability that lets damaging actions happen in response to unexpected, ambiguous or manipulated LLM outputs, whatever the cause (LLM06:2025). Its root causes are usually excessive functionality (a tool does more than needed), excessive permissions (a tool runs with more privilege than needed) and excessive autonomy (high-impact actions with no human approval).
Prompt injection is the trigger; excessive agency sets the size of the blast. If an AI Agent can only read, a successful injection does far less harm than when it can send email, delete files or move money.
Tool-based data exfiltration
OWASP LLM06 gives a typical scenario: an email-summarising assistant gets a mailbox extension to read messages, but the extension can also send them. A malicious incoming email carries hidden instructions that make the LLM scan the inbox for sensitive data and forward it to the attacker. OWASP's fix: use a read-only extension, authenticate with a read-only scope, and require the user to approve every send action manually.
The exfiltration channel does not have to be a 'send' tool. The second scenario in OWASP LLM01 shows that an image link inserted by the LLM into its answer is enough to carry conversation data out when the interface loads that image automatically.
- Illustrative example: a sales AI Agent reads customer notes in a CRM. A customer types an instruction into a 'notes' field telling the agent to attach other customers' details to its reply. If the agent can read the whole CRM and send email without review, that instruction may be carried out.
- Illustrative example: an internal knowledge assistant using RAG retrieves a tampered document. A hidden instruction in it makes the answer skip risk clauses. OWASP describes exactly this case in LLM01 (Scenario #4).
Why prompt injection cannot be 'patched' like SQL injection
SQL injection is controlled with parameterised queries: the database keeps commands and data strictly apart. The NCSC argues that comparing prompt injection with SQL injection is dangerous because no such separation exists inside an LLM, and it assesses that prompt injection may never be fully mitigated in the way SQL injection can be (NCSC).
OWASP likewise says that, given the stochastic nature of generative models, it is unclear whether fool-proof prevention exists. The practical consequence: the goal is not to 'block everything' but to assume some injections will get through and design so that the damage stays small when they do. The NCSC recommends focusing on deterministic, non-LLM safeguards that constrain what the system can do, rather than only trying to keep malicious content away from the model.
One easy rule the NCSC cites: when an LLM processes information from a party, its privileges should drop to that party's level. An AI Agent reading email from strangers should not also hold privileged tools.
How to defend against prompt injection: layered defenses
The table groups OWASP LLM01's seven mitigations, the LLM06 controls and the NCSC's design advice into layers. No layer is enough on its own; the value is in the combination. For the permissions and logging layers in depth, see AI Agent access control and audit logs.
| Layer | Control | What it reduces | Limits |
|---|---|---|---|
| 1. Constrain behaviour | System prompt defines role and scope and tells the model to ignore attempts to change core instructions (OWASP LLM01 #1) | Simple override attempts | The model can still be persuaded; not a security boundary |
| 2. Segregate untrusted content | Clearly mark external content (web, email, documents) apart from instructions (OWASP LLM01 #6) | Influence of indirect injection | Marking is a signal to the model, not an enforcement mechanism |
| 3. Filter inputs and outputs | Semantic filters, string checks, deterministic validation of output formats (OWASP LLM01 #2, #3) | Known attack patterns, malformed output | The NCSC warns against relying on deny-lists: attacks can be rephrased endlessly |
| 4. Least privilege | Expose only the tools and functions truly needed; avoid open-ended tools such as shell or arbitrary URL fetch; keep API tokens in application code (OWASP LLM01 #4, LLM06) | Blast radius when an injection succeeds | Needs ongoing review as tools are added |
| 5. Authorise in downstream systems | Target systems check permissions against the user's identity instead of trusting the LLM (LLM06) | Agent cannot do what the user is not allowed to do | Requires careful integration design |
| 6. Human approval for risky actions | Manual approval before external sends, deletions, payments, permission changes (OWASP LLM01 #5) | Irreversible actions | Too many approvals lead to rubber-stamping; choose the points carefully |
| 7. Logging, monitoring, rate limits | Log every tool call; cap actions per time window (LLM06) | Time to detect and scale of damage | Does not prevent, only limits impact |
| 8. Adversarial testing | Regular red-teaming that treats the model as an untrusted user (OWASP LLM01 #7) | Finds weaknesses before attackers do | Results are valid only at the time of testing |
Put people in the right place
Layer 6 works best when approval points sit on high-impact or irreversible actions and the approver sees enough context: what the agent wants to do, with which data, and why. Our human-in-the-loop guide covers how to choose approval points without slowing the whole workflow.
Prompt injection red-team checklist for AI Agents
This list is based on the attack scenarios in OWASP LLM01 and LLM06. Run it only against your own systems or with written permission, in a test environment, with synthetic data.
- Inventory every data source the agent reads (chat, email, web, RAG documents, tool outputs) and every tool it can call, with each tool's actual permissions.
- Test direct injection: requests to ignore instructions, reveal the system prompt, or call out-of-scope tools.
- Test indirect injection: plant instructions in web pages, emails, PDFs, CRM notes or RAG documents the agent will read.
- Test hidden content: white-on-white text, HTML comments, metadata, and instructions inside images for multimodal models (OWASP Scenario #7).
- Test filter evasion: payload splitting across fragments, mixed languages, Base64 encoding (OWASP Scenarios #6 and #9).
- Test exfiltration: can the agent be led to put data into URLs, images, emails or outbound API calls?
- Check excessive agency: do tools have unneeded functions, run with more privilege than the user, or take any action without approval?
- Check approval points: does the approver see enough to spot an unusual request?
- Check logs: can you reconstruct the chain (data source → decision → tool call) after each test?
- Record results, fix at the architecture level (permissions, tools, approval points) before tuning prompts, then re-run everything.
Fit AI Agent security into a risk framework
NIST's AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1) organises the work into four functions: GOVERN, MAP, MEASURE and MANAGE (NIST AI RMF Core). It is voluntary, but useful for turning the technical controls above into a process with clear owners. NIST AI 100-2 supplies the attack vocabulary; the AI RMF supplies the management process.
| Function | What to do for AI Agents |
|---|---|
| GOVERN | Policy on which tools and permissions agents get; who approves new tools; who is accountable in an incident |
| MAP | Inventory of untrusted data sources, tools and irreversible actions; decide whether an LLM is appropriate for the task at all |
| MEASURE | Regular red-teaming; track attack block rate and false-block rate on legitimate requests |
| MANAGE | Incident plan: revoke tokens, disable tools, review logs, notify affected parties |
When not to hand a task to an AI Agent
The NCSC is blunt: if a system cannot tolerate the remaining risk, it may not be a good use case for an LLM. For tasks that read external content, hold sensitive data and can send data out, split the work into controlled steps or keep the final step with a person.
How EYC approaches AI Agent security
EYC (Everything You Connect) builds AI Agent systems on a human-led principle: agents propose and coordinate, and people approve important actions. In our AI & Automation and R&D & Applied AI work, we start from an inventory of data and tools, grant least privilege, put approval points on irreversible actions and log tool calls. EYC operates internal processes for information security, role-based access control, audit logging and incident response. Commitments such as scheduled testing or incident-response support are provided under contract.
If your agent connects to data through a standard protocol, our what is MCP guide explains why each MCP server should also be treated as an untrusted input source.
Frequently asked questions
What does prompt injection mean?
It is an attack that uses text, typed directly or hidden in data the AI reads, to make a language model follow the attacker's instructions instead of the developer's. OWASP ranks it as the top risk (LLM01:2025) for LLM applications.
What is indirect prompt injection?
It is the form in which malicious instructions sit in external content the AI is asked to process, such as web pages, emails, PDFs or documents in a RAG store. The attacker never interacts with the application; a legitimate user unknowingly hands the content to the AI.
Can any tool fully block prompt injection?
According to OWASP and the NCSC, no fool-proof prevention exists today because LLMs do not separate instructions from data. Filters catch known patterns, but they must be combined with least privilege, human approval of risky actions and monitoring to limit damage.
How is prompt injection different from jailbreaking?
OWASP treats jailbreaking as a form of prompt injection aimed at making the model ignore its safety rules. In business applications, prompt injection usually targets the data and tools the application can use, so the outcome can be a real action rather than just forbidden content.
Where should a small business start with prompt injection defense?
Inventory every data source and tool your agent uses, remove unneeded permissions and functions, require manual approval for external sends and irreversible actions, turn on tool-call logging, then run the red-team checklist in a test environment.