Threat model for LLM applications (OWASP Top 10 for LLMs 2025)
Threat model an LLM app by writing down three things: what it can read, what it can do, and who can talk to it. Then walk through the OWASP Top 10 for LLM Applications (2025), and for each risk that applies, write what it would look like in your system and the first control you'll put in place. Do it on one page per application, before launch, and update it whenever the app gets a new data source or a new tool.
Start with three questions
- What can it read? The system prompt, user messages, retrieved documents, web pages, emails, tool results, other users' data, secrets.
- What can it do? Answer only, or call tools that send, write, pay, delete or run code.
- Who can talk to it? Employees, logged-in customers, anyone on the internet, other systems.
An internal summariser that reads approved documents and can't act has a small threat model. A public agent that reads email and can send messages has a large one.
OWASP Top 10 for LLM Applications (2025)
| ID | Risk | What it looks like | First control |
|---|---|---|---|
| LLM01 | Prompt injection | A document or message tells the model to ignore its instructions | Treat all external text as data, restrict tool permissions, approval for risky actions |
| LLM02 | Sensitive information disclosure | The model reveals personal data, another customer's records or secrets | Mask PII, permission-filtered retrieval, no secrets in prompts |
| LLM03 | Supply chain | A compromised model, dataset, package or MCP server | Pin and verify models, datasets, packages and servers |
| LLM04 | Data and model poisoning | Bad data in training, fine-tuning or the RAG index changes behaviour | Control and review sources, track lineage |
| LLM05 | Improper output handling | Model output is executed as code, SQL or HTML without checks | Validate and escape outputs before using them |
| LLM06 | Excessive agency | An agent with more tools or permissions than its task needs | Least-privilege tools, human approval, no production credentials in dev agents |
| LLM07 | System prompt leakage | Users extract the system prompt, and anything secret in it | No secrets in system prompts, test extraction attempts |
| LLM08 | Vector and embedding weaknesses | Retrieval returns data across tenants or permissions | Tenant isolation and access control in vector stores |
| LLM09 | Misinformation | Confident wrong answers that people act on | Grounding, citations, abstention, verification |
| LLM10 | Unbounded consumption | Inputs that cause huge outputs, loops or bills | Rate limits, token and step budgets, spend caps |
A one-page threat model
System: customer support assistant (public, logged-in customers)
Reads: help centre, customer's own orders, system prompt
Does: create ticket (write), check refund eligibility (read)
Talks to: logged-in customers
| Risk | Applies? | How it shows up here | Control | Owner |
|-------|----------|----------------------------------------------|------------------------------------------|-------|
| LLM01 | Yes | Customer asks it to ignore policy | Policy checks in code, no refund tool | Eng |
| LLM02 | Yes | Another customer's order in an answer | Order tool scoped to session customer | Eng |
| LLM05 | Yes | Ticket text rendered as HTML in admin panel | Escape output in admin UI | Eng |
| LLM06 | Low | Only one write tool, low impact | Rate limit ticket creation | Eng |
| LLM07 | Yes | Prompt extraction attempts | No secrets in prompt, red-team cases | Sec |
| LLM09 | Yes | Wrong policy answer | Citations, policy evals, handoff | Support |
| LLM10 | Yes | Very long inputs, repeated requests | Input length limit, per-user rate limit | Eng |
Review it with someone from security before launch.
Combine with your architecture
Map each risk to a zone of your pipeline: injection and PII at the input, permissions in retrieval and tools, output handling and leakage at the output, consumption at the gateway. See the LLM inference pipeline.
From the floor
In the production pipeline I teach, three of these risks are handled before the expensive model call: personal data is masked first, then the query is checked for prompt injection and jailbreak attempts, and only then does it reach the model, while the output zone checks for leaked prompts, internal data and toxic content before the answer returns.