LLM Application Security
Secure applications built on language models: prompt injection, unsafe output handling, excessive agency, data leaks, RAG access control, supply chain, abuse and cost attacks, testing and incident response, with attacks and defences you can run.
Syllabus
Why LLM Apps Need Their Own Security Thinking
- What Is Different About LLM Applications
- Assets, Attackers and Trust Boundaries
- Blast Radius: What Can a Compromised Model Do
- A Map of the Main Risk Families
Prompt Injection and Jailbreaks
- Direct and Indirect Prompt Injection
- Why Keyword Filters Are Not Enough
- System Prompt Leakage and Canary Tokens
- Defence in Depth Against Injection
Insecure Output Handling
- The Rule: Treat Model Output Like User Input
- SQL Injection Through Model Output
- Cross-Site Scripting and Markdown/Link Exfiltration
- Command Injection and Path Traversal
- Tools That Fetch URLs: SSRF and Allow-Lists
Excessive Agency and Tool Security
- Over-Broad Tools, Permissions and Autonomy
- A Policy Engine Outside the Model
- Human Approval That Cannot Be Tampered With
- Per-User Authorisation and Audit Logging
Protecting Data: Disclosure, RAG and Poisoning
- Sensitive Information Disclosure and Redaction
- RAG Access Control and Vector Store Security
- Data Poisoning and Untrusted Knowledge Sources
Supply Chain, Abuse and Cost Attacks
- Models, Packages and Plugins as Dependencies
- Plugins, MCP Servers and Agent Tool Trust
- Unbounded Consumption: Rate Limits and Denial of Wallet
Testing, Monitoring and Response
- Red-Teaming and Security Regression Tests
- Monitoring, Detection and Logging
- Incident Response and Secure Development Habits