What we protect, and the plan
The goals of LLM security (confidentiality, integrity, availability and safe behaviour), why LLM apps are different from ordinary software, the route from threat model to defences and red-teaming, and the rules of engagement for the labs.
#What we are protecting
| Goal | In LLM terms |
|---|---|
| Confidentiality | Prompts, documents, user data and secrets do not leak through outputs, logs or tools |
| Integrity | Attackers cannot change what the system does, by injecting instructions, poisoning data or tampering with models |
| Availability | One user cannot exhaust your GPUs or your budget |
| Safe behaviour | The system does not take harmful actions or produce harmful output |
#Why LLM apps are different
In ordinary software, code and data are separate: a database field cannot rewrite your program. In an LLM application, instructions and data travel in the same channel: text. A web page, an email or a document can contain text that the model treats as instructions. That one fact explains most of this course: prompt injection, data leakage through outputs, insecure handling of model output, and agents that do too much.
#The route
- 1Start hereplan, web basics, threat-modelling vocabulary and safe setup
- 2Threatsthreat model, prompt injection, leakage, output handling, agents and MCP, extraction and denial of wallet
- 3Defencesdefence in depth, privacy, supply chain, compliance
- 4Red teaminga lab and a methodology
- 5Capstonea security review of an LLM app you choose
| Part | Lessons | Time | Checkpoint |
|---|---|---|---|
| Start here | 3 | about 1.5 hours | You can draw a data-flow diagram and mark trust boundaries |
| Threats | 6 | about 6 hours | You can name the attack, the asset and the control for each class of threat |
| Defences | 4 | about 5 hours | You can design layered defences for a RAG or agent app |
| Red teaming | 2 | about 4 hours | You have attacked your own app and recorded findings |
| Capstone | 1 | about 5 hours | A written review with severity ratings and fixes |
#What you will be able to do
- Threat-model an LLM application and rank the risks.
- Recognise and test for prompt injection, data leakage and excessive agency.
- Apply layered defences without relying on the model to behave.
- Handle privacy, supply chain and compliance basics.
- Run a red-team exercise and write it up usefully.
#Prerequisites
- Basic web-application knowledge: HTTP, requests and responses, authentication. The next lesson refreshes it.
- Helpful: building applications (RAG, tools, agents) so you recognise the systems being attacked.
#Rules of engagement
- Only test what you own or are authorised to test. Use local models and your own demo apps for the labs.
- Do not test other people's chatbots or services for vulnerabilities.
- Never use real personal data in experiments.
- Disclose responsibly: if you find a real issue in someone else's system, report it privately to them.
- Defence is the point. The course explains attacks so you can build and test defences.
#Exercise
A company builds an assistant that reads customer emails and can send replies and create refunds. Name the untrusted input and the most privileged action.
Answer: the untrusted input is the email text (anyone can write it); the most privileged action is creating a refund. Any path from the first to the second without a hard check is a review point.
Quick quiz
+10 XP per correct answer, +25 XP for a perfect score
1. What makes LLM applications different for security?
2. Which is a correct rule of engagement for the labs?
3. What is the best first step in securing an LLM feature?
Found this useful?
Send it to someone who is learning the same thing.