Defense in depth for LLM apps

No single control stops every attack. Defense in depth stacks independent layers so that when one fails (and each will sometimes fail), another catches the problem. The goal is not a perfect wall but a system where no single failure is catastrophic, and where an attacker must defeat several different kinds of control at once.

#The layers

1. Input controls. Authentication, rate limits, size limits, file-type checks, and optional injection or abuse classifiers.
2. Context construction. Include only data the user is authorised to see; delimit untrusted text; keep secrets out.
3. Model-level guardrails. A well-chosen system prompt, a safety-tuned model, and policy classifiers (for example Llama Guard-style models run locally) to screen prompts and outputs.
4. Output validation. Schema checks, escaping, URL allowlists, PII and secret scanners.
5. Capability limits. Least-privilege tools, server-side authorisation, sandboxes, confirmation for consequential actions.
6. Monitoring and response. Logging of tool calls, anomaly alerts, kill switches, and an incident process.

Each layer in a sentence of detail:

#Which layers are deterministic?

Put your trust in the deterministic layers and treat probabilistic ones as a way to reduce volume and add friction. A useful test of any control: would it still work if the model were fully controlled by the attacker? If yes, it is a real boundary. If no, it is a speed bump.

#Why layers multiply, and when they do not

If three layers each independently stop 90 percent of attempts, the chance an attempt passes all three is 0.1 × 0.1 × 0.1 = 0.1%. That is the Swiss-cheese idea: holes rarely line up.

The catch is independence. Two layers that fail for the same reason (both are prompts, both are the same classifier family, both rely on the model noticing something) fail together. A hardened system prompt plus a prompt-based guard model that uses the same base model are highly correlated. Mix different kinds of controls: a deterministic permission check, a schema validator, a rate limit and a classifier fail in unrelated ways.

#Mapping attacks to layers

Use a table like this in your design review. It shows which attacks have only one layer of protection (a red flag):

Rows with a single tick need another layer. Try it live in the diagram above: switch layers off and see which attacks get through.

#Guardrail models

Open classifier models can screen both inputs and outputs for categories such as violence, self-harm, illegal activity and prompt-injection patterns. They:

• add latency and cost (an extra model call),
• produce false positives (blocking legitimate users) and false negatives,
• need evaluating on your traffic, since categories and tone differ by domain,
• are themselves attackable (an injection that targets the classifier).

Tips: run them in parallel with the main call where possible, use a small fast model, tune thresholds with data, log what they block, and treat them as one layer among several. For structured, narrow apps, rules and schemas often beat classifiers: if the output must be one of five labels, enforce that with a grammar and skip the classifier (structured output).

#Fail safely

Decide what happens when a layer fails or is unavailable:

Write the decision down per layer. The worst outcome is the accidental one: a classifier timeout silently returning "safe".

#Kill switches

Every feature with side effects needs a way to turn it off fast: a feature flag that disables a tool, a switch that forces confirmation for all actions, a way to revoke a model release. Test the switch before you need it, and make sure on-call knows where it is (reliability patterns).

#Security testing as a layer

Include adversarial test cases (injection, leakage, abuse) in your release evaluations, track the success rate over time, and re-run them whenever the model, prompt or tools change. Test layers individually (does the output scanner catch a planted canary? does the authoriser reject a cross-user ID?) and together (what is the end-to-end attack success rate?). The red-team lab builds a small harness, and CI shows how to run it on every change.

#A worked example: a support assistant

Seven concerns, each with a primary deterministic control, and probabilistic layers on top.

#Practical order of work

1. Remove secrets from prompts and set up authorisation on tools and data (largest risk reduction)
2. Limit tools and add confirmations
3. Validate and escape output
4. Add rate limits and size caps
5. Add monitoring and logging with redaction
6. Add classifiers and prompt hardening on top

Items 1 to 4 are mostly ordinary engineering and give most of the protection. Items 5 and 6 add detection and polish.

#Common mistakes

1. Counting probabilistic layers as boundaries.
2. Stacking correlated layers (three prompts, one model) and calling it depth.
3. A guardrail with no false-positive measurement, which annoys users into disabling it.
4. No kill switch.
5. Defending only the input, ignoring outputs and tools.
6. No tests, so layers silently decay as prompts and models change.

#Exercise

An internal assistant summarises uploaded contracts and can email the summary to colleagues. Today the only defence is a system prompt saying "never send confidential data outside the company".

1. Which kind of control is that, and how reliable?
2. Propose four layered controls.

Answers:

1. A probabilistic, model-level control: low to medium reliability, and it can be bypassed by an instruction hidden in a contract.
2. (a) Capability: restrict recipients to company-domain addresses in code and require confirmation showing recipient and body. (b) Context: process contracts in a tool-less call producing a schema-validated summary. (c) Output: scan the summary for secrets and external URLs. (d) Monitoring: log every send, alert on unusual volume or new recipients, and provide a kill switch for the email tool.