AI security: secure and test LLM apps and agents

Stage 1 of 9 · 7 lessons · ~4 weeks

Security foundations for AI builders

The security ideas every AI security task stands on: threat modelling, trust boundaries, least privilege, secrets, the classic web weaknesses AI apps inherit, and the rules on what you may test.

Start: Threat modelling in four questions

Lessons· one lesson for each topic, in this order

  1. 1Threat modelling in four questions
  2. 2Trust boundaries and untrusted input
  3. 3Least privilege and who is asking
  4. 4Secrets: keys, tokens and where they live
  5. 5The web weaknesses AI apps inherit
  6. 6Permission, scope and the law
  7. 7Severity when failures are probabilistic

Do this

  • Draw a threat model diagram for one small web app and answer the four questions.
  • Mark every trust boundary on it with a dashed line.
  • Move every secret in your projects out of code and scan your repositories with Gitleaks.
  • Complete three web security labs on injection, XSS and access control.
  • Write your own one-paragraph rules of engagement for practising on apps you own.

Start with· follow one of these

Practise

  • Free labs on injection, XSS, access control and SSRF, the weaknesses AI apps inherit

  • A deliberately vulnerable web app to run locally and practise finding and fixing classic bugs

Keep handy

  • Short, reliable guides on threat modelling, secrets, output encoding and SSRF prevention

  • Scan your repositories for committed keys before you build anything with a model API

Also worth your time

Before moving on: A one-page note in your own words on threat modelling, trust boundaries, least privilege, secrets and the classic web weaknesses, plus a threat model diagram of one ordinary web app with its boundaries marked.

Tauseef's advice
Testers who skip the basics write weak reports. Before you test anything AI, make sure you can explain each classic web weakness, rate a finding honestly and write rules of engagement from memory. The Cybersecurity roadmap at /roadmaps/cybersecurity/ covers the general side in depth if you need it.

Mark this stage as not done?