Roadmap · Specialise
AI security: secure and test LLM apps and agents
Opens with creditsFor developers and security learners who want to secure LLM applications and AI agents, or to test them as a red-teamer. By the end you can threat model an AI feature, defend it in layers against prompt injection, lock down agents, tools and MCP servers, check the data and model supply chain, test it within clear rules, watch it in production and show all of it in a portfolio.
Curated by Tauseef Fayyaz9 stages, 66 topics~11 months at a few hours a weekBy the end, you will be able to
- Threat model an LLM app or agent: trust boundaries, data flows, tools and the worst a fooled model could do
- Explain each OWASP Top 10 for LLM Applications risk, the agentic risks and MITRE ATLAS techniques in plain words, with the defences that work
- Build layered defences against prompt injection: input and content guards, output guards, privilege separation and human approval
- Secure agents, tools and MCP servers with narrow permissions, proper token handling, sandboxes and egress control
- Check the data and model supply chain: poisoning, unsafe model files, provenance and leakage of data and system prompts
- Test AI systems within written rules of engagement using red-teaming, security evals and tools such as garak, PyRIT, promptfoo and Inspect, and report findings clearly
- Monitor an AI app in production, respond to AI incidents and turn each one into a regression test
The path
9 stages, in the order to learn them. Open one to start.
1Security foundations for AI buildersThe security ideas every AI security task stands on: threat modelling, trust boundaries, least privilege, secrets, the classic web weaknesses AI apps inherit, and the rules on what you may test.7 topics · ~4 weeks
- 1Threat modelling in four questionsThreat modelling is a structured way to find what could go wrong in a system before an attacker does.
- 2Trust boundaries and untrusted inputA trust boundary is a line in a system where data or control passes between parts that deserve different levels of trust: the internet and your server, your…
- 3Least privilege and who is askingAuthentication answers "who are you?" and authorisation answers "what may you do?".
- 4Secrets: keys, tokens and where they liveA secret is anything that grants access: model API keys, database passwords, OAuth tokens, signing keys.
- 5The web weaknesses AI apps inheritMost LLM apps are web apps, so the classic weaknesses still apply, and several come back through the model.
- 6Permission, scope and the lawIn security, testing a system you do not own without written permission is unauthorised access in most countries, including Pakistan, India, the UK and the…
- 7Secure defaults and layersDefence in depth means stacking independent layers so one failure does not become a breach.
7 in this stageOpen this stage2How LLM apps and agents work, and where they breakFollow a request through an LLM app and an agent: how the context window is assembled, why instructions and data mix, where retrieval and tools add risk, why behaviour varies run to run, and the practice app you will defend and test for the rest of the roadmap.7 topics · ~5 weeks
- 1How a request becomes a context windowWhen a user sends a message, the app builds the text the model will read: the system prompt (the developer's instructions), earlier conversation, retrieved…
- 2Instructions and data share one channelIn a normal program, code and data are kept apart: a database knows which part of a query is the command and which part is a value.
- 3Where retrieval goes wrongRetrieval-augmented generation (RAG) splits documents into chunks, turns them into embeddings (lists of numbers that capture meaning), stores them in a vector…
- 4The agent loop as an attack surfaceAn agent runs a loop: the model reads the context, asks for a tool call, your code runs it, the result goes back into the context, and the loop repeats until…
- 5Probabilistic behaviour and what it means for securityThe same prompt can produce different answers on different runs, because models sample from probabilities, and providers update models over time.
- 6Private data, untrusted content, a way outSimon Willison named three capabilities that, together, let a prompt injection steal data.
- 7Build the app you will defend and testThe best practice target is an app you built and own.
7 in this stageOpen this stage- Risk maps: the OWASP lists and MITRE ATLASThe shared vocabularies of AI security: the OWASP Top 10 for LLM Applications (2025 edition), the OWASP Top 10 for Agentic Applications, MITRE ATLAS, and the NIST and Google maps, used to threat model a real feature.7 topics · ~4 weeksOpens with credits
- Prompt injection and layered defencesPrompt injection as a concept, direct and indirect, and why no single control fixes it; then the layers that reduce it: input and content guards, output guards, privilege separation and human approval. No payloads.7 topics · ~6 weeksOpens with credits
- Agent, tool and MCP securitySecuring what agents can do: narrow tool permissions, the confused deputy problem, trusting and authorising MCP servers, token and scope rules, sandboxes for code, egress control, and memory and state.8 topics · ~6 weeksOpens with credits
- Data and model supply chainThe risks that arrive before a user types anything: poisoned training and retrieval data, model files that run code, provenance and signing, an AI bill of materials, leakage of sensitive data and system prompts, model theft and hallucinated packages.8 topics · ~5 weeksOpens with credits
- Testing: red-teaming and safety evalsHow AI systems are tested responsibly: rules of engagement, red-teaming versus evals, a test plan from the risk list, garak, PyRIT, promptfoo and Inspect by purpose, scoring results, writing findings and AI bug bounty programmes.8 topics · ~7 weeksOpens with credits
- Monitoring and incident response for AI appsRunning an AI app safely after launch: what to log for every model and tool call, keeping logs private, signals worth an alert, kill switches, an AI incident playbook, cost abuse, and turning incidents into tests.7 topics · ~4 weeksOpens with credits
- Career: roles, certifications and a first jobWhere AI security work actually sits, which certifications are worth it and when, a portfolio that proves your path, open source contributions, responsible disclosure, keeping up and a realistic route to a first role.7 topics · ~5 weeksOpens with credits
7 more stages, 52 more topics, free with an account.
You have seen the open stages of the AI security roadmap. Every topic it covers is listed above, and an account opens the rest: what each topic means, a question to check yourself, the courses, the practice and Tauseef's advice for every stage. Early access is open: sign up and your account is active shortly after.
Everything here is free to learn from. A few things, like adding a roadmap, keeping a cheat sheet or asking Tauseef a question, use credits. You get 15 when you join, your first roadmap is free, and if you are learning seriously and run out, you can ask for more. Credits only make sure the platform goes to people who are actually learning.
Build a project
A finished project proves more than a certificate.
- Start with this onebeginner1 semester · 1 to 2 studentsHallucinated Package Guard for AI Coding Assistants (Slopsquatting Defence)
Supply chain defence for AI-written code: check every new dependency before a made-up package name gets installed
- Python or TypeScript
- npm and PyPI registry APIs
- GitHub Actions
- pre-commit
- SQLite
- intermediate2 semesters · 2 studentsGuarded Text-to-SQL Assistant for Small Business Sales Data
A real LLM feature where output handling, least privilege and permission checks decide whether it is safe
- Python
- LLM API or small open model
- sqlglot
- PostgreSQL
- DuckDB
- intermediate1 semester · 2 to 3 studentsUrdu and English Personal Data Redaction Before Prompts Reach Cloud AI Tools
An input guard that keeps personal data out of hosted models, with measured catches and false blocks
- Python
- Microsoft Presidio
- Hugging Face Transformers
- A multilingual NER model
- FastAPI
- intermediate1 semester · 2 studentsModel File Supply Chain Scanner for Open Model Downloads with an AI Bill of Materials
Covers the AI supply chain: unsafe model formats, provenance, licences and an AI bill of materials
- Python
- pickletools and the safetensors library
- Hugging Face Hub API
- SQLite or PostgreSQL
- Typer for the command line
Related on Sefism
Picked from the other libraries by shared topics.
Best practices
See all- Threat Modeling Best PracticesMake threat modeling a routine part of building features: the four questions, data flow diagrams, STRIDE, abuse cases, tickets and tests, and LLM features.
- Secrets Management Best PracticesKeep API keys, passwords and tokens out of git, logs and browser bundles: secret managers, scanning, rotation, OIDC in CI and leak response.
- Authentication Best PracticesBuild sign-in that holds up: password rules from NIST, Argon2id hashing, login throttling, MFA and passkeys, secure sessions and safe account recovery.
Checklists
See all- Web Application Security ChecklistA practical web application security checklist built on the OWASP Top 10:2025 and ASVS 5.0, with a severity and a way to verify every check.
- API Security ChecklistCheck your REST or GraphQL API against the OWASP API Security Top 10: authentication, object authorization, rate limits, input validation, SSRF, CORS and logging.
- Linux Server Hardening ChecklistHarden an Ubuntu, Debian or RHEL server step by step: accounts and sudo, SSH, updates, firewall and open ports, permissions, logging and CIS benchmark scans.
Project structures
See all- AI agent with an MCP server project structureFor developers who want to give AI assistants safe access to their own data or API. A TypeScript MCP server built on the official SDK, organised by feature (tasks, projects) with each feature owning its tools, resources and prompts, served over stdio for local hosts and Streamable HTTP with OAuth for remote ones, plus a small agent and the evals that prove a model can use it.
- Firebase (Cloud Functions and security rules) project structureFor students building an app on Firebase who need a real backend: Cloud Functions (2nd gen) in TypeScript, Firestore and Storage security rules with tests, and the Local Emulator Suite. The tree starts from
firebase initand grows into feature folders of callables, triggers and scheduled jobs, then adds CI deploys, environments, contract tests and runbooks. - Terraform project structureAn infrastructure repository for a small app on AWS: reusable modules, one folder per environment with its own remote state, pinned providers, tests and security scans, with plan on every pull request and apply from CI. For students who have finished the Terraform tutorials and want to know how a real team lays out the files. The same layout works unchanged with OpenTofu.
Career questions
See all- How do I get into AI security as a student or junior engineer?Build a normal security or software foundation first, then learn how LLM apps fail using the OWASP Top 10 for LLM Applications and MITRE ATLAS, and prove it by building, testing and hardening a small LLM app of your own. Most people reach AI security through AppSec, security engineering or ML engineering, not straight from university.
- Can I use AI coding tools at my internship, and what should I never paste into them?Usually yes, but only the tools your company has approved, on the code and data they allow, and in the way they allow. Ask in your first week. Never paste secrets, customer data or code you are not permitted to share, and review everything the tool writes as if your name is on it, because it is.
- Where do I start if I want a career in cybersecurity or IT operations?Start with the foundations both fields share: networking, Linux, and how systems fail. Then choose an entry role such as SOC analyst, IT support or junior cloud operations, and prove it with hands-on labs and one documented project rather than a stack of certificates.
Cheat sheets
See allWork with me
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
- 1:1 Mentorship
- Fresh Graduate Mentorship
- Career Guidance
Brand deals, promos and partnerships: tauseeffayyaz@gmail.com