Roadmap · Specialise
AI engineering
Opens with creditsFor developers who can already ship a web or backend app and want to build products on large language models. By the end you can build, evaluate and run an LLM feature with retrieval, tools and agents, and explain what it costs, how you measured it and where it fails.
Curated by Tauseef Fayyaz8 stages, 67 topics~12 months at a few hours a weekBy the end, you will be able to
- Estimate the token cost of a feature and decide what earns a place in the context window
- Build a RAG service over your own documents that cites its sources and refuses when retrieval finds nothing
- Measure every prompt, model or retrieval change with an eval suite that runs in CI
- Design tools and an MCP server a model uses reliably, and an agent that resumes after a crash
- Run a feature in production with tracing, guardrails, caching and model routing at a known cost per request
The path
8 stages, in the order to learn them. Open one to start.
1How LLMs work: tokens and contextWhat a model does when it answers: text becomes tokens and the model predicts one token at a time inside a fixed context window. Tokens are both your bill and your limit, so learn to decide what earns a place in the context.8 topics · ~3 weeks
- 1What a model does: predicting the next tokenA large language model (LLM) is a program that reads a piece of text and guesses what comes next.
- 2Tokens and tokenisationModels do not read letters or whole words.
- 3The context window as a budgetThe context window is the most tokens a model can handle in one request.
- 4Token pricing and rate limitsHosted models charge per million tokens.
- 5Temperature and samplingAt each step the model gives a probability for every possible next token, and sampling is how it picks one.
- 6Why models hallucinateA hallucination is a confident answer that is false.
- 7Reasoning models and effort settingsSome models think before they answer.
- 8Choosing a model: quality, cost, latencyThere is no best model, only the cheapest and fastest one that is good enough for your task.
8 in this stageOpen this stage2Prompting and model APIsTreat prompts as code and model calls as an unreliable network dependency. Be specific, fix the output format and show examples, then learn one provider's API properly: system prompts, streaming, retries and a local model for cheap experiments.9 topics · ~5 weeks
- 1The chat messages APIEvery LLM API call to a major provider has the same basic shape.
- 2Specific instructions and output formatMost weak outputs come from vague prompts, not weak models.
- 3System promptsThe system prompt holds the instructions that apply to the whole conversation.
- 4Few-shot examplesFew-shot examples are two to five sample inputs, each with its ideal output, placed in the prompt.
- 5Prompts in version controlA prompt is code: changing it changes how the product behaves.
- 6Streaming responsesA long reply can take 10 seconds or more to finish, but the first token often arrives in under a second.
- 7Timeouts, retries and backoffTreat a model API like any unreliable service across the network.
- 8Hosted vs local modelsHosted APIs give you the strongest models with no hardware to manage, but you pay per token and send your data to a provider.
- 9Open-weight models and where to run themOpen-weight models publish their trained weights, so you can download them, run them where you like and even fine-tune them.
9 in this stageOpen this stage- RAG: embeddings, chunking and retrievalGive a model your private data as context. Embed documents for semantic search, chunk them so you retrieve the passage rather than the whole file, combine vector and keyword search, and answer with citations or an honest refusal.11 topics · ~7 weeksOpens with credits
- Evals: measure before you add moreThe step everyone skips, and the one that separates a demo from something people use. Turn real failures into a test set, score every prompt, model or retrieval change against it, and include the attacks your app must survive.8 topics · ~5 weeksOpens with credits
- Tool calling, structured output and MCPLet the model act, not just talk. Design tools with clear names, parameters and errors, force outputs into a schema you validate, and expose your own systems through the Model Context Protocol so any MCP client can use them.10 topics · ~6 weeksOpens with credits
- Agents and durable executionA model in a loop that decides its own next step. Use one only when the steps cannot be known in advance, manage context across many steps, cap steps and cost, and make long runs resumable instead of restartable.7 topics · ~7 weeksOpens with credits
- Running AI systems in productionRunning an LLM feature for real users. Trace every model, retrieval and tool call, add guardrails on input and output, cut cost with caching and model routing behind a gateway, and design the system to survive traffic, outages and model changes.9 topics · ~2 monthsOpens with credits
- Put it together: ship a production GenAI appOne app that uses everything in this track, built the way a team would run it: clear layers, retrieval over real documents, a small tool loop, three guard layers, evals in CI, traces on every request, and a deploy with secrets, monitoring and cost limits.5 topics · ~3 monthsOpens with credits
6 more stages, 50 more topics, free with an account.
You have seen the open stages of the AI engineering roadmap. Every topic it covers is listed above, and an account opens the rest: what each topic means, a question to check yourself, the courses, the practice and Tauseef's advice for every stage. Early access is open: sign up and your account is active shortly after.
Everything here is free to learn from. A few things, like adding a roadmap, keeping a cheat sheet or asking Tauseef a question, use credits. You get 15 when you join, your first roadmap is free, and if you are learning seriously and run out, you can ask for more. Credits only make sure the platform goes to people who are actually learning.
Build a project
A finished project proves more than a certificate.
- Start with this oneintermediate1 semester · 2 studentsPrompt Change Regression Evals in Pull Requests with Statistical Diffs
Evals in CI: every prompt or model change runs the affected test sets before it merges
- TypeScript or Python
- GitHub Actions
- promptfoo
- SQLite or PostgreSQL
- LiteLLM
- advanced2 semesters · 2 to 3 studentsSentence-Level Groundedness Checker for RAG Answers with Human-Calibrated Thresholds
Evals for RAG at sentence level: flag answers that cite a source the sentence is not actually supported by
- Python
- Hugging Face Transformers
- An NLI model
- FastAPI
- PostgreSQL
Related on Sefism
Picked from the other libraries by shared topics.
Best practices
See all- Best Practices for Building LLM AppsHow to build LLM apps and agents that hold up in production: prompts, evals, prompt injection, tool permissions, cost, logging and privacy.
- Prompt Engineering Best PracticesHow to write prompts that work the same way on the hundredth input as on the first: clear instructions, structure, examples, reasoning, output control and testing.
- Best Practices for AI Coding AssistantsHow to get reliable code from AI coding assistants and agents: instruction files, task prompts, verification, review, security, dependencies and session hygiene.
Checklists
See all- RAG Production ChecklistA RAG production checklist: chunking, embeddings, hybrid search, reranking, per-user permissions, citations, evals, re-indexing and cost, with how to verify each check.
- LLM Evaluation ChecklistAn LLM evaluation checklist: success criteria, eval datasets, code and model graders, judge calibration, agent evals, safety tests and production monitoring, with how to verify each check.
- AI Feature Launch ChecklistAn AI feature launch checklist: use case fit, evals, UX for errors, disclosure, human override, abuse filters, rate limits, cost caps, prompt privacy, monitoring and rollback.
Project structures
See all- GenAI RAG app (FastAPI) project structureFor developers turning a "chat with your documents" demo into an app people can rely on. A FastAPI backend where every layer has a home: ingestion workers, retrieval, prompts as versioned files, model clients with fallbacks and cost caps, guards, evals that gate CI, and tracing, so a wrong answer can be traced to the step that caused it.
- AI agent with an MCP server project structureFor developers who want to give AI assistants safe access to their own data or API. A TypeScript MCP server built on the official SDK, organised by feature (tasks, projects) with each feature owning its tools, resources and prompts, served over stdio for local hosts and Streamable HTTP with OAuth for remote ones, plus a small agent and the evals that prove a model can use it.
- Machine learning project project structureFor students and analysts whose model works in a notebook and now needs to be reproducible, reviewable, served and watched. It starts from the Cookiecutter Data Science v2 layout (immutable raw data, numbered notebooks, a real Python package) and grows into versioned data with DVC, typed parameters, data validation with pandera, experiment tracking and a model registry with MLflow, a FastAPI prediction service, drift monitoring and, for a team, scheduled retraining, model cards, fairness checks and shadow deployment.
Career questions
See all- Which AI tools do employers expect junior developers to know in 2026?Employers rarely require a specific brand. They expect you to use an AI coding assistant well (in the editor and in the terminal), to review and test what it produces, to call an LLM API in a small feature, and to explain when you would not use AI. Pick one assistant, learn it properly, and show judgement in your projects.
- How do I use AI to become a better engineer instead of becoming dependent on it?Decide the job AI does before you open it. Use it as a tutor, reviewer and sparring partner that explains and challenges, not a vending machine that hands you finished code. Attempt first, ask second, and always be able to explain and defend what you ship.
- How do I start learning AI agents as a software engineer?Start with one LLM API and plain code: a loop that calls the model, lets it pick a tool, runs the tool and feeds the result back. Build one small agent that does a real task end to end, add evaluation and guardrails, and only then reach for a framework. Agents are software engineering first.
Cheat sheets
See allWork with me
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
- 1:1 Mentorship
- Fresh Graduate Mentorship
- Career Guidance
Brand deals, promos and partnerships: tauseeffayyaz@gmail.com