Roadmap · Build products
Choosing an LLM and AI stack
Opens with creditsFor developers, students and founders who are about to build something with a large language model and want to pick the model, hosting and tools on evidence rather than hype. By the end you can turn a task into requirements, compare three models on your own data, work out the cost per task, decide between an API and an open-weight model, and pick each layer of the stack with a reason you can defend.
Curated by Tauseef Fayyaz9 stages, 54 topics~4 months at a few hours a weekBy the end, you will be able to
- Turn a product idea into model requirements: quality bar, speed, volume, context, tools and privacy
- Decide when a reasoning model or a bigger model pays for itself, and when a small fast one is enough
- Work out the cost per task, including output, reasoning, caching and batch discounts, before you build
- Choose between a long context window, chat history tricks and retrieval for the data your feature needs
- Judge a model on tool calling, structured output and MCP support for the agent you want to build
- Decide between a cloud API and an open-weight model on your own servers, with privacy and compliance in view
- Pick hosting, embeddings, document extraction, a framework (or none) and a vector database with clear reasons
- Run a small comparison of three models on a golden test set, and re-run it every quarter in production
The path
9 stages, in the order to learn them. Open one to start.
1Start from the task, not the modelThere is no best model, only a best model for a job. Describe the job first: what kind of task it is, how good the answer must be, how fast, how often and with what data. Then use leaderboards to build a shortlist, not to make the decision.6 topics · ~1 week
- 1Five kinds of taskMost LLM features fall into five families, and each one stresses a model differently.
- 2Write the requirements before you look at modelsBefore opening any leaderboard, write six lines about your feature.
- 3The cheapest model that is good enoughThe goal is not the smartest model.
- 4Model families and tiersMost providers sell a family in three or four sizes.
- 5Leaderboards: a shortlist, not a verdictPublic leaderboards help you build a shortlist quickly.
- 6Model facts go stale: date everythingModel names, prices and context windows change every few weeks.
6 in this stageOpen this stage2Reasoning quality: when a bigger model pays offSome tasks need a model that thinks before it answers, and some only need a quick, obedient one. Learn what reasoning models and effort settings do, why multi-step tasks punish small mistakes, and how to tell when the price of a bigger model is cheaper than the price of a wrong answer.6 topics · ~2 weeks
- 1Reasoning models and fast modelsA reasoning model writes hidden working (reasoning tokens) before its final answer, a bit like a student using scrap paper.
- 2Effort and thinking settingsMost providers now let you control how much a model thinks.
- 3Why multi-step tasks punish small errorsIn a task with many steps, small error rates multiply.
- 4Instruction followingInstruction following is how faithfully a model does exactly what you asked: the format, the length, the tone, the things it must never do.
- 5When a bigger model pays for itselfA bigger model pays off when the cost of a wrong answer is higher than the extra price.
- 6What benchmarks can and cannot tell youBenchmarks are fixed test sets, such as maths problems, coding tasks or exam questions, with a published score per model.
6 in this stageOpen this stage- Speed and cost per taskPrices are quoted per million tokens, but what you pay is per task. Learn how tokens, input and output prices, latency, small models, batch discounts and caching combine, so you can work out a feature's monthly bill before you write it.7 topics · ~2 weeksOpens with credits
- Context windows and memoryA context window is how much a model can read at once. Learn what fits, why a full window is slow and expensive, how to handle long chat histories, and when retrieval (RAG) beats simply paying for a bigger window.5 topics · ~1 weekOpens with credits
- Tools, structured output and agentsIf your feature calls functions, returns JSON or runs as an agent, the model's tool skills matter as much as its intelligence. Learn what to check: function calling, structured output, MCP support and how reliably a model finishes long multi-step jobs.5 topics · ~2 weeksOpens with credits
- Privacy and deployment: API or your own serversWhere the model runs decides who can see your data, what it costs at scale and which rules you must follow. Compare cloud APIs with open-weight models on your own servers, understand licences, data residency and compliance, and choose with the facts on one page.6 topics · ~2 weeksOpens with credits
- The rest of the stackA model is one layer. Around it sit hosting, an embeddings model, tools to get text out of PDFs and websites, maybe a framework, a vector database and observability. Learn what each layer does and how to pick a sensible default for each.7 topics · ~2 weeksOpens with credits
- Run your own comparisonThis is where the decision is made. Build a small golden test set from your real data, agree a scoring rubric, run three models on it, and read the results with cost and latency next to quality.6 topics · ~2 weeksOpens with credits
- Production choices: routing, fallbacks and lock-inChoosing a model is not a one-off. In production you route requests between cheap and strong models, keep fallbacks for outages, avoid getting locked to one provider, and re-run your comparison every quarter as models and prices change.6 topics · ~2 weeksOpens with credits
7 more stages, 42 more topics, free with an account.
You have seen the open stages of the Choosing an LLM and AI stack roadmap. Every topic it covers is listed above, and an account opens the rest: what each topic means, a question to check yourself, the courses, the practice and Tauseef's advice for every stage. Early access is open: sign up and your account is active shortly after.
Everything here is free to learn from. A few things, like adding a roadmap, keeping a cheat sheet or asking Tauseef a question, use credits. You get 15 when you join, your first roadmap is free, and if you are learning seriously and run out, you can ask for more. Credits only make sure the platform goes to people who are actually learning.
Build a project
A finished project proves more than a certificate.
- Start with this oneintermediate1 semester · 2 to 3 studentsUrdu and English Personal Data Redaction Before Prompts Reach Cloud AI Tools
The privacy stage made real: strip personal data before it reaches a cloud API
- Python
- Microsoft Presidio
- Hugging Face Transformers
- A multilingual NER model
- FastAPI
- intermediate1 semester · 2 studentsWhich LLM Understands Roman Urdu and Bangla Business Messages? A Three-Model Evaluation Harness
Run your own comparison on messy local-language messages, where public leaderboards say little
- Python
- LiteLLM
- Ollama
- pandas
- DuckDB
- advanced2 semesters · 2 studentsOn-Device LLM Benchmark App for Budget Android Phones with Urdu Quality Tests
Measure small open models on real hardware: speed, memory and quality side by side
- Kotlin
- Jetpack Compose
- llama.cpp Android bindings
- MediaPipe LLM Inference or LiteRT
- Python for analysis
- advanced2 semesters · 2 studentsLearned LLM Router That Sends Easy Tasks to Cheap Models, with a Cost-Quality Frontier
The routing stage done with evidence: learn which requests a cheap model can handle and plot the cost and quality trade-off
- Python
- LiteLLM
- scikit-learn
- Sentence embeddings
- PostgreSQL
Related on Sefism
Picked from the other libraries by shared topics.
Best practices
See all- Best Practices for AI Coding AssistantsHow to get reliable code from AI coding assistants and agents: instruction files, task prompts, verification, review, security, dependencies and session hygiene.
- Best Practices for Building LLM AppsHow to build LLM apps and agents that hold up in production: prompts, evals, prompt injection, tool permissions, cost, logging and privacy.
- Prompt Engineering Best PracticesHow to write prompts that work the same way on the hundredth input as on the first: clear instructions, structure, examples, reasoning, output control and testing.
Checklists
See all- RAG Production ChecklistA RAG production checklist: chunking, embeddings, hybrid search, reranking, per-user permissions, citations, evals, re-indexing and cost, with how to verify each check.
- LLM Evaluation ChecklistAn LLM evaluation checklist: success criteria, eval datasets, code and model graders, judge calibration, agent evals, safety tests and production monitoring, with how to verify each check.
- AI Feature Launch ChecklistAn AI feature launch checklist: use case fit, evals, UX for errors, disclosure, human override, abuse filters, rate limits, cost caps, prompt privacy, monitoring and rollback.
Project structures
See all- GenAI RAG app (FastAPI) project structureFor developers turning a "chat with your documents" demo into an app people can rely on. A FastAPI backend where every layer has a home: ingestion workers, retrieval, prompts as versioned files, model clients with fallbacks and cost caps, guards, evals that gate CI, and tracing, so a wrong answer can be traced to the step that caused it.
- AI agent with an MCP server project structureFor developers who want to give AI assistants safe access to their own data or API. A TypeScript MCP server built on the official SDK, organised by feature (tasks, projects) with each feature owning its tools, resources and prompts, served over stdio for local hosts and Streamable HTTP with OAuth for remote ones, plus a small agent and the evals that prove a model can use it.
- Machine learning project project structureFor students and analysts whose model works in a notebook and now needs to be reproducible, reviewable, served and watched. It starts from the Cookiecutter Data Science v2 layout (immutable raw data, numbered notebooks, a real Python package) and grows into versioned data with DVC, typed parameters, data validation with pandera, experiment tracking and a model registry with MLflow, a FastAPI prediction service, drift monitoring and, for a team, scheduled retraining, model cards, fairness checks and shadow deployment.
Career questions
See all- Which AI tools do employers expect junior developers to know in 2026?Employers rarely require a specific brand. They expect you to use an AI coding assistant well (in the editor and in the terminal), to review and test what it produces, to call an LLM API in a small feature, and to explain when you would not use AI. Pick one assistant, learn it properly, and show judgement in your projects.
- Is it bad that I rely on ChatGPT or Cursor and cannot write code from scratch?Using AI tools is fine and increasingly expected. Being unable to write, debug or explain code without them is a real risk, because interviews, reviews and production incidents test exactly that. Keep the tools, but rebuild the ability to think and code independently first.
- How do I use AI to become a better engineer instead of becoming dependent on it?Decide the job AI does before you open it. Use it as a tutor, reviewer and sparring partner that explains and challenges, not a vending machine that hands you finished code. Attempt first, ask second, and always be able to explain and defend what you ship.
Cheat sheets
See all- Coding with AI tools cheat sheetClaude Code, Cursor, Copilot, used well.
- Prompt Engineering cheat sheetPrompt anatomy, few-shot examples, chain of thought, structured output, long context, prompts for code and study, and testing prompts.
- AI engineering cheat sheetBuild products on LLMs and generative AI.
Work with me
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
- 1:1 Mentorship
- Fresh Graduate Mentorship
- Career Guidance
Brand deals, promos and partnerships: tauseeffayyaz@gmail.com