Checklist · AI
LLM Cost Optimization Checklist
Opens with creditsA checklist for cutting what you spend on LLM APIs without cutting quality. It covers measuring cost per request and per feature, routing work to smaller models, prompt caching, batch APIs, trimming context, limiting output, caching whole responses, retries and rate limits, and budgets with alerts, each with a severity and a concrete way to verify it.
Written by Tauseef Fayyaz8 chapters, 31 checksAbout 17 minReviewed 13 October 2026What you will learn
8 chapters · about 17 min
1Measuring costYou cannot cut what you cannot see. These checks give you cost per request, per feature and per user, from the usage data every API already returns.4 checks · 2 min
- 1Log the full token usage of every call, including cached and output tokens
- 2Tag every call with the feature, prompt version and user or tenant
- 3Track cost per successful task, not only cost per call
- 4Count tokens before shipping a prompt or model change, on the target model
4 in this chapterOpen this chapter2Model choice and routingThe biggest single lever on cost per token is which model you call. These checks make sure each task uses the smallest model that passes your evals.5 checks · 3 min
- 1Use the smallest model that passes your evals for each task
- 2Route requests by difficulty instead of sending everything to one model
- 3Pick a model per step in agents and pipelines, not one for the whole flow
- 4Set reasoning or thinking effort deliberately where the API offers it
- 5Re-run model comparisons when providers release new models or change prices
5 in this chapterOpen this chapter- Prompt cachingPrompt caching bills a repeated prompt prefix at a reduced rate. It only works if the prefix is really identical, so most of these checks are about keeping it stable.4 checks · 2 minOpens with credits
- Batch APIsWork that does not need an answer in seconds can usually run through a batch API at a lower price. These checks find that work and run it safely.3 checks · 2 minOpens with credits
- Trimming contextInput tokens are usually the bulk of the bill. These checks remove what the model does not need to see on each call.4 checks · 2 minOpens with credits
- Output limitsOutput tokens are often priced higher than input tokens on your provider's price list, and long answers are often worse answers. These checks keep output as short as the task allows.3 checks · 2 minOpens with credits
- Caching responses and avoiding callsThe cheapest call is the one you do not make. These checks reuse answers safely and keep deterministic work out of the model.3 checks · 2 minOpens with credits
- Retries, rate limits and budgetsRunaway loops and abuse cause the worst cost surprises. These checks cap spend per request, per user and per account, and tell you before the invoice does.5 checks · 2 minOpens with credits
About this checklist
LLM bills grow quietly. A prompt picks up a few more examples, a conversation history is never trimmed, an agent retries a failing tool ten times, and a nightly job runs on the same model as the chat feature. None of it shows up as an error, so the first sign is usually the invoice. This LLM cost optimization checklist is a set of checks that find where the money goes and cut it in order of payoff, while your evals confirm quality holds.
The arithmetic is simple. OpenAI's production guide puts it as cost being a function of the number of tokens and the cost per token. Every check here lowers one of those two: fewer input tokens, fewer output tokens, fewer calls, or a lower price per token through a smaller model, prompt caching or a batch API. The mechanisms come from the providers' own documentation, mainly Anthropic and OpenAI, and work the same way in principle on most APIs.
This checklist deliberately does not quote per-token prices. They change often and differ by model, so look them up on your provider's pricing page when you do the maths. What it does cover is how each mechanism works, what can quietly stop it working, and how to verify it is saving you money, for example by reading the cache fields in each response's usage data.
Each check has a severity (must, should or nice) and a concrete way to verify it. Start with the measurement chapter: without cost per request and per feature, you cannot tell which of the other checks is worth doing first.
Common questions
What is the fastest way to reduce LLM API costs?
Measure cost per feature first, then pick the largest item. The usual quick wins are moving offline jobs to a batch API, making prompt caching work by putting stable content first, trimming conversation history and retrieved context, and switching simple tasks to a smaller model that passes your evals.
How does prompt caching reduce cost?
When the start of your prompt is identical to a recent request, the provider can reuse its processing of that prefix and bill those tokens at a reduced rate. It only works if the prefix matches exactly and is above the minimum length, so put shared instructions, tools and documents first and check the cached token fields in each response.
Are batch APIs cheaper than normal API calls?
Yes. Anthropic's Message Batches API and OpenAI's Batch API both describe a 50% cost reduction compared with standard requests, in exchange for asynchronous processing within a 24-hour window. They suit evals, backfills, classification and other jobs that do not need an immediate answer.
Should I use a smaller model to save money?
For many tasks, yes, but decide with evals rather than guesses. Run the same eval set on two or three model sizes and compare quality, cost and latency. Use the cheapest model that meets the quality threshold for each task, and route only the hard requests to a larger model.
How do I stop one user from running up my LLM bill?
Enforce per-user and per-tenant request and token quotas on your server, cap input length, cap steps and tokens per agent task, and set spend limits per environment at the provider. OWASP calls this attack denial of wallet and lists rate limiting, quotas, timeouts and monitoring as mitigations.
Sources (8)
Checked against these, last on 13 October 2026.
- Prompt caching, Claude API docsplatform.claude.com/docs/en/build-with-claude/prompt-caching
- Batch processing, Claude API docsplatform.claude.com/docs/en/build-with-claude/batch-processing
- Token counting, Claude API docsplatform.claude.com/docs/en/build-with-claude/token-counting
- Rate limits, Claude API docsplatform.claude.com/docs/en/api/rate-limits
- Prompt caching, OpenAI API docsdevelopers.openai.com/api/docs/guides/prompt-caching
- Batch API, OpenAI API docsdevelopers.openai.com/api/docs/guides/batch
- Production best practices, OpenAI API docsdevelopers.openai.com/api/docs/guides/production-best-practices
- LLM10:2025 Unbounded Consumption, OWASP GenAI Security Projectgenai.owasp.org/llmrisk/llm102025-unbounded-consumption/
Related on Sefism
Picked from the other libraries by shared topics.
Roadmaps
See allCheat sheets
See allBest practices
See all- Best Practices for Building LLM AppsHow to build LLM apps and agents that hold up in production: prompts, evals, prompt injection, tool permissions, cost, logging and privacy.
- Prompt Engineering Best PracticesHow to write prompts that work the same way on the hundredth input as on the first: clear instructions, structure, examples, reasoning, output control and testing.
- Best Practices for AI Coding AssistantsHow to get reliable code from AI coding assistants and agents: instruction files, task prompts, verification, review, security, dependencies and session hygiene.
Checklists
See all- RAG Production ChecklistA RAG production checklist: chunking, embeddings, hybrid search, reranking, per-user permissions, citations, evals, re-indexing and cost, with how to verify each check.
- LLM Evaluation ChecklistAn LLM evaluation checklist: success criteria, eval datasets, code and model graders, judge calibration, agent evals, safety tests and production monitoring, with how to verify each check.
- AI Feature Launch ChecklistAn AI feature launch checklist: use case fit, evals, UX for errors, disclosure, human override, abuse filters, rate limits, cost caps, prompt privacy, monitoring and rollback.
Project structures
See all- GenAI RAG app (FastAPI) project structureFor developers turning a "chat with your documents" demo into an app people can rely on. A FastAPI backend where every layer has a home: ingestion workers, retrieval, prompts as versioned files, model clients with fallbacks and cost caps, guards, evals that gate CI, and tracing, so a wrong answer can be traced to the step that caused it.
- AI agent with an MCP server project structureFor developers who want to give AI assistants safe access to their own data or API. A TypeScript MCP server built on the official SDK, organised by feature (tasks, projects) with each feature owning its tools, resources and prompts, served over stdio for local hosts and Streamable HTTP with OAuth for remote ones, plus a small agent and the evals that prove a model can use it.
- Machine learning project project structureFor students and analysts whose model works in a notebook and now needs to be reproducible, reviewable, served and watched. It starts from the Cookiecutter Data Science v2 layout (immutable raw data, numbered notebooks, a real Python package) and grows into versioned data with DVC, typed parameters, data validation with pandera, experiment tracking and a model registry with MLflow, a FastAPI prediction service, drift monitoring and, for a team, scheduled retraining, model cards, fairness checks and shadow deployment.
Career questions
See all- Which AI tools do employers expect junior developers to know in 2026?Employers rarely require a specific brand. They expect you to use an AI coding assistant well (in the editor and in the terminal), to review and test what it produces, to call an LLM API in a small feature, and to explain when you would not use AI. Pick one assistant, learn it properly, and show judgement in your projects.
- How do I use AI to become a better engineer instead of becoming dependent on it?Decide the job AI does before you open it. Use it as a tutor, reviewer and sparring partner that explains and challenges, not a vending machine that hands you finished code. Attempt first, ask second, and always be able to explain and defend what you ship.
- How do I start learning AI agents as a software engineer?Start with one LLM API and plain code: a loop that calls the model, lets it pick a tool, runs the tool and feeds the result back. Build one small agent that does a real task end to end, add evaluation and guardrails, and only then reach for a framework. Agents are software engineering first.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
- 1:1 Mentorship
- Fresh Graduate Mentorship
- Career Guidance
Brand deals, promos and partnerships: tauseeffayyaz@gmail.com