The AI Engineering Words You Keep Hearing, Explained Properly

Why the vocabulary is the hard part

The concepts underneath AI engineering are mostly not difficult. What makes the field feel impenetrable is that every explanation assumes you already know eight other terms, and the definitions you find are either marketing copy or a research paper.

So here are the ones that keep coming up, in plain language, with what each is actually for and where each one lets you down.

Token

A token is a chunk of text, usually a word or part of a word. Models do not read characters, they read tokens.

You care about this for three practical reasons. It is the unit you are billed in. It is the unit context limits are measured in. And it explains a whole class of strange behaviour, like a model miscounting the letters in a word, because it never saw the letters.

Rough rule for English: a token is about four characters, so 1,000 tokens is roughly 750 words.

Large language model

A model trained on an enormous amount of text to predict what comes next. Everything else it appears to do is built on that one capability.

What it is good for: drafting text, writing and explaining code, answering questions, transforming data from one format to another.

Where it lets you down: it will state something false with exactly the same confidence as something true. There is no tone shift to warn you. This is not a bug about to be fixed, and your habits should assume it permanently.

Prompt engineering

Writing the instruction well. Being specific about the task, the format you want back, and the constraints that matter.

What it is good for: it is free, it takes minutes, and it fixes more problems than people expect. Try this before anything more elaborate.

Where it lets you down: carefully tuned prompts can behave differently on a new model version, so anything you depend on needs a test rather than a hope.

Context engineering

The bigger idea that prompt engineering sits inside. Not just what you ask, but everything the model can see when it answers: the documents you retrieved, the conversation so far, the tools you exposed, the examples you included.

The context window is finite and everything in it competes. Filling it with marginally relevant material makes the answer worse, not better.

Once your system is more than a single question and answer, this is where nearly all of your quality comes from. It is also the least standardised part of the field, so expect to build your own tooling.

Embeddings

A model converts a piece of text into a list of numbers positioned so that things with similar meaning end up near each other.

What it is good for: searching by meaning rather than by keyword. A search for "how do I cancel" can find a document titled "ending your subscription", which keyword search would miss entirely.

Where it lets you down: the numbers are not interpretable. When retrieval returns something irrelevant, there is no line of reasoning to inspect, and debugging becomes empirical.

Vector database

Somewhere to store those number lists and search them quickly.

Exact search means comparing your query against every stored item, which does not scale. So these databases use approximate search: fast, and it occasionally misses a genuinely good match. That trade off is adjustable and worth understanding before you tune anything.

RAG

Retrieval augmented generation. Before answering, look things up in your own documents, put what you found into the context, and answer from that.

What it is good for: questions about information the model was never trained on, meaning your documentation, your policies, your data. You can update the documents at any time and the answers change immediately, with no retraining.

Where it lets you down: if retrieval fetches the wrong thing, the model will answer confidently from the wrong thing. Most RAG systems that give bad answers have a retrieval problem, not a model problem, and people spend weeks tuning prompts before checking.

Reranking

A second pass over retrieved results, using a slower and more accurate model, to reorder them before they reach the context window.

Cheap to add, and it often improves answer quality more than swapping to a larger model does. Worth trying early.

Fine tuning

Continuing to train a model on your own examples so it behaves a particular way by default.

What it is good for: consistent format, a specific tone, a narrow task done the same way every time. It changes behaviour more reliably than instructions do.

Where it lets you down: it teaches behaviour, not facts. If your goal is for the model to know your latest pricing, fine tuning is the wrong tool and RAG is the right one. It also costs real money and can make the model worse at everything you did not train it on.

Structured output

Constraining the model to return data in a schema you defined, so your code can parse it reliably instead of hoping the JSON is well formed.

If you are extracting data rather than producing prose, use this. It removes an entire category of parsing bugs.

Tool calling

Giving the model a set of functions it can request, so it can look something up or take an action rather than only producing text.

Where it lets you down: models fail at tools far more often than at reasoning, and it is usually the tool's fault. A vague parameter name or an unclear description produces wrong calls all day. Treat tool descriptions as the interface documentation they are.

MCP

Model Context Protocol. A standard way to expose tools and data to any AI application, so you write the integration once instead of once per client.

Still young. The permission and security story in particular is not mature, so think carefully before exposing anything that can write.

AI agent

A model that decides its own steps: plan, call a tool, look at the result, decide what to do next, repeat until done.

What it is good for: tasks where the steps depend on what you find along the way, which a fixed script cannot express.

Where it lets you down: errors compound. A small mistake at step two becomes a confidently wrong answer at step nine. Reliability drops sharply with the number of steps, so the useful skill is knowing when a plain script would have worked.

Evals

Testing, for AI systems. A set of inputs, criteria for a good output, and a score you can track over time.

This is the least exciting item here and the one that separates systems that improve from systems that merely change. Without evals you cannot tell whether last week's prompt edit helped, and you will end up making changes based on vibes.

Build these early, even a crude version with twenty examples in a spreadsheet.

Guardrails

Checks around the model that block unsafe, malicious or out of scope input and output.

Necessary once real users are involved, and not sufficient on its own. Treat them as one layer, not as the security model.

Observability

Tracing what actually happened: which documents were retrieved, what the prompt looked like after assembly, which tools were called, how long each step took and what it cost.

The first thing to add when something behaves strangely, and the thing most people add last. Without it you are debugging by re-running and guessing.

What to learn in what order

If you are starting: tokens, prompting, then RAG, because RAG is where most real applications actually live.

After that: evals, before anything else, because everything past this point involves changes you need to be able to measure.

Then: tool calling, structured output, and observability. That set covers the majority of production systems.

Agents, fine tuning and multi agent architectures come last, not because they are the hardest to understand, but because they are the easiest to reach for when something simpler would have done the job.