What a model does: predicting the next token

A large language model (LLM) is a program that reads a piece of text and guesses what comes next. The text is split into small chunks called tokens. For every possible next token, the model gives a probability.

To write a reply, it picks one token, adds it to the end and runs again. It repeats this hundreds of times. Chat replies, tool calls and JSON output all come from this same loop, just with different text going in.

The model has no memory between requests. A conversation only exists because your code sends the full history every time. Keep this picture in mind and the rest of the stage follows: cost grows with tokens, the context window is the only thing the model sees, and a fluent answer is a likely answer, not a checked one.