Six Ways to Structure Work for an AI Agent

Agent is doing a lot of work as a word

People use it for a chatbot with one tool attached and for a system that runs for twenty minutes making its own decisions. Those are not the same thing and they do not fail the same way.

Underneath, almost everything being built falls into six shapes. Knowing which one you are building tells you what will go wrong and what you need to handle.

There is a thread running through all six, so it is worth stating once at the top: every one of these can fail halfway through. A model call times out, an API returns a 500, a step loops. What separates a demonstration from something you can leave running is whether that partial failure is handled or whether the whole job simply dies with half its work done.

One: routing

Look at what came in, decide which model, prompt or path should handle it, send it there.

A support system that sends billing questions one way, technical problems another, and anything it cannot classify to a human. A cost saving layer that sends easy requests to a small model and hard ones to a large one.

Why it helps: each path can be tuned for its own job instead of one prompt trying to be adequate at everything.

What goes wrong: the classifier itself is wrong sometimes, and a misrouted request gets confidently handled by the wrong specialist. You need a fallback path for low confidence, and you need to look at what is being routed where, because misclassification is invisible unless you check.

Two: parallelisation

Run several known tasks at once, then combine the results.

Checking a document against five policies at the same time. Asking three models the same question and taking the majority answer. Summarising twenty files simultaneously rather than in sequence.

Why it helps: it is faster, obviously, and running the same question several ways gives you a confidence signal you would not otherwise have.

What goes wrong: one branch fails and now you have four results out of five. Decide in advance whether that is an answer or an error. Retrying only the failed branch, rather than the whole batch, is the difference between a two second recovery and starting over.

Three: orchestrator and workers

One model breaks a task into subtasks, hands each to a worker, then assembles what comes back.

Research across several sources. A code change touching multiple files. Anything where the number of pieces is not known until you look.

Why it helps: it handles tasks where you cannot write the subtask list in advance, which is the honest reason to use it.

What goes wrong: this is the most expensive pattern here and the most over used. Workers duplicate each other's effort. Token cost grows faster than quality does. A worker failing at step six leaves the job half done, and without checkpoints the recovery is to run all six again.

Before choosing this, try one model with good tools. It is frequently enough, and it is an order of magnitude simpler to debug.

Four: tool use

The model decides which function to call, calls it, reads the result, continues.

This is the foundation the others sit on rather than a competitor to them.

Why it helps: it connects the model to real, current information and real actions.

What goes wrong: almost always the tool, not the model. Two things to get right:

Descriptions are the interface. The model has your name, your parameters and your description text, and nothing else. No colleague to ask. Write them accordingly.

Retries need idempotency. A tool call that times out may or may not have run. If the retry can charge a card twice, you have not built a retry, you have built a bug. Give operations an idempotency key so a repeat is safe.

Five: reflection

The model produces something, reviews its own output, revises it.

Drafting then critiquing then rewriting. Generating code, running the tests, fixing what failed.

Why it helps: the second pass genuinely catches things the first missed, and when there is an objective signal to react to, like a failing test, it works well.

What goes wrong: without a stopping condition it will keep going, and without an objective signal it converges on agreeing with itself. Set a hard iteration limit. Save the best version so far, because revision four is sometimes worse than revision two and you need to be able to keep the good one.

Six: planning

The model writes out a sequence of steps, executes them, and revises the plan when reality disagrees.

Why it helps: the plan is inspectable. You can read it before anything runs, which is the strongest argument for this pattern.

What goes wrong: everything downstream of a bad plan is wasted. Step seven fails and without checkpoints you restart from step one. If the model is allowed to revise its plan freely it can wander away from what you asked for, so bound how many times it may replan.

The part that separates a demo from a system

Once a task has multiple steps, the interesting engineering stops being about prompts.

Checkpoint after each step. Save what has been completed. If step six fails, resume at six.

Retry the step, not the job. One flaky API call should cost you seconds, not the whole run.

Make actions idempotent. Retries are only safe if repeating an action is safe.

Bound everything. Maximum steps, maximum retries, maximum spend, maximum wall clock time. An unbounded loop with a paid API on the end of it is a bill you will remember.

Trace it. Every model call, every tool call, every input and output. When something goes wrong at step nine of a twenty minute run, you are not reproducing that by re-running it hopefully.

None of this is new. It is the same durable execution problem that background jobs and payment processing solved years ago. The difference is that a job queue fails loudly and an agent fails by producing a confident, wrong, incomplete answer.

Choosing between them

Start at the bottom. Can you write the steps down? Write them down and run them in order. That is not an agent and that is fine.

Do the steps depend on results you cannot predict? Now you need tool use, and possibly planning.

Is there an objective signal to improve against, like tests passing? Add reflection.

Are there genuinely independent chunks of work? Parallelise them.

Is the work large and its shape unknown until you start? Only then reach for orchestrator and workers, and expect to pay for it.

The most common expensive mistake in this area is building the sixth pattern for a problem the first would have solved.