Frontend and backend layers

What the browser owns (chat, sign-in, uploads, streaming, history) and what the server owns before any model is called: identity, permissions, validation, business rules, limits and logs.

Chat UI

What: Message list, input box, stop button, retry, copy, citations shown as links and a thumbs up or down on each answer.
Why: The stop and feedback buttons are not decoration: stop saves tokens, and feedback is your cheapest source of eval cases.

Streaming to the browser

Also called SSE, server-sent events

What: The backend sends tokens as they arrive, usually over server-sent events, and the UI appends them to the last message.
Why: First words in under a second feel fast even when the full answer takes ten.

TypeScript

app.post('/chat', requireUser, async (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  for await (const token of answer(req.body, req.user)) {
    res.write(`data: ${JSON.stringify(token)}\n\n`);
  }
  res.end();
});

File uploads

What: The browser uploads to object storage (often with a short-lived signed URL), then tells the backend the file id so a worker can parse and index it.
Why: Large files never pass through your API process, and the backend can check type and size before anything reads them.

Watch out

Trusting the file extension from the browser instead of checking the real type and size on the server.

Auth and chat history in the UI

What: The frontend holds a session or token and loads past conversations from the backend, never from local storage alone.
Why: History tied to the account follows the user across devices and lets you delete it on request.

No model keys in the frontend

What: The browser never calls the model provider directly with your API key; every call goes through your backend.
Why: A key in frontend code is public within minutes, and without the backend you lose auth, limits and logging.

Authentication and authorisation

Also called authn, authz

What: Authentication proves who the user is; authorisation decides which chats, documents and tools they may use. Both run in the API, before the orchestrator.
Why: The model cannot enforce permissions. If a user may not read a document, it must never reach the prompt.

Request validation

What: Check message length, file ids, conversation ownership and allowed options with a schema before doing any work.
Why: It blocks oversized prompts that burn tokens and requests for other users' conversations.

TypeScript

const ChatIn = z.object({
  conversationId: z.string().uuid(),
  message: z.string().min(1).max(4000),
  fileIds: z.array(z.string().uuid()).max(5).default([]),
});

Business logic outside the prompt

What: Plans, quotas, pricing, which documents a team owns and what an action may change are decided in normal code and data, then passed to the model as facts.
Why: Rules in code are testable and cannot be talked out of; rules only in a prompt can.

Rate limits and quotas

What: Limits per user and per IP on requests per minute, plus a daily token or cost budget per user or team.
Why: One script or one curious user can spend a month of budget in an hour without them.

Structured logging

What: Every request logs a request id, user id, route, status, latency, model, tokens and cost as structured fields, not free text.
Why: You can then answer "which users cost the most" or "which route got slow" with one query.

Watch out

Logging full prompts with personal data into a log store everyone can read.

Go deeper