Best Practices for Building LLM Apps

Chapter 7 · Practice 4 of 4

Repair or reject output that fails validation, then stop

From Failures, fallbacks and misinformation

Why it matters

Output that fails validation is a signal, not something to paper over. When output fails your schema or business rules, you have two good options:

  1. Retry once with the validation error included in the prompt.
  2. Fail cleanly.

Do not loop forever, and do not quietly save partial data.

Either way, the user gets a clear result instead of a broken record in your database.

Track the validation failure rate per prompt version. A rise after a change is a regression your evals should catch next time, so add the failing inputs to the eval set.