Chapter 7 of 7 · 4 practices · 2 min · 4-question quiz
Failures, fallbacks and misinformation
Model APIs time out, rate limit and occasionally return nonsense. Plan for each case so users get a clear answer instead of a spinner or a confident mistake.
Start: Retry transient errors with backoff and a timeout4 practices in this chapter
- 1Retry transient errors with backoff and a timeoutRate limits and server errors are normal at scale. Retry the ones that can succeed, and fail fast on the ones that cannot.
- 2Plan a fallback for when the model is unavailable or wrongDecide in advance what users see when the model cannot answer, so an outage is a plan, not a scramble. Options include: a second model or provider; a cached or rule-based answer; a reduced feature; a…
- 3Ground factual answers in sources and let the model say it does not knowMisinformation is on the OWASP list because models state wrong facts confidently. For factual features, ground every answer in sources you control.
- 4Repair or reject output that fails validation, then stopOutput that fails validation is a signal, not something to paper over. When output fails your schema or business rules, you have two good options: Retry once with the validation error included in the…
Check yourself on this chapter
4 questions, right here once you are signed in. Your progress is saved as you read.