Best Practices for Building LLM Apps

Chapter 6 · Practice 4 of 4

Assign an owner and review AI risks like any other release risk

From Observability and data privacy

Why it matters

AI features fail in new ways, but they still need the same release discipline as everything else. The NIST AI Risk Management Framework organises the work into four functions: govern, map, measure and manage.

In practice that means:

  • a named owner for each AI feature;
  • a short written list of what could go wrong for users;
  • the evals and monitors that measure those risks;
  • a plan for what happens when one fires.

Revisit the list when you add a tool, a data source or a new user group. The Generative AI Profile, NIST AI 600-1, is a useful prompt for risks you may not have considered.