Chapter 6 of 7 · 4 practices · 2 min · 3-question quiz
Observability and data privacy
You need enough logging to debug and evaluate, and enough restraint to protect the people whose data passes through. NIST's Generative AI Profile lists data privacy among the risks generative AI makes worse.
Start: Trace every model and tool call with the same request id4 practices in this chapter
- 1Trace every model and tool call with the same request id"The bot gave a weird answer yesterday" cannot be investigated unless you logged the call. Trace every model and tool call under one request or conversation id.
- 2Redact personal data in logs and set a retention periodPrompts and responses often contain names, emails, phone numbers and whatever users paste in. Logging them in full forever creates a privacy and breach risk.
- 3Send the model only the data the task needsEvery field you put in a prompt can leak through the answer, a log or a provider. Send only what the task needs. Load only the current user's data; Select the fields the task needs rather than whole…
- 4Assign an owner and review AI risks like any other release riskAI features fail in new ways, but they still need the same release discipline as everything else. The NIST AI Risk Management Framework organises the work into four functions: govern, map, measure…
Check yourself on this chapter
3 questions, right here once you are signed in. Your progress is saved as you read.