Best Practices for Building LLM Apps

Chapter 6 · Practice 2 of 4

Redact personal data in logs and set a retention period

From Observability and data privacy

Why it matters

Prompts and responses often contain names, emails, phone numbers and whatever users paste in. Logging them in full forever creates a privacy and breach risk.

  1. Redact obvious identifiers before storage.
  2. Restrict who can read raw transcripts.
  3. Delete raw text after a fixed period that matches your privacy policy.

Keep aggregate metrics, such as tokens, latency and cost, longer than raw text. They answer most operational questions without holding anyone's data.

When you need real examples for evals, copy them into the eval set deliberately and anonymise them, rather than mining old logs.