Run a regression eval on every prompt, model or retrieval change

Changing one sentence in a prompt can fix the case you were looking at and break five others. Only a full regression run shows both effects.

Run your eval set in CI whenever any of these change:

• the prompt
• the model id
• temperature
• a tool definition
• a retrieval setting

Compare each run against the last accepted run. Fail the build on a drop beyond a threshold you agree on, and show the cases that flipped from pass to fail so the reviewer sees exactly what broke.

When a provider announces a new model, run the same suite before you switch, rather than trusting general benchmarks.