Chapter 1 · Practice 1 of 4
Write down the prediction, the decision it drives and the cost of each kind of mistake
Why it matters
A model is a means to a decision: approve a loan, flag an email, recommend an article. Before choosing an algorithm, write one paragraph that answers:
- What is predicted, for which unit (a user, a transaction, a day), and at what moment.
- What decision uses the prediction, and who or what makes it.
- What a wrong answer costs. A missed fraud case and a blocked honest customer are different mistakes with different costs.
- What data exists at prediction time. Anything not known at that moment cannot be a feature.
Google's Rules of ML suggest choosing a simple, observable and attributable metric for your first objective (Rule #13) and not overthinking the choice early on (Rule #12). The paragraph above is how you pick it.
Do
- Write the prediction moment down, then check every feature against it
Avoid
- Starting with an algorithm before you know which mistake is more expensive