Stage 2 · Lesson 4 of 7
Validating data before you use it
means automatic checks that data looks the way your code expects before anything is trained or scored: the right columns and types, values in sensible ranges, a limited share of missing values, unique keys and a row count within a normal band.
Write them as code with a library such as Great Expectations, pandera or 's tests, or as plain assertions to start. Run them as the first step of every and fail loudly. An upstream team renaming a column or switching units from rupees to thousands of rupees is far more common than a subtle modelling bug, and it ruins a model silently.
Common trapvalidating training data once and never checking the live data the model receives, where most surprises appear.
A question to check yourself, an AI practice session and 1 cheat sheet entry come with the full roadmap. Unlock
Resources for this stage
Start with
- CourseFreeMade With ML
Goku Mohandas · Advanced
Free: the versioning and data testing lessons, built around one real project