Stage 2 · Lesson 1 of 7
Why Git alone is not enough
Git is built for text files that change line by line. Datasets and model files are large binary files that change completely, and putting them in Git makes the repository slow and huge. Yet a model is the product of code and data together, so versioning only the code leaves half the story untold.
If data/train.csv is overwritten in place, every past experiment becomes impossible to reproduce, and nobody can say which data trained the live model. The answer is to keep large files in (cloud file storage such as Amazon S3) and put small pointer files, which record a hash (a fingerprint of the content), in Git.
Common trapoverwriting data in place because it is quicker, which silently breaks for every model trained before.
A question to check yourself, an AI practice session and 2 cheat sheet entries come with the full roadmap. Unlock
Resources for this stage
Start with
- CourseFreeMade With ML
Goku Mohandas · Advanced
Free: the versioning and data testing lessons, built around one real project