Stage 2 · Lesson 5 of 9
Gradient descent
is how almost every model learns. Start with some and measure the loss. Work out the partial derivatives of the loss for each weight, which together point uphill.
Take a small step the opposite way, then repeat. The update is w = w - learning_rate × gradient. The sets the step size.
Too small and training crawls; too large and the loss bounces around or blows up. This stage's milestone is trained this way in NumPy. It ties together vectors, matrix multiplication and gradients in about 20 lines.
Stochastic and mini-batch gradient descent use a random slice of the data for each step. That is noisier but far cheaper on large datasets.
Common trapunscaled : if one feature is in thousands and another in fractions, no single learning rate suits both, and training zigzags.
A question to check yourself, an AI practice session and 2 cheat sheet entries come with the full roadmap. Unlock
Resources for this stage
Start with· follow one of these
- CoursePaidMathematics for Machine Learning and Data Science Specialization
DeepLearning.AI · Beginner · Coursera subscription; financial aid available
Paid, with financial aid: linear algebra, calculus and statistics taught visually with Python labs
- CourseFreeKhan Academy Statistics and Probability
Khan Academy · Beginner
Free, complete statistics and probability with practice questions on every topic
Keep handy
Also worth your time
- Mathematics for Machine LearningGo deeper
Free book that ties the maths directly to regression, PCA and classifiers